Re: [openstack-dev] [Savanna] Spark plugin status

2014-01-09 Thread Matthew Farrellee
This is definitely great news! +2 to the things Sergey mentioned below. Additionally, will you fill out the blueprint or wiki w/ details that will help others write integration tests for your plugin? And, did you integrate (or have plans to integrate) Spark into the EDP workflows in Horizon?

[openstack-dev] [savanna] savannaclient v2 api

2014-01-14 Thread Matthew Farrellee
https://blueprints.launchpad.net/savanna/+spec/v2-api I've finished a review of the v1.0 and v1.1 APIs with an eye to making them more consistent and RESTful. Please use this thread to comment on my suggestions for v1.0 & v1.1, or to make further suggestions. Best, matt _

Re: [openstack-dev] [savanna] client release 0.4.1

2014-01-14 Thread Matthew Farrellee
On 01/13/2014 02:27 PM, Sergey Lukjanov wrote: Hi folks, I'm planning to release python-savannaclient Jan 14/15 due to the number of important fixes and improvements including, for example, basic impl of CLI. This changes are needed for updating savanna-dashboard, integration tests, for adding s

[openstack-dev] [savanna] paramiko requirement of >= 1.9.0?

2014-01-18 Thread Matthew Farrellee
jon, please confirm a suspicion of mine. the neutron-private-net-provisioning bp impl added a sock= parameter to the ssh.connect call in remote.py (https://github.com/openstack/savanna/commit/9afb5f60). we currently require paramiko >= 1.8.0, but it looks like the sock param was only added

Re: [openstack-dev] [savanna] savannaclient v2 api

2014-01-19 Thread Matthew Farrellee
remove all its related info from savanna-db. DELETE removes it completely. [1] http://docs.openstack.org/developer/savanna/devref/plugin.spi.html#convert-config-plugin-name-version-template-name-cluster-template-create [2] https://blueprints.launchpad.net/savanna/+spec/modifying-cluster-template [3] https://b

Re: [openstack-dev] [savanna] savannaclient v2 api

2014-01-19 Thread Matthew Farrellee
doesn't remove all its related info from savanna-db. DELETE removes it completely. [1] http://docs.openstack.org/developer/savanna/devref/plugin.spi.html#convert-config-plugin-name-version-template-name-cluster-template-create [2] https://blueprints.launchpad.net/savanna/

Re: [openstack-dev] [Savanna] Spark plugin status

2014-01-19 Thread Matthew Farrellee
On 01/10/2014 04:05 AM, Daniele Venzano wrote: On 01/09/14 19:12, Matthew Farrellee wrote: This is definitely great news! +2 to the things Sergey mentioned below. Additionally, will you fill out the blueprint or wiki w/ details that will help others write integration tests for your plugin

Re: [openstack-dev] [savanna] savannaclient v2 api

2014-01-20 Thread Matthew Farrellee
(inline, trying to make this readable by a text-only mail client that doesn't use tabs to indicate quoting) On 01/20/2014 02:50 AM, Andrey Lazarev wrote: -- FIX - @rest.get('/jobs/config-hints/__') - should move to GET /plugins//<__plugin_version>, similar to

Re: [openstack-dev] [savanna] savannaclient v2 api

2014-01-20 Thread Matthew Farrellee
(inline-ish) On 01/20/2014 02:36 AM, Andrey Lazarev wrote: On Sun, Jan 19, 2014 at 7:53 AM, Matthew Farrellee mailto:m...@redhat.com>> wrote: On 01/16/2014 09:19 PM, Andrey Lazarev wrote: REMOVE - @rest.get('/job-executions/<__job_executi

Re: [openstack-dev] [savanna] savannaclient v2 api

2014-01-20 Thread Matthew Farrellee
On 01/20/2014 12:50 PM, Andrey Lazarev wrote: Inlined. On Mon, Jan 20, 2014 at 8:15 AM, Matthew Farrellee mailto:m...@redhat.com>> wrote: (inline, trying to make this readable by a text-only mail client that doesn't use tabs to indicate quoting) On 01/20/2014 02:50

[openstack-dev] [savanna] why swift-internal:// ?

2014-01-23 Thread Matthew Farrellee
trevor, job binaries are stored in swift or an internal savanna db, represented by swift-internal:// and savanna-db:// respectively. why swift-internal:// and not just swift://? fyi, i see mention of a potential future version of savanna w/ swift-external:// best, matt _

Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-23 Thread Matthew Farrellee
T (like data sources). Internal and external swifts are different since we need additional credentials. Thanks, Andrew. On Thu, Jan 23, 2014 at 5:30 PM, Matthew Farrellee mailto:m...@redhat.com>> wrote: trevor, job binaries are stored in swift or an internal savanna db, represented

Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Matthew Farrellee
6:17 PM, Matthew Farrellee mailto:m...@redhat.com>> wrote: what makes it internal vs external? swift-internal needs user & pass swift-external needs user & pass & ?auth url? best, matt On 01/23/2014 08:43 PM, Andrew Lazarev wrote: Matt,

Re: [openstack-dev] [savanna] why swift-internal:// ?

2014-01-24 Thread Matthew Farrellee
y, where the URL determines the treatment. This could easily be extended to look at other indicators. Best, Trev On Fri, 2014-01-24 at 07:50 -0500, Matthew Farrellee wrote: > andrew, > > what about having swift:// which defaults to the configured tenant and

Re: [openstack-dev] [savanna] savannaclient v2 api

2014-01-24 Thread Matthew Farrellee
ov On 20 Jan 2014, at 22:04, Matthew Farrellee wrote: On 01/20/2014 12:50 PM, Andrey Lazarev wrote: Inlined. On Mon, Jan 20, 2014 at 8:15 AM, Matthew Farrellee mailto:m...@redhat.com>> wrote: (inline, trying to make this readable by a text-only mail client that doesn't use tab

Re: [openstack-dev] [savanna] Savanna 2014.1.b2 (Icehouse-2) dev milestone available

2014-01-27 Thread Matthew Farrellee
On 01/23/2014 11:59 AM, Sergey Lukjanov wrote: Hi folks, the second development milestone of Icehouse cycle is now available for Savanna. Here is a list of new features and fixed bug: https://launchpad.net/savanna/+milestone/icehouse-2 and here you can find tarballs to download it: http://ta

Re: [openstack-dev] [savanna] Choosing provisioning engine during cluster launch

2014-01-30 Thread Matthew Farrellee
i imagine this is something that can be useful in a development and testing environment, especially during the transition period from direct to heat. so having the ability is not unreasonable, but i wouldn't expose it to users via the dashboard (maybe not even directly in the cli) generally i

[openstack-dev] [savanna] plugin version or hadoop version?

2014-02-17 Thread Matthew Farrellee
$ savanna plugins-list +-+--+---+ | name| versions | title | +-+--+---+ | vanilla | 1.2.1| Vanilla Apache Hadoop | | hdp | 1.3.2| Hortonworks Data Platform | +-+-

Re: [openstack-dev] [savanna] plugin version or hadoop version?

2014-02-17 Thread Matthew Farrellee
IDH uses version of IDH distro and there is no direct mapping between distro version and hadoop version. E.g. IDH 2.5.1 works with apache hadoop 1.0.3. I suggest to call the field as just 'version' everywhere and assume this version as plugin specific property. A

Re: [openstack-dev] [savanna] Nominate Andrew Lazarew for savanna-core

2014-02-21 Thread Matthew Farrellee
On 02/19/2014 05:40 PM, Sergey Lukjanov wrote: Hey folks, I'd like to nominate Andrew Lazarew (alazarev) for savanna-core. He is among the top reviewers of Savanna subprojects. Andrew is working on Savanna full time since September 2013 and is very familiar with current codebase. His code contr

Re: [openstack-dev] [savanna] client 0.5.0 release

2014-03-01 Thread Matthew Farrellee
builds for fedora rawhide and epel6 - rawhide - http://koji.fedoraproject.org/koji/taskinfo?taskID=6582748 epel6 - http://koji.fedoraproject.org/koji/taskinfo?taskID=6582798 On 02/25/2014 03:50 PM, Sergey Lukjanov wrote: Hi folks, I'm glad to announce that python-savannaclient v0.5.0 released!

Re: [openstack-dev] [sahara] Asia friendly IRC meeting time

2014-11-25 Thread Matthew Farrellee
On 11/25/2014 02:37 AM, Zhidong Yu wrote: Current meeting time: 18:00UTC: Moscow (9pm)China(2am) US West(10am) My proposal: 18:00UTC: Moscow (9pm)China(2am) US West(10am) 00:00UTC: Moscow (3am)China(8am) US West(4pm) fyi, a number of us are UW East (US W

Re: [openstack-dev] [sahara] Discussion on cm_api for global requirement

2014-11-25 Thread Matthew Farrellee
On 11/24/2014 01:22 PM, Trevor McKay wrote: Hello all, at our last Sahara IRC meeting we started discussing whether or not to add a global requirement for cm_api.py https://review.openstack.org/#/c/130153/ One issue (but not the only issue) is that cm_api is not packaged for Fedora, Cento

Re: [openstack-dev] [sahara] team meeting Nov 27 1800 UTC

2014-11-26 Thread Matthew Farrellee
On 11/26/2014 01:10 PM, Sergey Lukjanov wrote: Hi folks, We'll be having the Sahara team meeting as usual in #openstack-meeting-alt channel. Agenda: https://wiki.openstack.org/wiki/Meetings/SaharaAgenda#Next_meetings http://www.timeanddate.com/worldclock/fixedtime.html?msg=Sahara+Meeting&iso=2

Re: [openstack-dev] [sahara] cancel next team meeting May 1

2014-04-25 Thread Matthew Farrellee
On 04/25/2014 07:23 AM, Sergey Lukjanov wrote: Hey folks, May 1 is a non-working day in Russia and I'm starting traveling next day, so, I'll not be able to chair it. So, I'm proposing to cancel this meeting. Any thoughts/objections? if folks have topics they'd like to cover, use the mailing

Re: [openstack-dev] [sahara] Design Summit Sessions

2014-04-25 Thread Matthew Farrellee
On 04/24/2014 10:51 AM, Sergey Lukjanov wrote: Hey folks, I've pushed the draft schedule for Sahara sessions on ATL design summit. The description isn't fully completed, I'm working on it. I'll do it till the end of week and add an etherpad to each session. Sahara folks, please, take a look on

Re: [openstack-dev] [sahara] Design Summit Sessions

2014-05-05 Thread Matthew Farrellee
nough input on other areas. IMO we should discuss first what we need to have and then how it'll looks like. On Fri, Apr 25, 2014 at 9:29 PM, Matthew Farrellee wrote: On 04/24/2014 10:51 AM, Sergey Lukjanov wrote: Hey folks, I've pushed the draft schedule for Sahara sessions on ATL des

Re: [openstack-dev] [sahara] Nominate Trevor McKay for sahara-core

2014-05-15 Thread Matthew Farrellee
On 05/12/2014 05:31 PM, Sergey Lukjanov wrote: Hey folks, I'd like to nominate Trevor McKay (tmckay) for sahara-core. He is among the top reviewers of Sahara subprojects. Trevor is working on Sahara full time since summer 2013 and is very familiar with current codebase. His code contributions a

Re: [openstack-dev] [sahara] summit wrap-up: subprojects

2014-05-28 Thread Matthew Farrellee
On 05/28/2014 12:02 PM, Sergey Lukjanov wrote: Hey folks, it's a small wrap-up for the topic "Sahara subprojects releasing and versioning" that was discussed partially on summit and requires some more discussions. You can find details in [0]. common We'll include only one tarball for sahara

Re: [openstack-dev] [sahara] summit wrap-up: backward compat

2014-05-28 Thread Matthew Farrellee
On 05/28/2014 09:14 AM, Sergey Lukjanov wrote: Hey folks, it's a small wrap-up for the two topics "Sahara backward compat" and " "Hadoop cluster backward compatibility", both were discussed on design summit, etherpad [0] contains info about them. There are some open questions listed in the end o

Re: [openstack-dev] [sahara] summit wrap-up: backward compat

2014-05-28 Thread Matthew Farrellee
On 05/28/2014 01:59 PM, Michael McCune wrote: - Original Message - Open questions 1. How should we handle addition of new functionality to the API, should we bump minor version and just add new endpoints? I think we should not include the minor revision number in the url. Looking a

Re: [openstack-dev] [sahara] summit wrap-up: backward compat

2014-05-28 Thread Matthew Farrellee
On 05/28/2014 03:50 PM, Andrew Lazarev wrote: for juno we should just have a v1 api (there can still be a v1.1 endpoint, but it should be deprecated), and maybe a v2 api +1 any semantic changes require new major version number +1 api should only have a major number (no 1.1 or 2

Re: [openstack-dev] Mahout-as-a-service [sahara]

2014-05-29 Thread Matthew Farrellee
On 05/28/2014 12:37 PM, Dat Tran wrote: Hi everyone, I have a idea for new project: Mahout-as-a-service. Main idea of this project: - Install OpenStack - Deploying OpenStack Sahara source - Deploying Mahout on Sahara OpenStack system. - Construction of the API. Through web or mobile interface,

Re: [openstack-dev] [sahara] summit wrap-up: subprojects

2014-05-29 Thread Matthew Farrellee
On 05/29/2014 07:23 AM, Alexander Ignatov wrote: On 28 May 2014, at 20:02, Sergey Lukjanov wrote: sahara-image-elements We're agreed that some common parts should be merged into the diskimage-builder repo (like java support, ssh, etc.). The main issue of keeping -image-elements separated is

Re: [openstack-dev] [sahara] summit wrap-up: subprojects

2014-05-29 Thread Matthew Farrellee
On 05/29/2014 09:59 AM, Trevor McKay wrote: below, sahara-extra sahara-extra Keep it as is, no need to stop releasing, because we're not publishing anything to pypi. No real need for tags. Even if we keep the repo for now, I think we could simplify a little bit. The edp-examples could be

Re: [openstack-dev] [sahara] summit wrap-up: subprojects

2014-05-29 Thread Matthew Farrellee
provide us good flexibility - for example, we'll be able to specify to use "latest release dib". - all scripts and dib will not be installed with sahara (50/50) On Thu, May 29, 2014 at 3:57 PM, Matthew Farrellee wrote: On 05/29/2014 07:23 AM, Alexander Ignatov wrote: On 28 May 2014, at

Re: [openstack-dev] [sahara] summit wrap-up: subprojects

2014-05-29 Thread Matthew Farrellee
On 05/29/2014 10:15 AM, Michael McCune wrote: - Original Message - Re sahara-image-elements we found a bunch of issues that we should solve and that's why I think that keeping current releasing is still the best option. - we should test it better and depend on stable diskimage-builder

Re: [openstack-dev] [sahara] summit wrap-up: backward compat

2014-05-29 Thread Matthew Farrellee
On 05/29/2014 10:22 AM, Sergey Lukjanov wrote: So, it looks like we have an agreement on all question. There is only one technical question - keeping release images means that we need to keep the whole matrix of images: plugin X version X OSy [X root-passwdord]. I'll take a look on total size of

Re: [openstack-dev] [sahara] Sahara (ex. Savanna) project renaming process [savanna]

2014-03-08 Thread Matthew Farrellee
rint is [1], it'll depend on all other blueprints and it's currently consists of general renaming tasks. Current plan is to assign each subproject blueprint to volunteer. Please, contact me and Matthew Farrellee if you'd like to take the renaming bp. Please, share your ideas/suggestio

[openstack-dev] [sahara] context module update

2014-03-10 Thread Matthew Farrellee
andrew, chad, trevor, please take another look at https://review.openstack.org/#/c/78208/ best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [sahara] team meeting minutes March 13 [savanna]

2014-03-13 Thread Matthew Farrellee
On 03/13/2014 03:24 PM, Jay Pipes wrote: On Thu, 2014-03-13 at 23:13 +0400, Sergey Lukjanov wrote: Thanks everyone who have joined Savanna meeting. You mean Sahara? :P -jay sergey now has to put some bitcoins in the jar... ___ OpenStack-dev mail

Re: [openstack-dev] [savanna] Savanna 2014.1.b3 (Icehouse-3) dev milestone available

2014-03-14 Thread Matthew Farrellee
On 03/06/2014 04:00 PM, Sergey Lukjanov wrote: Hi folks, the third development milestone of Icehouse cycle is now available for Savanna. Here is a list of new features and fixed bug: https://launchpad.net/savanna/+milestone/icehouse-3 and here you can find tarballs to download it: http://tar

[openstack-dev] [sahara] Re: Change in openstack/sahara-image-elements[master]: Add inject hadoop native libs to vanilla hadoop 2

2014-04-21 Thread Matthew Farrellee
On 04/10/2014 08:25 AM, Dmitry Mescheryakov (Code Review) wrote: Dmitry Mescheryakov has posted comments on this change. Change subject: Add inject hadoop native libs to vanilla hadoop 2 .. Patch Set 2: Looks good to me, but s

Re: [openstack-dev] [horizon][sahara] Merging Sahara-UI Dashboard code into horizon

2014-04-21 Thread Matthew Farrellee
On 04/17/2014 03:06 PM, Chad Roberts wrote: Per blueprint https://blueprints.launchpad.net/horizon/+spec/merge-sahara-dashboard we are merging the Sahara Dashboard UI code into the Horizon code base. Over the last week, I have been working on making this merge happen and along the way some i

[openstack-dev] [sahara] apache hive no longer distributing 0.11.0 - image-elements don't build

2014-04-22 Thread Matthew Farrellee
https://sahara.mirantis.com/logs/97/86997/4/check/diskimage-integration-ubuntu/d37fe82/console.html https://sahara.mirantis.com/logs/97/86997/4/check/diskimage-integration-fedora/da83f57/console.html https://sahara.mirantis.com/logs/97/86997/4/check/diskimage-integration-centos/27d14d1/console.htm

Re: [openstack-dev] [sahara] apache hive no longer distributing 0.11.0 - image-elements don't build

2014-04-22 Thread Matthew Farrellee
On 04/22/2014 08:21 AM, Matthew Farrellee wrote: https://sahara.mirantis.com/logs/97/86997/4/check/diskimage-integration-ubuntu/d37fe82/console.html https://sahara.mirantis.com/logs/97/86997/4/check/diskimage-integration-fedora/da83f57/console.html https://sahara.mirantis.com/logs/97/86997/4

Re: [openstack-dev] [Sahara] Verbosity of Sahara overview image

2014-09-27 Thread Matthew Farrellee
On 09/26/2014 02:27 PM, Sharan Kumar M wrote: Hi all, I am trying to modify the diagram in http://docs.openstack.org/developer/sahara/overview.html so that it syncs with the contents. In the diagram, is it nice to mark the connections between the openstack components like, Nova with Cinder, Nova

Re: [openstack-dev] [sahara] Nominate Sergey Reshetniak to sahara-core

2014-11-11 Thread Matthew Farrellee
On 11/11/2014 12:35 PM, Sergey Lukjanov wrote: Hi folks, I'd like to propose Sergey to sahara-core. He's made a lot of work on different parts of Sahara and he has a very good knowledge of codebase, especially in plugins area. Sergey has been consistently giving us very well thought out and con

Re: [openstack-dev] [sahara] Nominate Michael McCune to sahara-core

2014-11-11 Thread Matthew Farrellee
On 11/11/2014 12:37 PM, Sergey Lukjanov wrote: Hi folks, I'd like to propose Michael McCune to sahara-core. He has a good knowledge of codebase and implemented important features such as Swift auth using trusts. Mike has been consistently giving us very well thought out and constructive reviews

Re: [openstack-dev] [Sahara] Error config?

2014-07-09 Thread Matthew Farrellee
On 07/09/2014 07:24 AM, Dat Tran wrote: Hi all, I'm install sahara follow the instructions below: http://docs.openstack.org/developer/sahara/devref/quickstart.html With script install sahara as follows: https://docs.google.com/document/d/18j4zR4ENibxA-WBVkryzkMFU9PRiuop3U2pEU6JQrOk/edit But wh

Re: [openstack-dev] [Sahara] Error config?

2014-07-09 Thread Matthew Farrellee
On 07/09/2014 11:26 AM, Dat Tran wrote: Thank matt! But, I have a error: ERROR: Could not find Sahara endpoint in catalog you do have to setup sahara to be in the keystone service catalog, something like step 4 from, docs.openstack.org/developer/sahara/horizon/installation.guide.html#sahara

Re: [openstack-dev] [Sahara] Error config?

2014-07-09 Thread Matthew Farrellee
you should use the sahara cli for all functions. the documentation needs to be updated. in the meantime you can adapt from the "savanna cli" discussed in - http://spinningmatt.wordpress.com/2014/01/29/hadoop-on-openstack-with-a-cli-creating-a-cluster/ best, matt On 07/09/2014 01:58 PM, Trev

Re: [openstack-dev] [savanna] Program name and Mission statement

2013-09-11 Thread Matthew Farrellee
You caught me trying to be fancy! On 09/10/2013 03:54 PM, Glen Campbell wrote: "performant" isn't a word. Or, if it is, it means "having performance." I think you mean "high-performance." On Tue, Sep 10, 2013 at 8:47 AM, Matthew Farrellee mailto:m...

Re: [openstack-dev] [savanna] Program name and Mission statement

2013-09-11 Thread Matthew Farrellee
That sounds quite good. Best, matt On 09/11/2013 11:42 AM, Andrei Savu wrote: +1 I guess this will also clarify how Savanna relates to other projects like OpenStack Trove. -- Andrei Savu On Wed, Sep 11, 2013 at 5:16 PM, Mike Spreitzer mailto:mspre...@us.ibm.com>> wrote: > To provide

Re: [openstack-dev] [savanna] Program name and Mission statement

2013-09-16 Thread Matthew Farrellee
gt; wrote: "performant" isn't a word. Or, if it is, it means "having performance." I think you mean "high-performance." On Tue, Sep 10, 2013 at 8:47 AM, Matthew Farrellee mailto:m...@redhat.com>> wrote:

Re: [openstack-dev] [savanna] neutron and private networks

2013-10-03 Thread Matthew Farrellee
On 10/03/2013 11:21 AM, Jon Maron wrote: Hi, I'd like to raise an issue in the hopes of opening some discussion on the IRC chat later today: We see a critical requirement to support the creation of a savanna cluster with neutron networking while leveraging a private network (i.e. without the as

[openstack-dev] [savanna] What's the recipe to build Oozie-4.0.0.tar.gz?

2013-10-22 Thread Matthew Farrellee
Having diskimage-create.sh is a great addition for the Savanna user community. It greatly simplifies the image building process (using DIB for those of you not familiar), making it repeatable and giving everyone a hope of debugging issues. One thing it does is install oozie. It pulls oozie fro

Re: [openstack-dev] [savanna] Loading of savanna.conf properties

2013-11-10 Thread Matthew Farrellee
Jon, I ran into this issue and we discussed on IRC a couple weeks ago. At least for the Vanilla plugin you have to supply a instance->infra routable addr/ip for os_auth_host in your savanna.conf. https://bugs.launchpad.net/savanna/+bug/1244309 Ultimately I'd like to add some code that uses t

Re: [openstack-dev] [Savanna] Release 0.3 retrospective

2013-11-14 Thread Matthew Farrellee
On 10/31/2013 01:58 PM, Alexander Ignatov wrote: Hi, Here is the wiki page with Savanna release 0.3 retrospective: https://wiki.openstack.org/wiki/Savanna/Release_0.3_Retrospective Thanks everyone who sent your opinions. If someone wants to add more thoughts you are welcome to edit above page!

Re: [openstack-dev] [Savanna] DiskBuilder / savanna-image-elements

2013-11-15 Thread Matthew Farrellee
On 11/15/2013 12:14 PM, Clint Byrum wrote: Excerpts from Erik Bergenholtz's message of 2013-11-15 08:20:36 -0800: Team - We’d like to move our disk creation mechanism over to using DiskBuilder so that users can build (and modify) their own VM images. We’d like to piggy back off of the existin

[openstack-dev] [savanna-all] merging savanna-extra elements

2013-07-10 Thread Matthew Farrellee
Ivan, $ tree elements/hadoop/install.d elements/hadoop/install.d |-- 70-setup-java |-- 80-setup-hadoop `-- 90-setup-ssh 0 directories, 3 files $ tree elements/hadoop_fedora/post-install.d elements/hadoop_fedora/post-install.d |-- 11-setup-java |-- 12-setup-hadoop `-- 13-connection-setup 0 direct

Re: [openstack-dev] [Savanna-all] [savanna-all] merging savanna-extra elements

2013-07-11 Thread Matthew Farrellee
Ivan 2013/7/10 Matthew Farrellee mailto:m...@redhat.com>> Ivan, $ tree elements/hadoop/install.d elements/hadoop/install.d |-- 70-setup-java |-- 80-setup-hadoop `-- 90-setup-ssh 0 directories, 3 files $ tree elements/hadoop_fedora/post-__install.d

[openstack-dev] [Savanna-all] Cluster scaling discussion

2013-07-12 Thread Matthew Farrellee
A comment on how you go about this. I believe you've already run into issues w/ using the start/stop-*.sh scripts as a foundation for this feature. Long term I believe that an active cluster need not mean every instance is up and running. The core infrastructure must be (ambari + jt + nn), an

Re: [openstack-dev] [Savanna] Merge Fedora and Ubuntu DIB elements

2013-07-15 Thread Matthew Farrellee
On 07/15/2013 07:34 AM, Ivan Berezovskiy wrote: Matt, I've sent a comment at https://review.openstack.org/#/c/36690/ . So if I believe the issue is a hadoop.rpm that is out of spec w/ fedora. For instance, it claims to own things like /usr. It also doesn't have a proper post-install to hand

Re: [openstack-dev] [savanna]error while accessing Savanna UI

2013-07-15 Thread Matthew Farrellee
On 07/15/2013 08:45 AM, Arindam Choudhury wrote: Hi, I did: git clone https://github.com/stackforge/savanna-dashboard.git cd savanna-dashboard python setup.py install pip show savannadashboard --- Name: savannadashboard Version: 0.2.rc2 Location: /usr/lib/python2.6/site-packages/savannadashb

Re: [openstack-dev] [Savanna] Savanna 0.2 is released!

2013-07-15 Thread Matthew Farrellee
Well done all, this release was no small effort! Especially, great collaboration and use of tools available from the OpenStack community. Best, matt On 07/15/2013 06:14 PM, Sergey Lukjanov wrote: Hello everyone, I'm very happy to announce the immediate release of Savanna 0.2. This release

Re: [openstack-dev] [savanna] scalable architecture

2013-07-24 Thread Matthew Farrellee
On 07/23/2013 12:32 PM, Sergey Lukjanov wrote: Hi evereyone, We’ve started working on upgrading Savanna architecture in version 0.3 to make it horizontally scalable. The most part of information is in the wiki page - https://wiki.openstack.org/wiki/Savanna/NextGenArchitecture. Additionally the

[openstack-dev] [savanna] Savanna PTL election proposal

2013-08-12 Thread Matthew Farrellee
This is a request for feedback from the community. The Savanna project has been operating with a benevolent dictator. It wants to upgrade to an elected PTL. There's no set process for a project that isn't incubating or integrated. Our goal is to mirror the standard election process as closel

Re: [openstack-dev] [savanna] Savanna PTL election proposal

2013-08-13 Thread Matthew Farrellee
On 08/13/2013 07:29 AM, Thierry Carrez wrote: Matthew Farrellee wrote: 2. Candidate nomination - a. anyone can list names in https://etherpad.openstack.org/savanna-ptl-candidates-0 b. anyone mentioned during this week's IRC meeting c. both (a) and (b) - Current direction is

Re: [openstack-dev] OpenStack Savanna issue

2013-08-13 Thread Matthew Farrellee
On 08/04/2013 12:01 PM, Linus Nova wrote: HI, I installed OpenStack Savanna in OpenStack Grizzely release. As you can see in savanna.log, the savanna-api start and operates correctly. When I launch the cluster, the VMs start correctly but soon after they are removed as shown in the log file. D

[openstack-dev] [savanna] tarballs of savanna-extra

2013-08-19 Thread Matthew Farrellee
Will someone setup a tarballs.os.o release of savanna-extra's master (https://github.com/stackforge/savanna-extra), and make sure it gets an official release for 0.3? Best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http:

Re: [openstack-dev] [savanna] tarballs of savanna-extra

2013-08-20 Thread Matthew Farrellee
looks like that we should not package all of them to one heterogeneous tarball. Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. On Aug 20, 2013, at 0:25, Matthew Farrellee wrote: Will someone setup a tarballs.os.o release of savanna-extra's master (https://github

Re: [openstack-dev] [savanna] tarballs of savanna-extra

2013-08-21 Thread Matthew Farrellee
e are more pros than cons, so, we can do it. Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. On Aug 20, 2013, at 16:31, Matthew Farrellee wrote: Is there a downside to having it? A positive is it gives a snapshot of everything for each release. I'm not at fan o

[openstack-dev] [Savanna] PTL Election result: Sergey Lukjanov wins

2013-08-22 Thread Matthew Farrellee
Recorded in https://wiki.openstack.org/wiki/Savanna/PTL Results: Sergey Lukjanov (14), None (0) Electorate: 20 voters (70% participation) His term is effective immediately (22 Aug 2013) until the OpenStack Icehouse release. ___ OpenStack-dev mailing

[openstack-dev] [savanna] Fwd: Savanna - Hadoop on OpenStack

2013-08-26 Thread Matthew Farrellee
FYI Original Message Subject: Savanna - Hadoop on OpenStack Date: Mon, 26 Aug 2013 14:29:44 -0400 From: Matthew Farrellee Reply-To: Fedora Big Data SIG To: Fedora Big Data SIG Hello Big Data SIG folks, If you aren't familiar, Savanna is an OpenStack project that pro

[openstack-dev] [savanna] savanna-extra and hadoop-patch

2013-08-26 Thread Matthew Farrellee
https://review.openstack.org/#/c/42926/ I didn't get back to this on Friday and it got merged this morning, so here's my feedback. The savanna-extra repository now appears to hold (a) DIB image elements as well as (b) the source for the Swift backed HCFS (Hadoop Compatible File System) imple

Re: [openstack-dev] [savanna] savanna-extra and hadoop-patch

2013-08-27 Thread Matthew Farrellee
b.com/gluster/hadoop-glusterfs Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. On Aug 27, 2013, at 5:18, Matthew Farrellee wrote: https://review.openstack.org/#/c/42926/ I didn't get back to this on Friday and it got merged this morning, so here's my feedback. The

Re: [openstack-dev] [savanna] savanna-extra and hadoop-patch

2013-08-27 Thread Matthew Farrellee
doop folks don't intend to include this patch into Hadoop 1.x Ruslan On Tuesday, August 27, 2013 at 2:41 PM, Matthew Farrellee wrote: Howdy Ivan, FYI, https://issues.apache.org/jira/browse/HADOOP-8545 is currently targeting 1.2.0 and 2.0.3-alpha. And the code (HADOOP-8545-034.patch) app

Re: [openstack-dev] [savanna] migration to pbr completed

2013-08-28 Thread Matthew Farrellee
On 08/27/2013 04:46 PM, Sergey Lukjanov wrote: Hi folks, migration of all Savanna sub projects to pbr has been completed. Please, inform us and/or create bugs for all packaging-related issues. Thanks. Sincerely yours, Sergey Lukjanov Savanna Technical Lead Mirantis Inc. Thanks for pushing t

[openstack-dev] [savanna] Fwd: Change in stackforge/savanna-extra[master]: Add diskimage-creating script, elements for mirrors

2013-09-03 Thread Matthew Farrellee
rors Date: Thu, 29 Aug 2013 14:37:36 + From: Ivan Berezovskiy (Code Review) Reply-To: iberezovs...@mirantis.com CC: Sergey Lukjanov ,Dmitry Mescheryakov ,Nadya Privalova , Matthew Farrellee Ivan Berezovskiy has posted comments on this change. Change subject: Add dis

Re: [openstack-dev] [savanna] Fwd: Change in stackforge/savanna-extra[master]: Add diskimage-creating script, elements for mirrors

2013-09-04 Thread Matthew Farrellee
mage-create" should not be run with itself too. Thanks, Ivan. 2013/9/3 Matthew Farrellee mailto:m...@redhat.com>> Long weekend here in the US, so I didn't get a chance to comment before this was merged, so... Re Oozie - How did you create the oozie-3.3.2.tar.gz?

Re: [openstack-dev] [Savanna] Guidance for adding a new plugin (CDH)

2013-09-04 Thread Matthew Farrellee
On 09/04/2013 04:06 PM, Andrei Savu wrote: Hi guys - I have just started to play with Savanna a few days ago - I'm still going through the code. Next week I want to start to work on a plugin that will deploy CDH using Cloudera Manager. What process should I follow? I'm new to launchpad / Gerrit

Re: [openstack-dev] [Savanna] Guidance for adding a new plugin (CDH)

2013-09-06 Thread Matthew Farrellee
d the following blueprint (check the full specification for more details): https://blueprints.launchpad.net/savanna/+spec/cdh-plugin I'm now working on some code to get early feedback. Regards, -- Andrei Savu / axemblr.com <http://axemblr.com/> On Wed, Sep 4, 2013 at 11:35 PM,

Re: [openstack-dev] [Savanna] Guidance for adding a new plugin (CDH)

2013-09-10 Thread Matthew Farrellee
On 09/06/2013 03:00 PM, Andrei Savu wrote: On Fri, Sep 6, 2013 at 3:00 PM, Matthew Farrellee mailto:m...@redhat.com>> wrote: Once done, what will the procedure be for me to verify it without becoming a Cloudera customer? What will the limitations be to its use, if any, if I&#

Re: [openstack-dev] [savanna] Program name and Mission statement

2013-09-10 Thread Matthew Farrellee
Rough cut - Program: OpenStack Data Processing Mission: To provide the OpenStack community with an open, cutting edge, performant and scalable data processing stack and associated management interfaces. On 09/10/2013 09:26 AM, Sergey Lukjanov wrote: It sounds too broad IMO. Looks like we nee

Re: [openstack-dev] [sahara] Proposing Ethan Gafford for the core reviewer team

2015-08-13 Thread Matthew Farrellee
On 08/13/2015 10:56 AM, Sergey Lukjanov wrote: Hi folks, I'd like to propose Ethan Gafford as a member of the Sahara core reviewer team. Ethan contributing to Sahara for a long time and doing a great job on reviewing and improving Sahara. Here are the statistics for reviews [0][1][2] and commit

Re: [openstack-dev] [sahara] Proposing Vitaly Gridnev to core reviewer team

2015-10-13 Thread Matthew Farrellee
+1! On 10/12/2015 07:19 AM, Sergey Lukjanov wrote: Hi folks, I'd like to propose Vitaly Gridnev as a member of the Sahara core reviewer team. Vitaly contributing to Sahara for a long time and doing a great job on reviewing and improving Sahara. Here are the statistics for reviews [0][1][2] and