Re: [VOTE] Release Apache Aurora 0.18.1 RC1

2017-10-30 Thread Joshua Cohen
ise work against 0.18.1-rc1, but no > guarantees. > > On Sun, Oct 29, 2017 at 3:20 PM, Joshua Cohen wrote: > > > I'm trying to run the verify-release-candidate script, but getting a 404 > > for the sha512 signature? > > > > + download_rc_file apache-aurora

Re: [VOTE] Release Apache Aurora 0.18.1 RC1

2017-10-29 Thread Joshua Cohen
I'm trying to run the verify-release-candidate script, but getting a 404 for the sha512 signature? + download_rc_file apache-aurora-0.18.1-rc1.tar.gz.sha512 + download_dist_file 0.18.1-rc1/apache-aurora-0.18.1-rc1.tar.gz.sha512 + curl -f -O https://dist.apache.org/repos/dist/dev/aurora/0.18.1-rc1/

Re: Future of storage in Aurora

2017-10-03 Thread Joshua Cohen
due to (b) > > > > e) schema backwards compatibility guarantees due to persistence- > > > > friendly > > > > migration-scripts > > > > f) straightforward normalization to facilitate sharing of > > > > otherwise-redundant state (I.e. TaskConfig) >

Re: Make drain MAX_STATUS_WAIT configurable

2017-09-05 Thread Joshua Cohen
echoing my +1 from Slack here. On Tue, Sep 5, 2017 at 2:05 PM, Erb, Stephan wrote: > +1 > > On 05.09.17, 19:17, "David McLaughlin" wrote: > > +1 > > On Tue, Sep 5, 2017 at 10:09 AM, Mauricio Garavaglia < > mauriciogaravag...@gmail.com> wrote: > > > Hi folks, > > > > The

Re: Redesign of the Aurora UI

2017-08-18 Thread Joshua Cohen
; > > > > As for the development strategy, I'm in favor of the incremental > approach > > that posts one page at a time. The main benefit is that we are > educating > > the developers while iterating on it, and this will improve the > adoption >

Re: Reducing Failover Time by Eagerly Reading/Replaying Log in Followers

2017-07-26 Thread Joshua Cohen
I'm supportive of this plan in general, but I agree with Bill that the proposal is somewhat lacking in detail. Specifically I think it would be good to document in a less broad sense what the proposed changes to Mesos and Aurora actually entail. Right now it's a little bit hand-wavy ;). On Wed, Ju

Re: [VOTE] Release Apache Aurora 0.18.x packages

2017-07-24 Thread Joshua Cohen
+1, everything looks good to me! On Sat, Jul 22, 2017 at 1:36 AM, Erb, Stephan wrote: > +1 > > The verification scripts for all distributions in the test repository have > passed for me. > > > On 19.07.17, 01:58, "Santhosh Kumar Shanmugham" > > wrote: > > I missed to update one of the bint

Re: Redesign of the Aurora UI

2017-07-19 Thread Joshua Cohen
t allows us to delete the vast > majority of our 3rdparty assets that we had to copy into the repo. We can > most likely extend this to remove all of them, including our CSS frameworks > like Bootstrap. > 2) ReactJS was proposed before (by Joshua Cohen) and ironically I was one > of

Re: Future of storage in Aurora

2017-03-30 Thread Joshua Cohen
My understanding of the H2-backed stores is that at least part of the original rationale behind them was that they were meant to be an interim point on the way to external SQL-backed stores which should theoretically provide significant benefits w.r.t. to GC (obviously unproven, especially at scale

Re: Dynamic Reservations

2017-03-13 Thread Joshua Cohen
Dmitriy, There's a fair number of comments both here and on the doc. Will you have time to respond to these so we can find a path forward? Cheers, Joshua On Wed, Mar 8, 2017 at 8:44 PM, David McLaughlin wrote: > Ticket for replace task primitive already exists: > https://issues.apache.org/jir

Re: Issue with DB Migrations and Tests

2017-02-16 Thread Joshua Cohen
I think the proper solution would be to delete the V004_ migration. This migration originally landed in the 0.14.0 release. Removing this migration means an upgrade directly from 0.13.0 to 0.18.0 would not be possible, but I suspect it wouldn't be possible for other reasons as well (i.e. too large

Welcome new committers and PMC members!

2017-02-06 Thread Joshua Cohen
I'm happy to announce that we've got two new members to add to our ranks! Mehrdad Nurolahzade is now a committer and PMC member. Santhosh Kumar Shanmugham is now a committer. Please join me in congratulating them on their new roles! Cheers, Joshua

Re: Design Doc: Mesos V1 API

2017-02-01 Thread Joshua Cohen
Overall proposal sounds reasonable to me, thanks Zameer! My main question is whether we're setting ourselves up for possible performance regressions by switching to the HTTP API? We'll obviously need to switch eventually, but would be good to understand the performance impact (if any) of the switch

Re: [DRAFT][REPORT]: Apache Aurora - December 2016

2016-12-07 Thread Joshua Cohen
+1, thanks for doing this Jake! On Wed, Dec 7, 2016 at 9:55 AM, Jake Farrell wrote: > Below is our draft board report which is due next week. Please let me know > if you see any additions or changes that should be made, I'll plan on > submitting this Friday if there are no objections > > -Jake

Re: Preparations for 0.17.0

2016-11-03 Thread Joshua Cohen
I added https://issues.apache.org/jira/browse/AURORA-1782 to 0.17.0 Hopefully I'll have time to look into that soon. On Wed, Nov 2, 2016 at 4:53 PM, Zameer Manji wrote: > Thanks for stepping up! > > I think picking up Mesos 1.1 is ideal for the release and we should block > our release until it

Re: [VOTE] Release Apache Aurora 0.16.0 packages

2016-10-25 Thread Joshua Cohen
This vote has succeeded. There were three +1 (binding) votes: Joshua Cohen Stephan Erb John Sirois The official packages are now available here: https://dl.bintray.com/apache/aurora/ Thanks all! On Mon, Oct 24, 2016 at 3:18 PM, Joshua Cohen wrote: > Yeah, I've also fixed the

Re: [VOTE] Release Apache Aurora 0.16.0 packages

2016-10-24 Thread Joshua Cohen
Yeah, I've also fixed the release candidate script to just generate the email for us (c.f. https://reviews.apache.org/r/53102/) to avoid these problems in the future! On Mon, Oct 24, 2016 at 3:05 PM, John Sirois wrote: > On Oct 24, 2016 3:50 PM, "Joshua Cohen" wrote: > &

Re: [VOTE] Release Apache Aurora 0.16.0 packages

2016-10-24 Thread Joshua Cohen
lawlessly. > > On Thu, Oct 20, 2016 at 5:39 PM, Joshua Cohen wrote: > > > The git url does? This one works for me: > > https://git1-us-west.apache.org/repos/asf?p=aurora- > > packaging.git;a=tree;hb=refs/heads/0.16.x > > > > $ curl -I > > https://git1-us-we

Re: [VOTE] Release Apache Aurora 0.16.0 packages

2016-10-20 Thread Joshua Cohen
: Apache/2.4.7 (Ubuntu) Vary: Accept-Encoding Content-Type: text/html; charset=utf-8 On Thu, Oct 20, 2016 at 4:29 PM, Henry Saputra wrote: > Seems like it returns 404? > > On Thu, Oct 20, 2016 at 10:38 AM, Joshua Cohen wrote: > > > Also, apparently gmail didn't update the

Re: [VOTE] Release Apache Aurora 0.16.0 packages

2016-10-20 Thread Joshua Cohen
g/repos/asf?p=aurora-packaging.git;a=tree;hb=refs/heads/0.15.x> On Thu, Oct 20, 2016 at 10:34 AM, Joshua Cohen wrote: > I'll start the voting off with my own +1. I verified packages for all > three platforms against the test vagrant images from the packaging repo. > > On Thu, Oct 2

Re: [VOTE] Release Apache Aurora 0.16.0 packages

2016-10-20 Thread Joshua Cohen
I'll start the voting off with my own +1. I verified packages for all three platforms against the test vagrant images from the packaging repo. On Thu, Oct 20, 2016 at 10:33 AM, Joshua Cohen wrote: > All, > > > I propose that we accept the following artifacts as the offic

[VOTE] Release Apache Aurora 0.16.0 packages

2016-10-20 Thread Joshua Cohen
All, I propose that we accept the following artifacts as the official deb and rpm packaging for Apache Aurora 0.16.0: *https://dl.bintray.com/jcohen/aurora/ * The Aurora deb and rpm packaging includes the following: --- The branch used to create the pa

Re: Trouble publishing Aurora 0.16.0 binaries to bintray

2016-10-20 Thread Joshua Cohen
for > this to help automate the process > > -Jake > > > > On Wed, Oct 19, 2016 at 5:25 PM, Joshua Cohen wrote: > > > Hi all, > > > > I'm trying to get a vote out for the 0.16.0 binaries, but I'm running > into > > some issues publi

Trouble publishing Aurora 0.16.0 binaries to bintray

2016-10-19 Thread Joshua Cohen
Hi all, I'm trying to get a vote out for the 0.16.0 binaries, but I'm running into some issues publishing the binaries. I'm wondering if those who have done this in the past have any advice. The problem seems to boil down to this: when I upload the tarball generated by the release-candidate scrip

Re: A mini postmortem on snapshot failures

2016-10-04 Thread Joshua Cohen
Hi Zameer, Thanks for this writeup! I think one other option to consider would be using a connection for writing the snapshots that's not bound by the pool's maximum checkout time. I'm not sure if this is feasible or not, but I worry that there's potentially no upper bound on raising the maximum

[RESULT][VOTE] Release Apache Aurora 0.16.0 RC2

2016-09-27 Thread Joshua Cohen
All, The vote to accept Apache Aurora 0.16.0 RC2 as the official Apache Aurora 0.16.0 release has passed. +1 (Binding) -- Joshua Cohen Stephan Erb Maxim Khutornenko John Sirois Zameer Manji Jake Farrell +1 (Non-binding) -- Mehrdad

Re: [VOTE] Release Apache Aurora 0.16.0 RC2

2016-09-22 Thread Joshua Cohen
+0 Missing AURORA-1779 in CHANGELOG +1 Everything checks out via verify-release-candidate Overall +1 (binding). On Thu, Sep 22, 2016 at 2:18 PM, Joshua Cohen wrote: > Note: I forgot to mark AURORA-1779 as fixed in 0.16.0 before cutting this > RC, so that isn't reflected in the c

Re: [VOTE] Release Apache Aurora 0.16.0 RC2

2016-09-22 Thread Joshua Cohen
Note: I forgot to mark AURORA-1779 as fixed in 0.16.0 before cutting this RC, so that isn't reflected in the changelog. I don't consider that a blocker to release, but if others disagree I can cut rc3. On Thu, Sep 22, 2016 at 2:12 PM, Joshua Cohen wrote: > All, > > I propose

[VOTE] Release Apache Aurora 0.16.0 RC2

2016-09-22 Thread Joshua Cohen
All, I propose that we accept the following release candidate as the official Apache Aurora 0.16.0 release. Aurora 0.16.0-rc2 includes the following: --- The RELEASE NOTES for the release are available at: https://git-wip-us.apache.org/repos/asf?p=aurora.git&f=RELEASE-NOTES.md&hb=rel/0.16.0-rc2

[RESULT] [VOTE] Release Apache Aurora 0.16.0 RC1

2016-09-22 Thread Joshua Cohen
I'm declaring this vote a failure due to AURORA-1779. RC2 is incoming momentarily. On Wed, Sep 21, 2016 at 4:40 PM, Erb, Stephan wrote: > Unfortunate -1 from me. I bumped into this: https://issues.apache.org/ > jira/browse/AURORA-1779 > > > On 20/09/16 22:37, "Joshua Co

Re: [VOTE] Release Apache Aurora 0.16.0 RC1

2016-09-20 Thread Joshua Cohen
I'll start with my own +1 vote. Verified with the verify-release-candidate script. On Tue, Sep 20, 2016 at 3:01 PM, Joshua Cohen wrote: > All, > > I propose that we accept the following release candidate as the official > Apache Aurora 0.16.0 release. > > Aurora

[VOTE] Release Apache Aurora 0.16.0 RC1

2016-09-20 Thread Joshua Cohen
All, I propose that we accept the following release candidate as the official Apache Aurora 0.16.0 release. Aurora 0.16.0-rc1 includes the following: --- The RELEASE NOTES for the release are available at: https://git-wip-us.apache.org/repos/asf?p=aurora.git&f=RELEASE-NOTES.md&hb=rel/0.16.0-rc1

[RESULT] [VOTE] Release Apache Aurora 0.16.0 RC0

2016-09-20 Thread Joshua Cohen
Due to the issue uncovered by Zameer I'm closing this vote. I'll fix this and send out rc1 asap. On Mon, Sep 19, 2016 at 5:26 PM, Zameer Manji wrote: > -1 > > I discovered https://issues.apache.org/jira/browse/AURORA-1777 in this > release. > > On Mon, Sep 19, 20

[VOTE] Release Apache Aurora 0.16.0 RC0

2016-09-19 Thread Joshua Cohen
All, I propose that we accept the following release candidate as the official Apache Aurora 0.16.0 release. Aurora 0.16.0-rc0 includes the following: --- The RELEASE NOTES for the release are available at: https://git-wip-us.apache.org/repos/asf?p=aurora.git&f=RELEASE-NOTES.md&hb=rel/0.16.0-rc0

Re: Aurora 0.16.0 release

2016-09-12 Thread Joshua Cohen
point out that I would like to add a deprecation in this > release and there is a review out already. > https://reviews.apache.org/r/51712/ > > Otherwise it seems we are good to go for a release from my perspective. > > On Mon, Sep 12, 2016 at 11:38 AM, Joshua Cohen wrote: > &g

Re: Aurora 0.16.0 release

2016-09-12 Thread Joshua Cohen
some time. > > > > > > > > On Tue, Sep 6, 2016 at 10:54 AM, Zameer Manji > > wrote: > > > > > > > >> + 1 > > > >> > > > >> I think we are due for a release so folks can get Mesos 1.0 and GPU > > >

Re: [DRAFT][REPORT] Apache Aurora - September 2016

2016-09-06 Thread Joshua Cohen
I think the big thing in 0.16.0, if we're calling out features it will support, is task filesystem isolation. That said, I doubt the board really cares that much about the details, so +1 from me ;). On Tue, Sep 6, 2016 at 2:28 PM, Maxim Khutornenko wrote: > +1 > > On Tue, Sep 6, 2016 at 12:15 PM

Re: 答复: Discussion on review request 51536

2016-09-06 Thread Joshua Cohen
; > > +1 to Zameer's idea. Now we have three persons on board. > > > > > > -- > > *From:* Maxim Khutornenko > > *Sent:* Friday, September 2, 2016 18:36 > > *To:* dev@aurora.apache.org > > *Cc:* 黄 凯; Joshua Cohen; s.

Aurora 0.16.0 release

2016-09-06 Thread Joshua Cohen
Hi Aurorans, I plan to kick off the 0.16.0 release some time later this week. Please let me know if there are any outstanding patches you'd like to ship before this release. Thanks! Joshua

Re: [FEEDBACK] Transitioning Aurora leader election to Apache Curator (`-zk_use_curator`)

2016-08-24 Thread Joshua Cohen
I have this enabled in a test cluster and have not noticed any issues with it yet. I'd like to roll it out to production before we drop the old code though. On Wed, Aug 24, 2016 at 1:10 PM, Zameer Manji wrote: > Could we change the default and drop the old code at the same time? I don't > see an

Re: Support instance-specific TaskConfig in CreateJob API

2016-08-15 Thread Joshua Cohen
On Fri, Aug 12, 2016 at 11:05 AM, Min Cai wrote: > > > (1) There will be a spam of jobs with each only have one task instance. > This will make the Aurora UI almost unusable. Also, the user will have to > track the grouping of jobs and introduce a "job group" like concept on the > client side. I

Re: [PROPOSAL] Move Aurora discussions to Slack?

2016-08-02 Thread Joshua Cohen
I'm +0 on this. I agree that Slack is in many ways superior to IRC, but it also feels like the barrier to entry for Slack is much higher than it is for IRC which is potentially problematic for an open source community. On Tue, Aug 2, 2016 at 4:08 PM, Jake Farrell wrote: > There is an irc bridge

Re: [VOTE] Release Apache Aurora 0.15.0 RC1

2016-07-01 Thread Joshua Cohen
+1 everything checks out w/ verify-release-candidate. On Thu, Jun 30, 2016 at 9:32 PM, Maxim Khutornenko wrote: > +1 > > All verification tests passed for me and scheduler has been deployed > to our test cluster. > > On Thu, Jun 30, 2016 at 6:56 PM, Maxim Khutornenko > wrote: > > All, > > > > >

Re: [VOTE] Release Apache Aurora 0.13.0 packages

2016-06-29 Thread Joshua Cohen
+1, looks good to me. On Wed, Jun 29, 2016 at 1:16 PM, John Sirois wrote: > +1 the aurora-packaging test instructions worked for me for debian-jessie, > ubuntu-trusty and centos-7 > > On Wed, Jun 29, 2016 at 11:39 AM, Maxim Khutornenko > wrote: > > > +1 > > > > Tested with pkg_root="https://dl.

Re: Aurora 0.14.0 release

2016-06-07 Thread Joshua Cohen
+1 to releasing 0.14.0 and thanks for volunteering to act as RM, Stephan! On Tue, Jun 7, 2016 at 8:52 AM, Jake Farrell wrote: > Take a look at the "Creating a release" section in docs/development/ > committers-guide.md, you will also need to add you GPG key to the > necessary > files, feel free

Re: Agent / Slave renaming

2016-05-18 Thread Joshua Cohen
+1 for me. On Wed, May 18, 2016 at 7:15 AM, Erb, Stephan wrote: > I got some spare time yesterday and used it to rebase the first of a > series of stale review requests by Kevin that implement the Mesos agent > renaming in Aurora (https://reviews.apache.org/r/47495/). > > The renaming is ranked

Re: [PROPOSAL] Switch aurora client from service discovery to HTTP redirects.

2016-04-18 Thread Joshua Cohen
And apparently this is not part of our fork at all, the client already supports this today! The only potential change that would be required would be ensuring the client properly follows redirects. On Mon, Apr 18, 2016 at 5:35 PM, Joshua Cohen wrote: > Er, it's not `proxy_ur

Re: [PROPOSAL] Switch aurora client from service discovery to HTTP redirects.

2016-04-18 Thread Joshua Cohen
Er, it's not `proxy_url`, it's `scheduler_uri` (which makes much more sense ;)). On Mon, Apr 18, 2016 at 5:34 PM, Joshua Cohen wrote: > I'm not opposed to this, in fact we already do something similar > internally. We've forked clusters.py to allow configuring a `pro

Re: [PROPOSAL] Switch aurora client from service discovery to HTTP redirects.

2016-04-18 Thread Joshua Cohen
I'm not opposed to this, in fact we already do something similar internally. We've forked clusters.py to allow configuring a `proxy_url` for each cluster. If that's present, then the client will use it rather than performing service discovery to communicate with the scheduler. On Mon, Apr 18, 2016

Heads up: Mesos 0.27 and Vagrant

2016-04-18 Thread Joshua Cohen
Hi Aurorans, Just a heads up, I've landed the upgrade to Mesos 0.27.2. This is our first Mesos version bump since switching to a custom pre-built Vagrant box. If you try and build the Scheduler or Executor in an existing Vagrant image you'll likely run into errors. Please destroy and re-create you

Re: [VOTE] Release Apache Aurora 0.13.0 RC0

2016-04-12 Thread Joshua Cohen
Ahh, thanks for being on top of that John! On Tue, Apr 12, 2016 at 3:05 PM, John Sirois wrote: > On Tue, Apr 12, 2016 at 1:48 PM, Joshua Cohen wrote: > > > +1, ran the verification script, everything looks good to me. > > > > Nitpick: the NEWS link in the email 404s

Re: [VOTE] Release Apache Aurora 0.13.0 RC0

2016-04-12 Thread Joshua Cohen
+1, ran the verification script, everything looks good to me. Nitpick: the NEWS link in the email 404s because we renamed it to RELEASE-NOTES.md. We should probably update the script that generates the email? On Tue, Apr 12, 2016 at 10:02 AM, Erb, Stephan wrote: > +1 for releasing 0.13.0-rc0 as

Re: Looking for feedback - Setting CommandInfo.user by default when launching tasks.

2016-04-12 Thread Joshua Cohen
As things stand today, once a task is scheduled, the scheduler can die, or be shut down for maintenance, etc. with no impact to running tasks. If the scheduler were responsible for announcing and it maintained the current practice of creating znodes as ephemeral, it would need to maintain a persist

Re: [PROPOSAL] Support GPU resources in Aurora

2016-04-06 Thread Joshua Cohen
+1 On Wed, Apr 6, 2016 at 12:41 PM, Maxim Khutornenko wrote: > Mesos community is finalizing their MVP for supporting GPUs: > https://issues.apache.org/jira/browse/MESOS-4424 > Design doc: > https://docs.google.com/document/d/10GJ1A80x4nIEo8kfdeo9B11PIbS1xJrrB4Z373Ifkpo/edit > > Would anyone hav

Re: Are we ready to remove the observer?

2016-04-04 Thread Joshua Cohen
hough! On Mon, Apr 4, 2016 at 9:32 AM, Bill Farner wrote: > > > > 2) Providing an easy view of a process's command-line > > 3) Providing a holistic view of the task config > > > Just to check my understanding - these could be trivially handled in > text/log form

Re: Are we ready to remove the observer?

2016-04-04 Thread Joshua Cohen
I'm -1 on this until we have an actual replacement for the Observer. I think that the observer provides significant value outside of just sandbox browsing: 1) Exporting task-level statistics. 2) Providing an easy view of a process's command-line 3) Providing a holistic view of the task config 4) R

Re: Populate DiscoveryInfo in Mesos

2016-03-30 Thread Joshua Cohen
Job names are not unique though, what would happen if multiple jobs had the same name (either across roles or across environments in the same role)? On Wed, Mar 30, 2016 at 5:33 PM, Zhitao Li wrote: > Stephan, > > So I've managed to run the official Mesos DNS docker container >

Re: Compute event at Twitter HQ

2016-03-30 Thread Joshua Cohen
ue, Mar 15, 2016 at 4:04 PM, Joshua Cohen wrote: > Hi Aurorans[1], > > I'd like to call attention to an event the Compute group at Twitter is > holding at the end of the month where there will be a few of > Aurora/Mesos-related talks: > > >1. David Robinson, one of

Re: Handling migrations if dbScript exists in snapshot?

2016-03-29 Thread Joshua Cohen
ctive on github.) I opened https://github.com/mybatis/migrations/issues/38 to see about addressing that. On Thu, Mar 24, 2016 at 11:00 AM, Joshua Cohen wrote: > Sorry, I should've been more clear above. What I meant to say is that the > lack of explicit rollback support was not a mar

Re: Handling migrations if dbScript exists in snapshot?

2016-03-24 Thread Joshua Cohen
y flyway) to > establish a new baseline upon forward migration that may be used by a > reverse flyway instance to apply rollback scripts in an event of a version > downgrade. > > I feel this is an important problem that we need to have full clarity > around before moving forward

Re: Handling migrations if dbScript exists in snapshot?

2016-03-24 Thread Joshua Cohen
ly they are complaining quite > > fast ;) )... > > > > Another thing that's similar to flyway is http://www.liquibase.org/ but > for > > that I have zero experience. > > > > Flo > > > > > > On 23 March 2016 at 22:41, Joshua Cohen wrot

Re: Handling migrations if dbScript exists in snapshot?

2016-03-23 Thread Joshua Cohen
If we go with option 2 (which I'm leaning towards as well), does anyone have thoughts on using (or experience with) something like Flyway: https://flywaydb.org/ rather than implementing from scratch? On Wed, Mar 23, 2016 at 3:29 PM, John Sirois wrote: > On Wed, Mar 23, 2016 at 2:21 PM

Handling migrations if dbScript exists in snapshot?

2016-03-23 Thread Joshua Cohen
Hi Aurorans, As you may have seen (https://issues.apache.org/jira/browse/AURORA-1648), we ran into an issue when upgrading a cluster that uses dbScripts in snapshots. To restate the problem from the ticket, when the scheduler starts up it creates the H2 database from schema.sql which contains newl

Re: aurora job scalability

2016-03-19 Thread Joshua Cohen
Hi Christopher, I think you already got an answer from Stephan in IRC, but just wanted to follow up for the sake of posterity (in case anyone in the future has a similar question and finds this thread). The only limit on the number of jobs that Aurora can run would currently be the amount of memor

Compute event at Twitter HQ

2016-03-15 Thread Joshua Cohen
Hi Aurorans[1], I'd like to call attention to an event the Compute group at Twitter is holding at the end of the month where there will be a few of Aurora/Mesos-related talks: 1. David Robinson, one of our SREs, will talk about how our small team of SREs manages what is possibly the larges

Re: [PROPOSAL] Supporting Mesos Universal Containers

2016-03-15 Thread Joshua Cohen
gt; > Kind Regards, > Stephan > > > From: Joshua Cohen > Sent: Monday, March 7, 2016 20:58 > To: dev@aurora.apache.org > Subject: [PROPOSAL] Supporting Mesos Universal Containers > > Hi all, > > I'd like to propose we ad

Fwd: Executors no longer inherit environment variables from the agent

2016-03-10 Thread Joshua Cohen
I just spent some time to dig into this and it looks like we *will* be impacted by this. I updated mesos-slave.conf in Vagrant to pass --executor_environment_variables='{}' and as predicted by this pull request: https://github.com/apache/aurora/pull/21, sys.executable is empty, so we fail to fork t

Re: [VOTE] Release Apache Aurora 0.12.0 debs

2016-03-09 Thread Joshua Cohen
+1, verified under both test/deb environments. On Tue, Mar 8, 2016 at 12:18 PM, John Sirois wrote: > On Mon, Mar 7, 2016 at 12:57 PM, Bill Farner wrote: > > > +1 > > > > Successfully installed and lightly exercised ubuntu and debian packages > in > > their respective systems (using test/ instru

[PROPOSAL] Supporting Mesos Universal Containers

2016-03-07 Thread Joshua Cohen
Hi all, I'd like to propose we adopt the Mesos universal container support for provisioning tasks from both Docker and AppC images. Please review the doc below and let me know what you think. https://docs.google.com/document/d/111T09NBF2zjjl7HE95xglsDpRdKoZqhCRM5hHmOfTLA/edit?usp=sharing Thanks!

Re: [DRAFT] [REPORT] Apache Aurora

2016-03-01 Thread Joshua Cohen
s period, with strong > > > community engagement. We have placed deliberate effort to build > release > > > notes as we iterate towards future releases, which has received > positive > > > feedback from users. > > > > > > ## PMC changes: > > > - Cu

Re: Weekly community meeting

2016-02-29 Thread Joshua Cohen
ently for Aurora > dlester, jfarrell, kts, wfarner , and zmanji are the only ones who > are registered and setup to start the meetings > > -Jake > > > [1]: > https://svn.apache.org/repos/private/committers/docs/freenode-cloaks.txt > > On Mon, Feb 29, 2016 at 10:53 AM

Re: Weekly community meeting

2016-02-29 Thread Joshua Cohen
I agree that it's unfortunate we've missed a few meetings. Historically speaking it's always been one of a few people who kick the meeting off, and if those folks aren't available the meetings seem to not happen. That said, I think we should take a more decentralized approach. We don't need to rely

Re: Further thoughts on config deprecations

2016-02-04 Thread Joshua Cohen
.org > Subject: Re: Further thoughts on config deprecations > > On Wed, Feb 3, 2016 at 6:34 PM, Joshua Cohen wrote: > > > How would folks feel about requiring any changes that deprecate job > config > > to include some sort of codemod[1]-like patch that would allo

Further thoughts on config deprecations

2016-02-03 Thread Joshua Cohen
How would folks feel about requiring any changes that deprecate job config to include some sort of codemod[1]-like patch that would allow cluster operators to automatically fix deprecated fields across their company's Aurora configs? We could either leverage codemod directly, or create nicer tooli

Re: New committer and PMC member: Stephan Erb

2016-02-03 Thread Joshua Cohen
Welcome Stephan! On Wed, Feb 3, 2016 at 1:10 PM, Zhitao Li wrote: > Congrats Stephan, well deserved! > > On Wed, Feb 3, 2016 at 11:03 AM, Erb, Stephan > > wrote: > > > Awesome, thanks! Great to be on board! :-) > > > > > > From: Bill Farner > > Sent: Wednesday,

Re: Rollback Testing

2016-02-02 Thread Joshua Cohen
https://issues.apache.org/jira/browse/AURORA-1608 https://issues.apache.org/jira/browse/AURORA-1609 On Tue, Feb 2, 2016 at 7:42 PM, Joshua Cohen wrote: > Ok, I'll file a ticket to at least track the intent. We'll see if I have > time to work on it or if it just languis

Re: Rollback Testing

2016-02-02 Thread Joshua Cohen
advantage of this. > > On Mon, Feb 1, 2016 at 2:27 PM, Joshua Cohen wrote: > > > I know the blocker for e2e tests in CI previously was the ability to run > > vagrant. Is running docker from CI doable today? > > > > On Mon, Feb 1, 2016 at 2:40 PM, Bill Farner wr

Re: NEWS Layout

2016-02-02 Thread Joshua Cohen
+1 On Tue, Feb 2, 2016 at 1:09 PM, Bill Farner wrote: > +1 > > On Tuesday, February 2, 2016, Erb, Stephan > wrote: > > > Hi everyone, > > > > I'd like to propose that we give our NEWS file a little bit more > > structure. Currently, it is quite cluttered [1]. > > > > To keep it simple, I'd sugg

Re: 0.12.0 RC status

2016-02-02 Thread Joshua Cohen
I'd only consider item 1 to be a blocker to 0.12.0, but 2 and 3 should be relatively quick so in general this sounds like a reasonable plan of action to me. On Tue, Feb 2, 2016 at 8:52 AM, John Sirois wrote: > Although the last blocker raised for the 0.12.0 RC series has been resolved > [1], it

Re: Rollback Testing

2016-02-01 Thread Joshua Cohen
hrough. > > e2e tests in jenkins can be done, but likely only if e2e tests start using > docker instead of vagrant. I am supportive of both, but cannot personally > invest the time at the moment. > > On Mon, Feb 1, 2016 at 12:14 PM, Joshua Cohen wrote: > > > Another topic

Re: Deprecation Cycles

2016-02-01 Thread Joshua Cohen
wrote: > Sounds reasonable. Can you firm up the proposal and apply numbers? > > On Mon, Feb 1, 2016 at 12:09 PM, Joshua Cohen wrote: > > > This came up briefly last week, and we also discussed in the IRC meeting > > today. Given the increased release cadence, I think we

Rollback Testing

2016-02-01 Thread Joshua Cohen
Another topic that came up today's IRC meeting was possibly adding some sort of automated rollback testing between builds. This is related to https://issues.apache.org/jira/browse/AURORA-1603 which was caused by an inability to rollback to an earlier commit after discovering https://issues.apache.o

Deprecation Cycles

2016-02-01 Thread Joshua Cohen
This came up briefly last week, and we also discussed in the IRC meeting today. Given the increased release cadence, I think we would benefit from moving from release-based deprecation to time-based. That is to say, instead of deprecating in release X and removing release X+1, we would deprecate in

Re: [VOTE] Release Apache Aurora 0.12.0 RC0

2016-01-28 Thread Joshua Cohen
Changing my vote to -1. We just found a critical bug with Maxim's commit: https://github.com/apache/aurora/commit/fee5943a95c4f08e148dc5f1366486a8c23d5773. This version is not suitable for release (or deploy for that matter). On Thu, Jan 28, 2016 at 2:49 PM, Bill Farner wrote: > (moved off priva

Re: [PROPOSAL] Job instance scaling APIs

2016-01-14 Thread Joshua Cohen
What happens if a job has been scaled out, but the underlying config is not updated to take that scaling into account? Would the next update on that job revert the number of instances (presumably, because what else could we do)? Is there anything we can do, tooling-wise, to improve upon this? On T

Re: [PROPOSAL] Amend 0.12.0 release goals

2016-01-14 Thread Joshua Cohen
Sounds good to me. On Thu, Jan 14, 2016 at 9:31 AM, Erb, Stephan wrote: > +1 for catching up > > From: John Sirois > Sent: Thursday, January 14, 2016 4:18 PM > To: dev@aurora.apache.org > Subject: Re: [PROPOSAL] Amend 0.12.0 release goals > > On Thu, Jan

Re: [PROPOSAL] Replace commons-args

2016-01-12 Thread Joshua Cohen
I'm in favor of this proposal and will be very happy to see commons-args go away :). On Tue, Jan 12, 2016 at 6:53 AM, John Sirois wrote: > On Mon, Jan 11, 2016 at 9:49 PM, Bill Farner wrote: > > > Any other thoughts on this? I will proceed with initial encapsulation > work > > in the meantime,

Re: New committer and PMC member: Steve Niemitz

2016-01-12 Thread Joshua Cohen
Welcome aboard Steve, thanks for the contributions, looking forward to more ;). On Tue, Jan 12, 2016 at 9:16 AM, Thanos Baskous wrote: > Congratulations, Steve! > > On Tue, Jan 12, 2016 at 9:15 AM, Tony Dong > wrote: > > > Congrats Steve! 👑 > > > > On Tuesday, January 12, 2016, Steve Niemitz w

Re: Ticket cleanup

2015-12-28 Thread Joshua Cohen
I'm not sure I agree with this sentiment. I don't have any problems with an aspirational backlog of tickets. At the very least it's a place to refer people who request commonly asked for features. Perhaps the solution isn't to close tickets that we don't imagine will see attention in the immediate

Re: Commits without reviews

2015-12-24 Thread Joshua Cohen
It could be, I just think it's easier to comment on a reviewboard than it is a commits@ email. On Thu, Dec 24, 2015 at 10:29 AM, Bill Farner wrote: > Can that be handled by subscribing to commits@? > > On Thursday, December 24, 2015, Joshua Cohen wrote: > > > I'm

Re: Commits without reviews

2015-12-24 Thread Joshua Cohen
I'm generally ok with this. Just curious: what do you think about maybe posting a review and then committing it right away in these cases though? A bit noisy on the reviews@ list, but at least it'd give people a chance to peruse/comment as they see fit (with the assumption that any comments would b

Re: [VOTE] Release Apache Aurora 0.11.0 RC1

2015-12-21 Thread Joshua Cohen
+1 non-binding On Sun, Dec 20, 2015 at 10:21 AM, Bill Farner wrote: > Looks like it was orphaned - not linked against the RC ticket and not > listed as a blocker of https://issues.apache.org/jira/browse/AURORA-1367. > I'll move the ticket underneath it to the 0.12.0. > > On Sun, Dec 20, 2015 at

Re: [VOTE] Release Apache Aurora 0.10.0-1 debs

2015-12-11 Thread Joshua Cohen
Did this vote pass? I don't see the 0.10.0 debs on bintray (or a result thread). On Mon, Nov 30, 2015 at 12:30 PM, Joshua Cohen wrote: > Yeah, I agree, that would be preferable. > > On Mon, Nov 30, 2015 at 12:27 PM, John Sirois wrote: > >> On Mon, Nov 30, 2015 at 11:18

Re: [RFC] REST / thrift & AURORA-987

2015-12-01 Thread Joshua Cohen
John: I share David's concerns, but it's not clear to me that they're incompatible with your proposal. I.e. we could design the new REST API according to any interface we like and still implement that API on top of the thrift API (though the desired interface should always be our guide, we should n

Re: [VOTE] Release Apache Aurora 0.10.0-1 debs

2015-11-30 Thread Joshua Cohen
Yeah, I agree, that would be preferable. On Mon, Nov 30, 2015 at 12:27 PM, John Sirois wrote: > On Mon, Nov 30, 2015 at 11:18 AM, Joshua Cohen wrote: > > > +1 > > > > One minor nit, not sure if it's worth fixing: > > test/deb/ubuntu-trusty/README.md still re

Re: [VOTE] Release Apache Aurora 0.10.0-1 debs

2015-11-30 Thread Joshua Cohen
+1 One minor nit, not sure if it's worth fixing: test/deb/ubuntu-trusty/README.md still references 0.9.0, should be updated to 0.10.0? On Fri, Nov 27, 2015 at 12:54 PM, John Sirois wrote: > On Fri, Nov 27, 2015 at 11:34 AM, Bill Farner wrote: > > > I propose that we accept the following artifa

Re: [VOTE] Release Apache Aurora 0.10.0 RC2

2015-11-13 Thread Joshua Cohen
+1, verification script passed for me. On Thu, Nov 12, 2015 at 4:33 PM, Maxim Khutornenko wrote: > +1 binding based on verification script run. > > On Thu, Nov 12, 2015 at 1:51 PM, Zameer Manji wrote: > > +1 (binding). I verified this release on a clean machine running OSX > 10.10 > > using the

IRC Meeting notes for October 12th

2015-10-12 Thread Joshua Cohen
No one with karma was around to start the meeting official, here are the ad-hoc meeting notes: Mon Oct 12 18:07:58 2015 jcohen: Hi folks, it seems that no one with karma is around to officially start the meeting, so let’s have an ad-hoc meeting and I’ll manually send out note. Mon Oct 12 18:08:0

Re: Moving forward on AURORA-1503

2015-10-05 Thread Joshua Cohen
+1 to releasing as 0.10.0. I hope that Mesos does move to a time-based deprecation cycle rather than +/- 1 version, since it could easily be the case that a Mesos release falls at a time when we are not prepared to release Aurora. Perhaps until Mesos addresses their deprecation cycles, we make off

Re: Upgrading from 0.7

2015-09-21 Thread Joshua Cohen
Filed https://issues.apache.org/jira/browse/AURORA-1497 to add documentation on how best to upgrade the various Aurora components. On Mon, Sep 21, 2015 at 12:58 PM, Joseph Smith wrote: > We also shut down all of the schedulers, upgrade to the new version, then > bring them all back up. This isn’

  1   2   >