from:"Kevin Sweeney"

Re: Moving forward on AURORA-1503

2015-10-05 Thread Kevin Sweeney

It seems that this is not a correct usage of semver and that we should
release the link to 0.23.0 as 0.10.0. Given the mesos deprecation cycle
described in that ticket, 0.22.0 would be compatible with 0.21.0. IMO a
patch release should never break compatibility with versions that the
previous patch version worked with - it should be for things like
showstopper security or reliability bugs.

I'd be +1 to releasing a 0.10.0 that contains only the patch changing our
link target to 0.23.0 and pushing our next "feature release" to 0.11.0.

On Mon, Oct 5, 2015 at 3:49 PM Zameer Manji  wrote:

> Hey,
>
> As I brought up on today's IRC meeting
>  we have a bit of a problem
> in AURORA-1503  and
> some
> discussion is required to figure out what we need to do.
>
> Since Aurora 0.9.0 was released the Mesos release cadence has increased to
> about one a month, while keeping their deprecation policy to be -1/+1.
> Aurora 0.9.0 was released against Mesos 0.22 which means Aurora 0.9.0 will
> work against Mesos 0.23 but not Mesos 0.24.
>
> The scenario means our users are not able to update to Mesos 0.24 (or
> later) if they are running Aurora 0.9.0. The deprecation policy also means
> we also need to carefully think about which versions of Mesos we release
> against.
>
> I'm proposing we do the following:
>
>- Release 0.9.1 which is 0.9.0 linked against Mesos 0.23. This will
>allow our users to upgrade their clusters to Mesos 0.24 in the short
> term.
>Mesos 0.23 is also compatible with Mesos 0.22 meaning existing users
> should
>not be impacted.
>- Assuming we do the above, we release the next major version of Aurora
>against Mesos 0.24. This provides a smooth upgrade path from 0.9.x to
>0.10.x.
>
> What are our thoughts on this?
>
> --
> Zameer Manji
>

Re: [DRAFT] [REPORT] Apache Aurora

2015-09-08 Thread Kevin Sweeney

+1

On Tue, Sep 8, 2015 at 5:04 PM, Bill Farner  wrote:

> Please take a moment to read through a draft of the board report i have
> prepared for Aurora.  I'm happy to take any suggestions for edits prior to
> submitting this tomorrow.
>
>
> ## Description:
>  - Apache Aurora lets you use an Apache Mesos cluster as a private cloud.
> It supports running long-running services, cron jobs, and ad-hoc jobs.
>
> ## Activity:
>  - Release planning underway (0.10.0)
>  - Major effort recently to produce official installable packages (RPMs and
> debs)
>- released of debs for 0.9.0, closed to release of RPMs
>- planning support for unofficial/unsupported nightly packages
>
"support for unsupported" - phrasing is weird, perhaps change to something
more neutral like "setting up infrastructure for nightly packages"

>
> ## Health report:
>  - There has been healthy development on Aurora for this period, though we
> hope to see growth in adoption, contributions, and committers.  There has
> been significant effort to minimize the effort required to evaluate
> alternatives to Aurora (e.g. Kubernetes, Marathon), which we believe
> impedes our adoption rate.  Packaging and better introductory documentation
> will our focus to address this.
>
> ## Issues:
>  - There are no issues requiring board attention at this time
>
> ## LDAP committee group/Committership changes:
>
>  - Currently 15 committers and 13 LDAP committee group members.
>  - No new changes to the LDAP committee group or committership since last
>report.
>
> ## Releases:
>
>  - 0.9.0 was released on Thu Jul 23 2015
>
> ## Mailing list activity:
>
>  - dev@aurora.apache.org:
> - 147 subscribers (up 11 in the last 3 months):
> - 293 emails sent to list (437 in previous quarter)
>
>  - revi...@aurora.apache.org:
> - 33 subscribers (up 0 in the last 3 months):
> - 1298 emails sent to list (1419 in previous quarter)
>
>  - iss...@aurora.apache.org:
> - 37 subscribers (down -1 in the last 3 months):
> - 753 emails sent to list (844 in previous quarter)
>
>
> ## JIRA activity:
>
>  - 136 JIRA tickets created in the last 3 months
>  - 103 JIRA tickets closed/resolved in the last 3 months
>

Re: Creating Aurora user@ and announcements@ lists?

2015-09-08 Thread Kevin Sweeney

+1

On Tue, Sep 8, 2015 at 11:07 AM, Zameer Manji  wrote:

> +1
>
> On Tue, Sep 8, 2015 at 10:35 AM, Mauricio Garavaglia <
> mauriciogaravag...@gmail.com> wrote:
>
> > +1
> >
> > On Tue, Sep 8, 2015 at 2:29 PM, Hussein Elgridly <
> > huss...@broadinstitute.org
> > > wrote:
> >
> > > +1. As a user, it felt kinda weird for me to be posting my questions to
> > > dev@
> > > .
> > >
> > > Hussein Elgridly
> > > Senior Software Engineer, DSDE
> > > The Broad Institute of MIT and Harvard
> > >
> > >
> > > On 8 September 2015 at 13:13, Dave Lester  wrote:
> > >
> > > > I'd like to propose establishing a few additional mailing lists for
> the
> > > > Aurora project:
> > > >
> > > > * a user@ list to field questions that may go unanswered related to
> > the
> > > > use and operations of Aurora. As the community grows, I increasingly
> > see
> > > > questions on IRC that go unanswered during non-business hours, or
> that
> > > > are better handled asynchronously. This list would be a space for
> those
> > > > discussions, along with other non-dev conversations.
> > > >
> > > > * an announcements@ list that only Aurora committers could publish
> to
> > > > for release, committer, and project announcements.
> > > >
> > > > Why should be create new lists? To increase communication with
> non-dev
> > > > members of the Aurora community. Additionally, I spent time digging
> > > > through stats from other Apache projects and found that non-dev lists
> > > > tend to have a higher subscriber rate -- particularly user@ lists.
> > Let's
> > > > give a space for those members of the community.
> > > >
> > > > Thoughts? If I don't hear any objections, I'll go ahead and file an
> > > > INFRA ticket later this week.
> > > >
> > > > Dave
> > > >
> > >
> >
> > --
> > Zameer Manji
> >
> >
>

Re: [VOTE] Release Apache Aurora 0.9.0 debs

2015-09-02 Thread Kevin Sweeney

+1 to these debs

On Sat, Aug 29, 2015 at 9:59 AM, Bill Farner  wrote:

> I propose that we accept the following artifacts as the official deb
> packaging for
> Apache Aurora 0.9.0.
>
>
> http://people.apache.org/~wfarner/aurora/distributions/0.9.0/deb/ubuntu-trusty/
>
> The Aurora deb packaging includes the following:
> ---
> The CHANGELOG is available at:
>
> https://git1-us-west.apache.org/repos/asf?p=aurora-packaging.git;a=blob_plain;f=specs/debian/changelog;hb=refs/heads/0.9.x
>
> The branch used to create the packaging is:
>
> https://git1-us-west.apache.org/repos/asf?p=aurora-packaging.git;a=tree;h=refs/heads/0.9.x
>
> The packages are available at:
>
> http://people.apache.org/~wfarner/aurora/distributions/0.9.0/deb/ubuntu-trusty/
>
> The GPG keys used to sign the packages are available at:
> https://dist.apache.org/repos/dist/release/aurora/KEYS
>
> Please download, verify, and test.
>
> The vote will close in 4 days, on Wed Sept 2 10:00:00 PT 2015
>
> [ ] +1 Release these as the deb packages for Apache Aurora 0.9.0
> [ ] +0
> [ ] -1 Do not release these artifacts because...
>
> I would like to get the voting started off with my own +1
>
>
> -=Bill
>

Re: [VOTE] Release Apache Aurora 0.9.0 debs

2015-08-26 Thread Kevin Sweeney

From my ASF email this time:

+1 (binding)

On Wed, Aug 26, 2015 at 11:31 AM, Kevin Sweeney kswee...@twitter.com
wrote:

 +1

 The one issue I have - thermos.pex seems to only appear in
 /usr/share/aurora/bin/thermos.pex and not as a globally-linked
 /usr/bin/thermos command. This makes it pretty unlikely to be discovered
 naturally.

 On Wed, Aug 26, 2015 at 11:02 AM, Bill Farner wfar...@apache.org wrote:

 To simplify testing these debs, i've got a branch that sets up the
 environment and dependencies in Vagrant:

 https://github.com/wfarner/aurora-packaging/tree/wfarner/deb_testing/test/deb/ubuntu-trusty

 If you clone that, you can then vagrant up, wget, and dpkg -i.

 I plan to formalize this testing to simplify things in the future, and add
 documentation for installation to the website...but that comes later.

 -=Bill

 On Wed, Aug 26, 2015 at 10:16 AM, Bill Farner wfar...@apache.org wrote:

  I propose that we accept the following artifacts as the official deb
  packaging for
  Apache Aurora 0.9.0.
 
 
 
 http://people.apache.org/~wfarner/aurora/distributions/0.9.0/deb/ubuntu-trusty/
 
  The Aurora deb packaging includes the following:
  ---
  The CHANGELOG is available at:
 
 
 https://git1-us-west.apache.org/repos/asf?p=aurora-packaging.git;a=blob_plain;f=specs/debian/changelog;hb=refs/heads/0.9.x
 
  The branch used to create the packaging is:
 
 
 https://git1-us-west.apache.org/repos/asf?p=aurora-packaging.git;a=tree;h=refs/heads/0.9.x
 
  The packages are available at:
 
 
 http://people.apache.org/~wfarner/aurora/distributions/0.9.0/deb/ubuntu-trusty/
 
  The GPG keys used to sign the packages are available at:
  https://dist.apache.org/repos/dist/release/aurora/KEYS
 
  Please download, verify, and test.
 
  The vote will close in 3 business days, on Mon Aug 31 10:00:00 PT 2015
 
  [ ] +1 Release these as the deb packages for Apache Aurora 0.9.0
  [ ] +0
  [ ] -1 Do not release these artifacts because...
 
  I would like to get the voting started off with my own +1
 
 
  -=Bill
 




 --
 Kevin Sweeney
 @kts

Re: [VOTE] Release Apache Aurora 0.9.0 debs

2015-08-26 Thread Kevin Sweeney

+1

The one issue I have - thermos.pex seems to only appear in
/usr/share/aurora/bin/thermos.pex and not as a globally-linked
/usr/bin/thermos command. This makes it pretty unlikely to be discovered
naturally.

On Wed, Aug 26, 2015 at 11:02 AM, Bill Farner wfar...@apache.org wrote:

 To simplify testing these debs, i've got a branch that sets up the
 environment and dependencies in Vagrant:

 https://github.com/wfarner/aurora-packaging/tree/wfarner/deb_testing/test/deb/ubuntu-trusty

 If you clone that, you can then vagrant up, wget, and dpkg -i.

 I plan to formalize this testing to simplify things in the future, and add
 documentation for installation to the website...but that comes later.

 -=Bill

 On Wed, Aug 26, 2015 at 10:16 AM, Bill Farner wfar...@apache.org wrote:

  I propose that we accept the following artifacts as the official deb
  packaging for
  Apache Aurora 0.9.0.
 
 
 
 http://people.apache.org/~wfarner/aurora/distributions/0.9.0/deb/ubuntu-trusty/
 
  The Aurora deb packaging includes the following:
  ---
  The CHANGELOG is available at:
 
 
 https://git1-us-west.apache.org/repos/asf?p=aurora-packaging.git;a=blob_plain;f=specs/debian/changelog;hb=refs/heads/0.9.x
 
  The branch used to create the packaging is:
 
 
 https://git1-us-west.apache.org/repos/asf?p=aurora-packaging.git;a=tree;h=refs/heads/0.9.x
 
  The packages are available at:
 
 
 http://people.apache.org/~wfarner/aurora/distributions/0.9.0/deb/ubuntu-trusty/
 
  The GPG keys used to sign the packages are available at:
  https://dist.apache.org/repos/dist/release/aurora/KEYS
 
  Please download, verify, and test.
 
  The vote will close in 3 business days, on Mon Aug 31 10:00:00 PT 2015
 
  [ ] +1 Release these as the deb packages for Apache Aurora 0.9.0
  [ ] +0
  [ ] -1 Do not release these artifacts because...
 
  I would like to get the voting started off with my own +1
 
 
  -=Bill
 




-- 
Kevin Sweeney
@kts

Re: Summary of IRC Meeting in #aurora

2015-08-17 Thread Kevin Sweeney

Thanks for sending this out Josh!

On Monday, August 17, 2015, Joshua Cohen jco...@twopensource.com wrote:

 No one with karma was around to kick off an official meeting, so we had an
 ad-hoc meeting. Here's the transcript:

 11:01 jcohen: ASFBot: meeting start
 11:01 ASFBot: You don't have enough karma for this request
 11:01 jcohen: didn’t think so!
 11:01 zmanji: ASFBot: meeting start
 11:01 ASFBot: You don't have enough karma for this request
 11:02 zmanji: I really thought jake added me to the karma list
 11:02 zmanji: ASFBot meeting start
 11:02 zmanji: dlester can you karma us here?
 11:02 zmanji: ASFBot: meeting start
 11:02 ASFBot: You don't have enough karma for this request
 11:02 jcohen: Let’s just do an ad-hoc meeting
 11:02 jcohen: we can manually send notes?
 11:03 jcohen: Let’s start with roll-call, why not.
 11:03 jcohen: here :)
 11:03 rdelvalle: here
 11:03 zmanji: here
 11:03 mkhutornenko: here
 11:04 thalin: here
 11:04 jcohen: #topic MesosCon
 11:04 jcohen: Several Aurora contributors will be at MesosCon this week,
 giving Aurora-related talks.
 11:05 jcohen: If you’re going to be at MesosCon, definitely look them up!
 11:05 jcohen: Anyone else have any topics?
 11:05 zmanji: #topic Forking Twitter Commons
 11:05 Yasumoto: I'm speaking on Thursday! :)
 11:05 zmanji: the IP Grant was given to fork Twitter Commons into our tree
 11:06 Yasumoto: [off] too late, sorry :P
 11:06 zmanji: look out for a patch this week that does that and follow up
 patches that upgrade guice and guava
 11:06 Yasumoto: excellent
 11:06 zmanji: I’m done
 11:06 thalin: GSOC update? That's about finished, correct?
 11:07 jcohen: Is there a GSOC update?
 11:07 jcohen: I believe Willy’s making good progress
 11:07 thalin: I just remember seeing emails about it recently
 11:08 jcohen: Ok, if that’s it I’ll call an end to the meeting
 11:08 jcohen: have a good week everyone!
 11:09 mkhutornenko: wait, one more topic
 11:09 jcohen: ok!
 11:09 mkhutornenko: #topic Mesos 0.23.0
 11:09 mkhutornenko: Mesos 0.23.0 enforces a new runtime dependency on
 MesosExecutorDriver
 11:09 mkhutornenko: FYI: You will need libcurl4-nss-dev installed on every
 slave
 11:10 mkhutornenko: More details here:
 http://markmail.org/thread/g6kkiapkartyvqn4
 11:10 mkhutornenko: Aurora RB: https://reviews.apache.org/r/37379/
 11:10 mkhutornenko: end of topic
 11:11 jcohen: Thanks Maxim
 11:11 jcohen: Last chance for topics
 11:12 jcohen: now if only I could get a decently formatted transcript out
 of colloquy!



-- 
Sent from Gmail Mobile

[PROPOSAL] Changes to the Python BUILD layout

2015-07-30 Thread Kevin Sweeney

I propose a simplification of the Python BUILD layout as follows and a set
of new conventions, as follows:

1) 1 BUILD per 3rd level directory. These are currently

```
% find src/main/python -maxdepth 3 -mindepth 3 -type d |while read dirname;
do echo $dirname | sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@';
done
apache.aurora.client
apache.aurora.common
apache.aurora.tools
apache.aurora.admin
apache.aurora.executor
apache.aurora.config
apache.thermos.monitoring
apache.thermos.common
apache.thermos.cli
apache.thermos.testing
apache.thermos.core
apache.thermos.runner
apache.thermos.observer
apache.thermos.config
```

2) Each BUILD file exports 1 python_library that provides a setup_py
containing each python_binary in the BUILD file, named the same as the
directory it's in so that it can be referenced without a ':' character. The
sources field in the python_binary will always be rglobs('*.py').

3) Other BUILD files may only depend on this single public python_library
target. Any other target is considered a private implementation detail and
should be prefixed with an _.

4) python_binary targets are always named the same as the exported console
script.

5) python_binary targets must have identical dependencies to the
python_library exported by the package and must use entry_point.

The advantage of this change is that a PEX file generated by pants will
contain exactly the same files that will be available on the PYTHONPATH in
the case of pip-installation of the corresponding library target. This will
help our migration off pants in the future.

Annotated example: apache.thermos.runner (renamed thermos/bin -
thermos/runner)

```
% find src/main/python/apache/thermos/runner
src/main/python/apache/thermos/runner
src/main/python/apache/thermos/runner/__init__.py
src/main/python/apache/thermos/runner/thermos_runner.py
src/main/python/apache/thermos/runner/BUILD
% cat src/main/python/apache/thermos/runner/BUILD
# License boilerplate omitted
import os


# Private target so that a setup_py can exist without a circular
dependency. Only targets within this file should depend on
# this.
python_library(
  name = '_runner',
  # The target covers every python file under this directory and
subdirectories.
  sources = rglobs('*.py'),
  dependencies = [
'3rdparty/python:twitter.common.app',
'3rdparty/python:twitter.common.log',
# Source dependencies are always referenced without a ':'.
'src/main/python/apache/thermos/common',
'src/main/python/apache/thermos/config',
'src/main/python/apache/thermos/core',
  ],
)

# Binary target for thermos_runner.pex. Nothing should depend on this -
it's only used as an argument to ./pants binary.
python_binary(
  name = 'thermos_runner',
  # Use entry_point, not source so the files used here are the same ones
tests see.
  entry_point = 'apache.thermos.bin.thermos_runner',
  dependencies = [
# Notice that we depend only on the single private target from this
BUILD file here.
':_runner',
  ],
)

# The public library that everyone importing the runner symbols uses.
# The test targets and any other dependent source code should depend on
this.
python_library(
  name = 'runner',
  dependencies = [
# Again, notice that we depend only on the single private target from
this BUILD file here.
':_runner',
  ],
  # We always provide a setup_py. This will cause any dependee libraries to
automatically reference this library
  # in their requirements.txt rather than copy the source files into their
sdist.
  provides = setup_py(
# Conventionally named and versioned.
name = 'apache.thermos.runner',
version = open(os.path.join(get_buildroot(),
'.auroraversion')).read().strip().upper(),
  ).with_binaries({
# Every binary in this file should also be repeated here.
# Always use the dict-form of .with_binaries so that commands with
dashes in their names are supported.
# The console script name is always the same as the PEX with .pex
stripped.
'thermos_runner': ':thermos_runner',
  }),
)
```

Let me know what you think, if y'all agree I'll prepare a patch shortly.

Re: [PROPOSAL] Changes to the Python BUILD layout

2015-07-30 Thread Kevin Sweeney

Posted https://reviews.apache.org/r/36972/ for feedback.

On Thu, Jul 30, 2015 at 5:17 PM, Bill Farner wfar...@apache.org wrote:

 +1

 -=Bill

 On Thu, Jul 30, 2015 at 4:24 PM, Kevin Sweeney
 kswee...@twitter.com.invalid
  wrote:

  Somewhat complementary work - this change would make the generated
 packages
  equivalent to the ultimate goal of replacing these top-level BUILD files
  directly with setup.py files. Users could begin testing deployment with
 the
  pants-generated sdists and seamlessly drop in setup.py-generated
  replacements. Users wishing to deploy via PEX could use the pex tool on
 the
  generated sdists rather than the pants pex plugin.
 
  On Thu, Jul 30, 2015 at 4:13 PM, Bill Farner wfar...@apache.org wrote:
 
   We've talked in the past about switching build tools.  Just to keep
 that
  in
   context - how would you weigh this effort against a tool change?
  
   -=Bill
  
   On Thu, Jul 30, 2015 at 4:00 PM, Kevin Sweeney kevi...@apache.org
  wrote:
  
I propose a simplification of the Python BUILD layout as follows and
 a
   set
of new conventions, as follows:
   
1) 1 BUILD per 3rd level directory. These are currently
   
```
% find src/main/python -maxdepth 3 -mindepth 3 -type d |while read
   dirname;
do echo $dirname | sed 's@src
   /main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@
';
done
apache.aurora.client
apache.aurora.common
apache.aurora.tools
apache.aurora.admin
apache.aurora.executor
apache.aurora.config
apache.thermos.monitoring
apache.thermos.common
apache.thermos.cli
apache.thermos.testing
apache.thermos.core
apache.thermos.runner
apache.thermos.observer
apache.thermos.config
```
   
2) Each BUILD file exports 1 python_library that provides a setup_py
containing each python_binary in the BUILD file, named the same as
 the
directory it's in so that it can be referenced without a ':'
 character.
   The
sources field in the python_binary will always be rglobs('*.py').
   
3) Other BUILD files may only depend on this single public
  python_library
target. Any other target is considered a private implementation
 detail
   and
should be prefixed with an _.
   
4) python_binary targets are always named the same as the exported
   console
script.
   
5) python_binary targets must have identical dependencies to the
python_library exported by the package and must use entry_point.
   
The advantage of this change is that a PEX file generated by pants
 will
contain exactly the same files that will be available on the
 PYTHONPATH
   in
the case of pip-installation of the corresponding library target.
 This
   will
help our migration off pants in the future.
   
Annotated example: apache.thermos.runner (renamed thermos/bin -
thermos/runner)
   
```
% find src/main/python/apache/thermos/runner
src/main/python/apache/thermos/runner
src/main/python/apache/thermos/runner/__init__.py
src/main/python/apache/thermos/runner/thermos_runner.py
src/main/python/apache/thermos/runner/BUILD
% cat src/main/python/apache/thermos/runner/BUILD
# License boilerplate omitted
import os
   
   
# Private target so that a setup_py can exist without a circular
dependency. Only targets within this file should depend on
# this.
python_library(
  name = '_runner',
  # The target covers every python file under this directory and
subdirectories.
  sources = rglobs('*.py'),
  dependencies = [
'3rdparty/python:twitter.common.app',
'3rdparty/python:twitter.common.log',
# Source dependencies are always referenced without a ':'.
'src/main/python/apache/thermos/common',
'src/main/python/apache/thermos/config',
'src/main/python/apache/thermos/core',
  ],
)
   
# Binary target for thermos_runner.pex. Nothing should depend on
 this -
it's only used as an argument to ./pants binary.
python_binary(
  name = 'thermos_runner',
  # Use entry_point, not source so the files used here are the same
  ones
tests see.
  entry_point = 'apache.thermos.bin.thermos_runner',
  dependencies = [
# Notice that we depend only on the single private target from
 this
BUILD file here.
':_runner',
  ],
)
   
# The public library that everyone importing the runner symbols uses.
# The test targets and any other dependent source code should depend
 on
this.
python_library(
  name = 'runner',
  dependencies = [
# Again, notice that we depend only on the single private target
  from
this BUILD file here.
':_runner',
  ],
  # We always provide a setup_py. This will cause any dependee
  libraries
   to
automatically reference this library
  # in their requirements.txt rather than copy the source files into
   their
sdist.
  provides = setup_py

Re: [PROPOSAL] Changes to the Python BUILD layout

2015-07-30 Thread Kevin Sweeney

Somewhat complementary work - this change would make the generated packages
equivalent to the ultimate goal of replacing these top-level BUILD files
directly with setup.py files. Users could begin testing deployment with the
pants-generated sdists and seamlessly drop in setup.py-generated
replacements. Users wishing to deploy via PEX could use the pex tool on the
generated sdists rather than the pants pex plugin.

On Thu, Jul 30, 2015 at 4:13 PM, Bill Farner wfar...@apache.org wrote:

 We've talked in the past about switching build tools.  Just to keep that in
 context - how would you weigh this effort against a tool change?

 -=Bill

 On Thu, Jul 30, 2015 at 4:00 PM, Kevin Sweeney kevi...@apache.org wrote:

  I propose a simplification of the Python BUILD layout as follows and a
 set
  of new conventions, as follows:
 
  1) 1 BUILD per 3rd level directory. These are currently
 
  ```
  % find src/main/python -maxdepth 3 -mindepth 3 -type d |while read
 dirname;
  do echo $dirname | sed 's@src
 /main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@
  ';
  done
  apache.aurora.client
  apache.aurora.common
  apache.aurora.tools
  apache.aurora.admin
  apache.aurora.executor
  apache.aurora.config
  apache.thermos.monitoring
  apache.thermos.common
  apache.thermos.cli
  apache.thermos.testing
  apache.thermos.core
  apache.thermos.runner
  apache.thermos.observer
  apache.thermos.config
  ```
 
  2) Each BUILD file exports 1 python_library that provides a setup_py
  containing each python_binary in the BUILD file, named the same as the
  directory it's in so that it can be referenced without a ':' character.
 The
  sources field in the python_binary will always be rglobs('*.py').
 
  3) Other BUILD files may only depend on this single public python_library
  target. Any other target is considered a private implementation detail
 and
  should be prefixed with an _.
 
  4) python_binary targets are always named the same as the exported
 console
  script.
 
  5) python_binary targets must have identical dependencies to the
  python_library exported by the package and must use entry_point.
 
  The advantage of this change is that a PEX file generated by pants will
  contain exactly the same files that will be available on the PYTHONPATH
 in
  the case of pip-installation of the corresponding library target. This
 will
  help our migration off pants in the future.
 
  Annotated example: apache.thermos.runner (renamed thermos/bin -
  thermos/runner)
 
  ```
  % find src/main/python/apache/thermos/runner
  src/main/python/apache/thermos/runner
  src/main/python/apache/thermos/runner/__init__.py
  src/main/python/apache/thermos/runner/thermos_runner.py
  src/main/python/apache/thermos/runner/BUILD
  % cat src/main/python/apache/thermos/runner/BUILD
  # License boilerplate omitted
  import os
 
 
  # Private target so that a setup_py can exist without a circular
  dependency. Only targets within this file should depend on
  # this.
  python_library(
name = '_runner',
# The target covers every python file under this directory and
  subdirectories.
sources = rglobs('*.py'),
dependencies = [
  '3rdparty/python:twitter.common.app',
  '3rdparty/python:twitter.common.log',
  # Source dependencies are always referenced without a ':'.
  'src/main/python/apache/thermos/common',
  'src/main/python/apache/thermos/config',
  'src/main/python/apache/thermos/core',
],
  )
 
  # Binary target for thermos_runner.pex. Nothing should depend on this -
  it's only used as an argument to ./pants binary.
  python_binary(
name = 'thermos_runner',
# Use entry_point, not source so the files used here are the same ones
  tests see.
entry_point = 'apache.thermos.bin.thermos_runner',
dependencies = [
  # Notice that we depend only on the single private target from this
  BUILD file here.
  ':_runner',
],
  )
 
  # The public library that everyone importing the runner symbols uses.
  # The test targets and any other dependent source code should depend on
  this.
  python_library(
name = 'runner',
dependencies = [
  # Again, notice that we depend only on the single private target from
  this BUILD file here.
  ':_runner',
],
# We always provide a setup_py. This will cause any dependee libraries
 to
  automatically reference this library
# in their requirements.txt rather than copy the source files into
 their
  sdist.
provides = setup_py(
  # Conventionally named and versioned.
  name = 'apache.thermos.runner',
  version = open(os.path.join(get_buildroot(),
  '.auroraversion')).read().strip().upper(),
).with_binaries({
  # Every binary in this file should also be repeated here.
  # Always use the dict-form of .with_binaries so that commands with
  dashes in their names are supported.
  # The console script name is always the same as the PEX with .pex
  stripped.
  'thermos_runner': ':thermos_runner

Re: [PROPOSAL] Source releases vs artifact releases

2015-07-27 Thread Kevin Sweeney

+1

On Mon, Jul 27, 2015 at 3:40 PM, Bill Farner wfar...@apache.org wrote:

 Hi folks,

 An issue that we have not yet officially addressed is release management as
 it pertains to binary artifacts we produce of Aurora.  Today, when we cut a
 release, say 0.9.0, we essentially take a snapshot of (most of) our
 repository as a basis for voting and eventual distribution.

 An outstanding question thus far has been how a release is affected when we
 need to change scripts and configurations for binary distributions [1] to
 produce a working binary artifact.  By some standards, a bug fix to an RPM
 spec might require another official source release/vote.

 I've had several useful discussions with Jake Farrell about this, and we
 brought the discussion to the asfinfra hipchat room to hopefully get some
 quick guidance from someone on the ASF board.  Please see the quote below
 if you would like to see the transcript.

 The summary is that the board allows us to produce binaries as we see fit,
 as the ASF does not consider them official releases.  As such, i propose
 that we treat build-support/packaging as distinct from the sources we vote
 on for a source release.  I further propose that we omit
 build-support/packaging from our source distributions.  This will make it
 clear that they are not part of what we are voting on when we cut a
 release.

 With this distinction, i would like for us to adopt the practice of
 considering binary distributions 'downstream' from source distributions,
 and bug fixes to packaging do not require a new source distribution
 release.


 Cheers,

 Bill


 [1]

 https://git-wip-us.apache.org/repos/asf?p=aurora.git;a=tree;f=build-support/packaging;h=c77d9b8ab2d8e6ecea2d8028bcdb250239240ffe;hb=HEAD

 Jake Farrell 12:03 PM
  question for the greater audience about packaging, we just cut and rc and
  voted on it, successfully passed. as part of the src release was code to
  create deb and rpm packages. went to cut the rpm's to vote on bin
 artifacts
  and there is a bug in the rpm spec
 
  Daniel Gruno (Humbedooh) 12:04 PM
  burn and reroll? :)
 
  Jake Farrell 12:05 PM
  would the recommendation be to cut a new rc or would having the deb/rpm
  packaging code in separate repo on its own release be more in line. i.e.,
  package 0.1.0-2.rpm in packaging land
 
  Daniel Gruno (Humbedooh) 12:07 PM
  you could cut a new release and bypass the 72h rule
 
  Bill Farner 12:09 PM
  a goal i'm seeking with this is to decouple source release from binary
  releases, if possible. that way we don't have things like releases for N
  distros that are no-ops because we fixed something in 1. it also means
 that
  we don't have source releases that have no binary releases because of a
  packaging spec bug
  we're currently in the latter situation - we have a perfectly fine 0.9.0
  src release. however, it turns out there's trivial cruft in packaging
 specs
  requiring a post-0.9.0 commit to generate working binaries (key detail -
  commit that does not touch the source)
 
  Daniel Gruno (Humbedooh) 12:12 PM
  aren't binaries still viewed as unofficial convenience?
  iow you can do what you like to it, 'cause it ain't ours
 
  Tony Stevenson ( pctony ) 12:13 PM
  AIUI, yes
  but $AOO etc
 
  Daniel Gruno (Humbedooh) 12:13 PM
  @rbowen you're a board pony, what say you?
 
  Tony Stevenson ( pctony ) 12:13 PM
  neeeigh
 
  Daniel Gruno (Humbedooh) 12:13 PM
  apart from neigh, that is
 
  Rich Bowen 12:13 PM
  Hmm. What?
 
  Daniel Gruno (Humbedooh) 12:14 PM
  bill has a question on ASF binary release policy
 
  Bill Farner 12:14 PM
  details in scrollback, let me know if i've worded poorly :-)
  + Jake's context a few messages before
  concrete example: this line in our rpm spec is incorrect
 
 https://github.com/apache/aurora/blob/master/build-support/packaging/rpm/aurora.spec#L188
  fixing it does not alter what i'd consider the source code of the
 project,
  just the tooling to assemble those sources
 
  Daniel Gruno (Humbedooh) 12:18 PM
  personally, I would favor burning the release and cutting a new with a
  speed vote
  so that people building from source can generate rpms as well
 
  Rich Bowen 12:19 PM
  I'm not certain what the policy would be in this specific case, since
  binary releases aren't official releases for anybody but OO, but I would
  say that if there's a question, you cut another release and fasttrack the
  vote to put it out there, and eliminate any question.
  Which ... appears to be what @Humbedooh just said.
 
  Daniel Gruno (Humbedooh) 12:20 PM
  72 hour rule can be voided (under pain of Greg's fists if you do
 something
  bad)
 
  Bill Farner 12:20 PM
  so for the end-user, that seems to mean people on different distros could
  have identical software, but be many releases apart if we had to iterate
 on
  one distro's packaging
  if so, that scenario seems unfortunate for users
  (or they could be on the same distro for that matter)
 
  Daniel Gruno (Humbedooh) 12:21 PM

Re: Forking twitter-commons into our tree

2015-07-06 Thread Kevin Sweeney

+1, I suspect we'll find several things that can be replaced by the Java 8
standard library or newer versions of Guava and Guice.

On Mon, Jul 6, 2015 at 11:20 AM, Zameer Manji zma...@apache.org wrote:

 Just to be clear, I'm proposing forking the java parts only.

 On Mon, Jul 6, 2015 at 9:06 AM, Joseph Smith yasumo...@gmail.com wrote:

  Also a (tough to concede) +1. Although I’m not a fan of the fork, it will
  help improve velocity and empower a migration away from twitter common.
 
   On Jul 3, 2015, at 8:15 PM, Bill Farner wfar...@apache.org wrote:
  
   That's roughly the eventual plan, which this move would help us
  facilitate.
   We use guava heavily already, most of our current dependence is on ZK
  and args handling code...but we would look towards dep-shallow
 alternatives.
  
  
  
  _
   From: Chris Aniszczyk caniszc...@gmail.com
   Sent: Friday, July 3, 2015 8:03 AM
   Subject: Re: Forking twitter-commons into our tree
   To:  dev@aurora.apache.org
   Cc: Jake Farrell jfarr...@apache.org
  
  
   I'll see what I can do about IP clearance.
  
   For giggles, how much work do you think it would be to shed
  twitter-commons
   and just rely on guava and other what I would consider more standard
   libraries.
  
   On Thu, Jul 2, 2015 at 10:34 PM, Bill Farner wfar...@apache.org
 wrote:
  
   Thanks, Jake!
  
   -=Bill
  
   On Thu, Jul 2, 2015 at 8:10 PM, Jake Farrell jfarr...@apache.org
  wrote:
  
   yes, makes it easier to donate when its Apache License 2.0, but still
   requires the IP clearance [1], which is handled through the IPMC.
 This
  is
   required so there is an audit trail of that software being donated to
  the
   ASF
  
   -Jake
  
   [1]: http://incubator.apache.org/ip-clearance/index.html
  
  
  
   On Thu, Jul 2, 2015 at 10:41 PM, Bill Farner wfar...@apache.org
  wrote:
  
   Jake - i'm not fully versed on licenses, but is that true even
 though
   it's
   all Apache License 2.0?
  
   -=Bill
  
   On Thu, Jul 2, 2015 at 5:28 PM, Jake Farrell jfarr...@apache.org
   wrote:
  
   no objections, but we would have to get an IP clearance doc from
   Twitter
   for this code in order to bring this code into the ASF
  
   -Jake
  
   On Thu, Jul 2, 2015 at 3:20 PM, Zameer Manji zma...@apache.org
   wrote:
  
   Hey,
  
   Aurora depends heavily on twitter-commons for lots of
   functionality.
   However upstream is not very active and I suspect that it will be
   less
   active in the future. Currently we depend on artifacts published
   from
   this
   project which causes us to depend on older versions of guava and
   guice.
  
   As a result, it seems that will be difficult to address tickets
   like
   AURORA-1380 https://issues.apache.org/jira/browse/AURORA-1380
   without
   changing something. I propose we fork all of the java portions of
   twitter-commons into our tree, remove the parts we don't use and
   update
   guava and guice so we can move forward on this front.
  
   What are people's thoughts on this?
  
   --
   Zameer Manji
  
  
  
  
  
  
  
  
   --
   Cheers,
  
   Chris Aniszczyk
   http://aniszczyk.org
   +1 512 961 6719
 
  --
  Zameer Manji
 
  %2B1%20512%20961%206719

Re: [jira] [Created] (AURORA-1375) When setting quota, the scheduler should prevent setting below current usage

2015-06-26 Thread Kevin Sweeney

This should be much easier now that we're using a relational database - we
can add a CHECK constraint.

On Fri, Jun 26, 2015 at 11:21 AM, Joe Smith (JIRA) j...@apache.org wrote:

 Joe Smith created AURORA-1375:
 -

  Summary: When setting quota, the scheduler should prevent
 setting below current usage
  Key: AURORA-1375
  URL: https://issues.apache.org/jira/browse/AURORA-1375
  Project: Aurora
   Issue Type: Story
   Components: Scheduler
 Reporter: Joe Smith


 Initially I thought this check should live in the admin client, but after
 discussing with [~zmanji], we believe the scheduler would be the right
 place to enforce this.



 --
 This message was sent by Atlassian JIRA
 (v6.3.4#6332)




-- 
Kevin Sweeney
@kts

Re: Culling items from 0.9.0 release

2015-06-23 Thread Kevin Sweeney

+1

On Tuesday, June 23, 2015, Bill Farner wfar...@apache.org wrote:

 Hey folks,

 I've moved the following tickets from the 0.9.0 release to the 0.10.0
 release so that we may release in a more timely manner:

 AURORA-334 Replace in-memory storage with H2
 This change is important, but ~entirely behind the scenes for now and
 should not hold up a release.

 AURORA-987 Create a first-class REST-like scheduler interface
 This is a big feature for users, but has no momentum yet and is therefore
 likely to take a while.

 AURORA-1229 Remove SessionKey struct and related code
 As discussed in the IRC meeting yesterday, we would like to lengthen the
 deprecation period on this, which has little downside.

 Please don't hesitate to speak up if you disagree with any of these moves!


 -=Bill



-- 
Sent from Gmail Mobile

Re: [VOTE] Release Apache Aurora 0.8.0 API Artifacts

2015-06-18 Thread Kevin Sweeney

+1

On Thu, Jun 11, 2015 at 1:35 PM, Jake Farrell jfarr...@apache.org wrote:

 Nexus upload automatically does that

 -Jake

 On Thu, Jun 11, 2015 at 4:33 PM, Kevin Sweeney kevi...@apache.org wrote:

 It looks like there are checksums of the signatures, is that intentional?

 On Thu, Jun 11, 2015 at 1:31 PM, Jake Farrell jfarr...@apache.org
 wrote:

 All,
 I have packaged up and staged the API jar/source artifacts from Apache
 Aurora 0.8.0
 and deployed them to the Apache staging repo [1]. I propose that we
 accept
 this
 staging repo as part of the official Apache Aurora 0.8.0 release
 artifacts.

 Aurora 0.8.0 API artifacts staging repo details include:
 ---
 The branch used to create the staged release candidate artifacts:

 https://git-wip-us.apache.org/repos/asf?p=aurora.git;a=shortlog;h=refs/heads/0.8.0

 The staged release candidate artifacts are available at:
 https://repository.apache.org/content/repositories/orgapacheaurora-1002

 The GPG key used to sign the release artifacts are available at:
 https://dist.apache.org/repos/dist/dev/aurora/KEYS

 Please download, verify, and test.

 The vote will close on Sun Jun 14 17:00:00 EDT 2015

 [ ] +1 Release this as Apache Aurora 0.8.0 API Artifacts
 [ ] +0
 [ ] -1 Do not release this as Apache Aurora 0.8.0 API Artifacts
 because...

 I would like to get the voting started with my own +1


 [1]:
 https://repository.apache.org/content/repositories/orgapacheaurora-1002

Re: [VOTE] Release Apache Aurora 0.8.0 API Artifacts

2015-06-11 Thread Kevin Sweeney

It looks like there are checksums of the signatures, is that intentional?

On Thu, Jun 11, 2015 at 1:31 PM, Jake Farrell jfarr...@apache.org wrote:

 All,
 I have packaged up and staged the API jar/source artifacts from Apache
 Aurora 0.8.0
 and deployed them to the Apache staging repo [1]. I propose that we accept
 this
 staging repo as part of the official Apache Aurora 0.8.0 release artifacts.

 Aurora 0.8.0 API artifacts staging repo details include:
 ---
 The branch used to create the staged release candidate artifacts:

 https://git-wip-us.apache.org/repos/asf?p=aurora.git;a=shortlog;h=refs/heads/0.8.0

 The staged release candidate artifacts are available at:
 https://repository.apache.org/content/repositories/orgapacheaurora-1002

 The GPG key used to sign the release artifacts are available at:
 https://dist.apache.org/repos/dist/dev/aurora/KEYS

 Please download, verify, and test.

 The vote will close on Sun Jun 14 17:00:00 EDT 2015

 [ ] +1 Release this as Apache Aurora 0.8.0 API Artifacts
 [ ] +0
 [ ] -1 Do not release this as Apache Aurora 0.8.0 API Artifacts because...

 I would like to get the voting started with my own +1


 [1]:
 https://repository.apache.org/content/repositories/orgapacheaurora-1002

Re: [VOTE] Release Apache Aurora 0.8.0 RC1

2015-05-08 Thread Kevin Sweeney

+1

Changelog looks good.
Release candidate passed the verification script.

On Fri, May 8, 2015 at 12:37 PM, Bill Farner wfar...@apache.org wrote:

 +1

 Passed ./build-support/release/verify-release-candidate 0.8.0-rc1 from
 master, which covers signature checks, unit tests, and integration tests.

 Thanks, Jake!

 -=Bill

 On Thu, May 7, 2015 at 9:23 PM, Justin Mclean jus...@classsoftware.com
 wrote:

  Hi,
 
  Sorry ignore that I got my lists mixed up and thought the VOTE was sent
 to
  general@incubator.
 
  Thanks,
  Justin




-- 
Kevin Sweeney
@kts

Re: [VOTE] Release Apache Aurora 0.8.0 RC0

2015-05-05 Thread Kevin Sweeney

overall +0 - I'm okay with proceeding with the changelog as the only
deficiency

+1 java tests pass
+1 python test pass
+1 e2e tests pass
+0 changelog is incorrect as wfarner notes



On Tue, May 5, 2015 at 3:36 PM, Bill Farner wfar...@apache.org wrote:

 -1 overall

 +1 checksums and signatures are good
 +1 unit tests and end-to-end tests pass
 -1 changelog contains tickets that were closed as won't fix.  i came up
 with a JIRA query [1] that seems to identify a bunch of these

 [1]

 https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20AND%20fixVersion%3D0.8.0%20and%20resolution!%3Dfixed



 -=Bill

 On Mon, May 4, 2015 at 7:41 PM, Jake Farrell jfarr...@apache.org wrote:

  All,
  I propose that we accept the following release candidate as the official
  Apache Aurora 0.8.0 release.
 
  Aurora 0.8.0-rc0 includes the following:
  ---
  The CHANGELOG for the release is available at:
 
 
 https://git-wip-us.apache.org/repos/asf?p=aurora.gitf=CHANGELOGhb=0.8.0-rc0
 
  The branch used to create the release candidate is:
 
 
 https://git-wip-us.apache.org/repos/asf?p=aurora.git;a=shortlog;h=refs/heads/0.8.0-rc0
 
  The release candidate is available at:
 
 
 https://dist.apache.org/repos/dist/dev/aurora/0.8.0-rc0/apache-aurora-0.8.0-rc0.tar.gz
 
  The MD5 checksum of the release candidate can be found at:
 
 
 https://dist.apache.org/repos/dist/dev/aurora/0.8.0-rc0/apache-aurora-0.8.0-rc0.tar.gz.md5
 
  The signature of the release candidate can be found at:
 
 
 https://dist.apache.org/repos/dist/dev/aurora/0.8.0-rc0/apache-aurora-0.8.0-rc0.tar.gz.asc
 
  The GPG key used to sign the release are available at:
  https://dist.apache.org/repos/dist/dev/aurora/KEYS
 
  Please download, verify, and test.
 
  The vote will close on Thu May  7 22:35:17 EDT 2015
 
  [ ] +1 Release this as Apache Aurora 0.8.0
  [ ] +0
  [ ] -1 Do not release this as Apache Aurora 0.8.0 because...
 
  I would like to get the voting started with my own +1
 
  -Jake
 




-- 
Kevin Sweeney
@kts

Re: [VOTE] Release Apache Aurora 0.8.0 RC0

2015-05-05 Thread Kevin Sweeney

overall +0 - I'm okay with proceeding with the changelog as the only
deficiency

+1 java tests pass
+1 python test pass
+1 e2e tests pass
+0 changelog is incorrect as wfarner notes

On Tue, May 5, 2015 at 3:36 PM, Bill Farner wfar...@apache.org wrote:

 -1 overall

 +1 checksums and signatures are good
 +1 unit tests and end-to-end tests pass
 -1 changelog contains tickets that were closed as won't fix.  i came up
 with a JIRA query [1] that seems to identify a bunch of these

 [1]

 https://issues.apache.org/jira/issues/?jql=project%20%3D%20AURORA%20AND%20fixVersion%3D0.8.0%20and%20resolution!%3Dfixed



 -=Bill

 On Mon, May 4, 2015 at 7:41 PM, Jake Farrell jfarr...@apache.org wrote:

  All,
  I propose that we accept the following release candidate as the official
  Apache Aurora 0.8.0 release.
 
  Aurora 0.8.0-rc0 includes the following:
  ---
  The CHANGELOG for the release is available at:
 
 
 https://git-wip-us.apache.org/repos/asf?p=aurora.gitf=CHANGELOGhb=0.8.0-rc0
 
  The branch used to create the release candidate is:
 
 
 https://git-wip-us.apache.org/repos/asf?p=aurora.git;a=shortlog;h=refs/heads/0.8.0-rc0
 
  The release candidate is available at:
 
 
 https://dist.apache.org/repos/dist/dev/aurora/0.8.0-rc0/apache-aurora-0.8.0-rc0.tar.gz
 
  The MD5 checksum of the release candidate can be found at:
 
 
 https://dist.apache.org/repos/dist/dev/aurora/0.8.0-rc0/apache-aurora-0.8.0-rc0.tar.gz.md5
 
  The signature of the release candidate can be found at:
 
 
 https://dist.apache.org/repos/dist/dev/aurora/0.8.0-rc0/apache-aurora-0.8.0-rc0.tar.gz.asc
 
  The GPG key used to sign the release are available at:
  https://dist.apache.org/repos/dist/dev/aurora/KEYS
 
  Please download, verify, and test.
 
  The vote will close on Thu May  7 22:35:17 EDT 2015
 
  [ ] +1 Release this as Apache Aurora 0.8.0
  [ ] +0
  [ ] -1 Do not release this as Apache Aurora 0.8.0 because...
 
  I would like to get the voting started with my own +1
 
  -Jake

Re: Moving forward on AURORA-1503

Re: [DRAFT] [REPORT] Apache Aurora

Re: Creating Aurora user@ and announcements@ lists?

Re: [VOTE] Release Apache Aurora 0.9.0 debs

Re: [VOTE] Release Apache Aurora 0.9.0 debs

Re: [VOTE] Release Apache Aurora 0.9.0 debs

Re: Summary of IRC Meeting in #aurora

[PROPOSAL] Changes to the Python BUILD layout

Re: [PROPOSAL] Changes to the Python BUILD layout

Re: [PROPOSAL] Changes to the Python BUILD layout

Re: [PROPOSAL] Source releases vs artifact releases

Re: Forking twitter-commons into our tree

Re: [jira] [Created] (AURORA-1375) When setting quota, the scheduler should prevent setting below current usage

Re: Culling items from 0.9.0 release

Re: [VOTE] Release Apache Aurora 0.8.0 API Artifacts

Re: [VOTE] Release Apache Aurora 0.8.0 API Artifacts

Re: [VOTE] Release Apache Aurora 0.8.0 RC1

Re: [VOTE] Release Apache Aurora 0.8.0 RC0

Re: [VOTE] Release Apache Aurora 0.8.0 RC0

19 matches

Site Navigation

Mail list logo

Footer information