Re: [DISCUSS][VOTE] Release Apache Brooklyn 0.10.0 [rc3]

[Aled Sage]

Hi all,

The PR [1] is merged, and cherry-picked into the 0.10.0 branch [2].
Is anything else needed before we kick off RC4? Has everyone who plans 
to test RC4 already done those tests against RC3 (or earlier RCs), so 
that we're as confident as we can be about this next RC?

Svet (or Richard): will you be available to build RC4?

Aled

[1] https://github.com/apache/brooklyn-server/pull/499
[2] https://github.com/apache/brooklyn-server/commits/0.10.0


On 21/12/2016 14:41, Aled Sage wrote:
+1 to Mike's comment in [CANCEL][VOTE] thread: we can leave vagrant 
as-is because it explicitly sets the "AnyoneSecurityProvider" [1]

I just tested this with a local build of brooklyn (with my PR #499 
changes) - it lets me connect without credentials.

Aled

[1] 
https://github.com/apache/brooklyn-dist/blob/0.10.0/vagrant/src/main/vagrant/files/brooklyn.properties#L21


On 21/12/2016 14:08, Aled Sage wrote:
Thanks all - please review 
https://github.com/apache/brooklyn-server/pull/499 for this change.

---
I still need to look at the vagrant configuration (stripping out the 
username:password, which is added to the config file).

Shout out if anyone else wants to pick that up.

---
I'll look at updating the docs as well, but that can be done after 
we've kicked off the RC.

Aled


On 21/12/2016 13:39, Svetoslav Neykov wrote:
+1, agree with Aled's suggestion.

Svet.


On 21.12.2016 г., at 15:05, Alex Heneveld 
<alex.henev...@cloudsoftcorp.com> wrote:

+1 ... default of password w/o https has always been a smell for me 
anyway.
unauth by default solves a lot of headaches.

the way the docker install sets up a pre-configured password is 
kinda nice

and/or w a switch to karaf we could look at being able to configure 
auth
etc in the product

--a


On 21 December 2016 at 12:50, Geoff Macartney <
geoff.macart...@cloudsoftcorp.com> wrote:

+1 to the proposal, as Aled says  I liked it during previous 
discussions.

Going forward to 0.11 I'd suggest we support just two modes of 
operation:
unauthenticated out-of-the-box for evaluation,  and authenticated 
with
https and jwt for production, with an easy process to get from the 
former
to the latter.


On Wed, 21 Dec 2016, 11:41 Richard Downer, <rich...@apache.org> 
wrote:

Hi Aled,

+1 to your proposal. In the wider view, many popular software 
projects are
unauthenticated (or fixed credentials) on a default install, and 
specific
hardening steps need to be followed before running the software on a
server.

I think authentication needs a bigger discussion post 0.10.0 - I have
several valid use cases from customers that Brooklyn does not 
currently
support.

Thanks
Richard.


On 21 December 2016 at 10:59, Aled Sage <aled.s...@gmail.com> wrote:

Hi all,

I lean towards us changing the existing behaviour for 0.10.0 (i.e.
producing an rc4 that fixes it).

It seems risky/complicated to revert to the "pre-JAAS 
authentication code
path". If there is another acceptable way, then I'd much prefer 
that.

**Proposal**
We disable password authentication entirely, as the default
configuration.
We ensure steps to configure security are well documented.

As per the very long email thread in [1], this is "Alternative 
Proposal
2"
in my email dated 2016-09-13 15:32.

This got Geoff's agreement (on 2016-09-13 16:25) and also Martin's
agreement (2016-09-13 16:56). John McCabe pointed out by way of
comparison
that Jenkins deploys out-of-the-box with no auth (2016-09-08 22:05).

The major benefits I see are:

1. Solves the immediate problem (i.e. backwards compatibility for
   localhost login with no password).
2. Super-simple user experience, including for installing on a VM
   (which currently still requires the horrible step of digging 
out the
   auto-generated username:password, buried in the log).
3. Consistency for localhost + remote (getting-started 
instructions can
   be simpler, rather than different depending how/where you've
   installed it. This has not been stressed enough in the 
discussions
   so far.)
4. Passwords are a false sense of security when using http (which is
   still the default), given that it would be sent in plain text.

We should also ensure that the vagrant install is consistent 
(e.g. it no
longer needs to have an auto-populated username:password in the 
config
file, and can instead rely on the default being no password).

**Longer Term**
For the next release (i.e. 0.11.0), we can continue the 
discussion for
what the user experience should be long-term (e.g. on first 
login, it
prompts for a username+password to be created automatically). This
includes
discussion of RPM installs: on `yum install ...`, the user 
doesn't have a
chance to modify the defaults in the config file before Brooklyn is
started
for the first time.

For 0.11.0, we re-visit the default Karaf configuration (which I 
believe
will still require the auto-generated username + password (buried 
in the
log), both for localhost and remote).

Aled

[1] https://lists.apache.org/thread.html/cf6f467cc63afc9bb7bb75d
7396becbd4ea2ac273031d31073f546d2@%3Cdev.brooklyn.apache.org%3E



On 20/12/2016 16:05, Richard Downer wrote:

Ok, I can reproduce this behaviour when trying to connect to a 
Brooklyn
instance on localhost.

*But...*

I've started up a new instance on AWS and I'm connecting to it 
on port
8081. A username and password is printed to the console on 
startup, and
I
can use those credentials to access the web UI. If I give the wrong
password, I am forbidden to access.

So it seems that this behaviour is unique to localhost. IMO this is
annoying but not a release blocker - but it would be nicer to 
our new
users
to have a fix.

The Brooklyn startup log says:
2016-12-20 15:54:32,668 INFO  Starting Brooklyn web-console with
passwordless access on localhost and protected access from any 
other
interfaces (no bind address specified)

So it seems that the "passwordless access on localhost" isn't quite
true.
In that case I'd go with Svet's suggestion to remove the localhost
exception.

Richard.

On 20 December 2016 at 15:08, Svetoslav Neykov <
svetoslav.ney...@cloudsoftcorp.com> wrote:

// Added DISCUSS tag
There's a long related [PROPOSAL] thread on the topic -
https://lists.apache.org/thread.html/cf6f467cc63afc9bb7bb75d7396bec 

bd4ea2ac273031d31073f546d2@%3Cdev.brooklyn.apache.org%3E <
https://lists.apache.org/thread.html/cf6f467cc63afc9bb7bb75d7396bec 

bd4ea2ac273031d31073f546d2@%3Cdev.brooklyn.apache.org%3E> which 
seems
to
have died off without consensus.

Anyone want to hazard a guess for how hard to fix BROOKLYN-417?
There are two possible fixes.
   * Require the autogenerated password - as simple as removing 
the
isRemoteAddressLocalhost[1] check.
   * Don't require a password. That's more complicated. It needs
reverting
to the pre-JAAS authentication code path, using a Filter to do the
authorization. Can't say right now whether we can just get some 
code
from
git history and plug it in or things have changed considerably,
assuming
JAAS under the hood.

Svet.

[1] https://github.com/apache/brooklyn-server/blob/master/
rest/rest-resources/src/main/java/org/apache/brooklyn/rest/
security/provider/BrooklynUserWithRandomPasswordSecurityProv
ider.java#L55

On 20.12.2016 г., at 16:48, Aled Sage <aled.s...@gmail.com> wrote:
Alex, all,

I've created 
https://issues.apache.org/jira/browse/BROOKLYN-417 to

track this - and have included details from my initial 
investigation.

I'm in two minds about whether it blocks the release. We 
expect people

to quickly move to supplying credentials (rather than Brooklyn
auto-generating a password). However, it does impact the 
initial user
experience (for those running on localhost, rather than vagrant 
etc) -
having to supply dummy username:password looks horrible. If we 
did ship
with this, we should probably update the docs to tell people to
configure
the credentials *before* starting brooklyn for the first time.

Anyone want to hazard a guess for how hard to fix BROOKLYN-417?

Aled


On 20/12/2016 14:06, Alex Heneveld wrote:

Hi Devs,

-0 ?  -1 ?

The "bin" build when run locally on a fresh system (no

brooklyn.properties)
requires that a username/password is supplied.  It doesn't do any
authentication of the local credentials but gives a 401 if not
supplied.
Specifically:

$ curl -v http://localhost:8081/ 2>&1 | grep "< HTTP"
< HTTP/1.1 401 Unauthorized

$ curl -u anyuser:passwordignored -v http://localhost:8081/ 
2>&1 |

grep "<
HTTP"
< HTTP/1.1 200 OK

I'm inclined to think this should block a release as it will 
break
any
workflow that expects local http to work, and will irritate 
new users
without any security benefits, but I could be talked in to 
forgiving
it.

--A


On 20 December 2016 at 13:41, Andrea Turli
<andrea.turli@cloudsoftcorp.
com>
wrote:
+1 (binding)
Tested by running Svet's verify script.

Do a clean extract of source repo for next steps.

-------------------------------------------

Checks successfully completed:
[✓] Download links work.
[✓] Checksums and PGP signatures are valid.
[✓] Expanded source archive matches contents of RC tag.
[✓] Expanded source archive builds and passes tests.
[✓] LICENSE is present and correct.
[✓] NOTICE is present and correct, including copyright date.
[✓] No compiled archives bundled in source archive.

Additional manual steps done (execute in source distribution 
folder
apache-brooklyn-0.10.0-src:
  * Check for files with invalid headers in source 
distribution.
There
are
already files excluded from RAT checks, do a sanity check.
  * Check for binary files in source distribution. Look for 
files
which

are
created/compiled based on other source files in the distribution.
"Primary"
binary files like images are acceptable.
Checks left to do manually with the help of above instructions:
[✓] All files have license headers where appropriate: I 
relied on
the
rat
check when running `mvn clean install` for checking that "all 
files
have
license headers where appropriate".
[-] All dependencies have compatible licenses.



On 19 December 2016 at 17:28, Svetoslav Neykov <
svetoslav.ney...@cloudsoftcorp.com> wrote:

Geoff,
I checked the checksums with the release verification 
script. Is
this
in
source control somewhere? I'll offer this to add to it:
See https://github.com/apache/brooklyn-dist/pull/65 <
https://github.com/apache/brooklyn-dist/pull/65> and in 
particular
https://github.com/neykov/brooklyn-dist/blob/
9e43edd4f7d65b0553ce41a0405e1be4c8abc364/release/verify_broo
klyn_rc.sh

<
https://github.com/neykov/brooklyn-dist/blob/
9e43edd4f7d65b0553ce41a0405e1be4c8abc364/release/verify_

brooklyn_rc.sh>
for an initial implementation of these kind of checks 
(obviously to be
expanded where possible).

Svet.


On 19.12.2016 г., at 18:17, Geoff Macartney <geoff.macartney@
cloudsoftcorp.com> wrote:

+1

[x] Download links work.
[x] Binaries work.
[x] Checksums and PGP signatures are valid.
[x] Expanded source archive matches contents of RC tag.
[x] Expanded source archive builds and passes tests.
[x] LICENSE is present and correct.
[x] NOTICE is present and correct, including copyright date.
[x] All files have license headers where appropriate.
[-] All dependencies have compatible licenses.
[x] No compiled archives bundled in source archive.
[x] I follow this project’s commits list.

I deployed the classic tarball and karaf versions to 
Ubuntu using

Java
8
and 7,
and the RPM to Centos 7, and was able to deploy sample
applications
to
AWS
and SL.

I checked the checksums with the release verification 
script. Is
this

in
source
control somewhere? I'll offer this to add to it:

    function licenses_and_notices () {
        local filename=$1
        local copyright='Copyright 2014-2016 The Apache 
Software

Foundation'

        for tarball in `ls *.tar.gz`; do
        container=$( tar tf $tarball | awk NR==1 )
        tar tf $tarball  | grep -q ${container}LICENSE
        tar tf $tarball  | grep -q ${container}NOTICE
        tar xOf $tarball ${container}NOTICE | grep -q
"${copyright}"
        done
    }

    licenses_and_notices

Notes:

I was able to deploy a WebServer + 3-node Riak demo app to
Softlayer
without problem,
and didn't see the "hostname is (none)" problem mentioned
previously
by
Svet.
I tested https access (with Karaf version) and access to 
it using

"br"
(from Mac) without
observing the crash Svet saw:

    ./br login https://10.10.10.102:8443 geoff password
    Get https://10.10.10.102:8443/v1/server/version: x509: 
cannot

validate

certificate for 10.10.10.102 because it doesn't contain 
any IP
SANs
    ./br --skipSslChecks login https://10.10.10.102:8443 
geoff

password
    Connected to Brooklyn version 0.10.0 at
https://10.10.10.102:8443

    ./br app
    Id   | Name   | Status   | Location


Tested the above on Linux and Mac.  I also tried 1000 
consecutive

logins
and didn't see the
segfault as mentioned in the comment on BROOKLYN-416. I 
don't know

why
Svet
is seeing these
problems; my own tests would indicate that br is working ok.
Anyone
else
able to say what
they are seeing?

Caveat:

I did see the behaviour that Svet describes on the 
main.uri on the

sample
applications.
I thought hard about whether this should prevent us 
releasing. On
the

one
hand, there
is a possible risk of a poor impression on users trying 
Brooklyn
for
the
first time if
its own sample apps appear not to work well. On the other 
hand
it's
fairly

clear just from
looking at the URL what network it is on, so it shouldn't 
really
be
too
confusing.  In
the end what persuaded me was checking back to 0.9.0, and 
this

behaviour
was already
there, so this is not a regression. Not that that matters 
as such,

but
as
far as I recall
there was no feedback on the mailing list about this, so 
it does
not
seem
to have actually
caused a problem for anyone in practice. I'm therefore happy
enough
to
say
+1.
At the same time I think it is something that we should 
tidy up
for
the
future.
Geoff





On Mon, 19 Dec 2016 at 11:51 Svetoslav Neykov <
svetoslav.ney...@cloudsoftcorp.com> wrote:

+1 (binding)
Caveats:
  * hostname on Softlayer-provisioned machines is "(none)"
  * On Softlayer App Wizard examples "Template 2: Python Web
Server",
"Template 3: Bash Web Server and Riak Cluster" list the 
private
IP
for
"main.uri" (with no "xxx.public/private" alternatives). It's 
easy to
think
that it's not working. If public IP is used it's 
reachable. Some
of
the
items have the sensor propagated to top, but it's still the 
private
IP
  * "main.uri" is not propagated to root app  (for most 
examples),
need
to
drill down to first child to get the sensors
  * br - getting fatal error sometimes (
https://issues.apache.org/jira/browse/BROOKLYN-416 <
https://issues.apache.org/jira/browse/BROOKLYN-416>). Not 
a new

problem
it seems.
  * br - can't use --skipSslChecks, still getting "Get
https://localhost:8443/v1/server/version: x509: 
certificate is

valid
for
web-console, not localhost" from Karaf dist.
  * If I log in into two localhost ports at the same time 
I get
CSRF

Token
errors from the page that got loaded first. Using them one 
by one
works
fine.
Could be easily convinced that item 2 from above doesn't 
deserve
a
+1,
thoughts?

-------------------


Tested by running the verify script from
https://github.com/neykov/brooklyn-dist/blob/

9e43edd4f7d65b0553ce41a0405e1be4c8abc364/release/verify_
brooklyn_rc.sh
<
https://github.com/neykov/brooklyn-dist/blob/

9e43edd4f7d65b0553ce41a0405e1be4c8abc364/release/verify_
brooklyn_rc.sh>
(https://github.com/apache/brooklyn-dist/pull/65 <
https://github.com/apache/brooklyn-dist/pull/65>).
Expanded the -bin & -karaf, running with local
brooklyn.properties.
Deployed the example apps to AWS & Softlayer (with 
preconfigured

locations).
Expanded the OS X br client and tried it against Classic 
(https) &
Karaf
(non-https);
[✓] Download links work.
[✓] Binaries work.
[✓] Checksums and PGP signatures are valid.
[✓] Expanded source archive matches contents of RC tag.
[✓] Expanded source archive builds and passes tests.
[✓] LICENSE is present and correct.
[✓] NOTICE is present and correct, including copyright date.
[✓] All files have license headers where appropriate.
[✓] All dependencies have compatible licenses.
[✓] No compiled archives bundled in source archive.
[-] I follow this project’s commits list.


Svet.


On 18.12.2016 г., at 19:14, Aled Sage <aled.s...@gmail.com>
wrote:
+1 (binding)

I downloaded the tar.gz, and ran it on OS X. I deployed

TomcatServer
to
AWS, Softlayer, Openstack (bluebox-london) and Google Compute
Engine.
I ran the verify_brooklyn_rc.sh script (see attachment in rc1
vote).
I relied on the rat check when running `mvn clean install` for
checking
that "all files have license headers where appropriate".
I checked that each tar.gz in [1] contained a top-level 
LICENSE
and

NOTICE file (including the text "Copyright 2014-2016 The 
Apache

Software
Foundation").
[-] Download links work.
[-] Binaries work.
[x] Checksums and PGP signatures are valid.
[-] Expanded source archive matches contents of RC tag.
[x] Expanded source archive builds and passes tests.
[x] LICENSE is present and correct.
[x] NOTICE is present and correct, including copyright 
date.
[x] All files have license headers where appropriate.
[-] All dependencies have compatible licenses.
[x] No compiled archives bundled in source archive.
[x] I follow this project’s commits list.

Aled

[1]

https://dist.apache.org/repos/dist/dev/brooklyn/apache-

brooklyn-0.10.0-rc3
On 16/12/2016 12:34, Svetoslav Neykov wrote:
This is to call for a vote for the release of Apache 
Brooklyn

0.10.0.
This release comprises of a source code distribution, and a
corresponding
binary distribution, and Maven artifacts.
The source and binary distributions, including signatures,

digests,
etc. can
be found at:

https://dist.apache.org/repos/dist/dev/brooklyn/apache-
brooklyn-0.10.0-rc3
The artifact SHA-256 checksums are as follows:
9ae6ec9f6e2356264fd2032d224965d191e5eab5a5346f8a9de5b0527165 

0157

*apache-brooklyn-0.10.0-rc3-1.noarch.rpm
b43c1fc20409594d17151d68b550bb17d8ea5a5592e0b7fbdcf489a81ca2 

118e

*apache-brooklyn-0.10.0-rc3-bin.tar.gz
cb7df99f024e918c5517097d904a68d9976c5bf913c9dd64edebb6b5b6e5 

a89e

*apache-brooklyn-0.10.0-rc3-bin.zip
d65d2525507d40fa0b554b13a57190a724213610a7615082d53137b7bab4 

b5d5

*apache-brooklyn-0.10.0-rc3-client-cli-linux.tar.gz
4d13118fca8e02aaf5b4ab3b3f305d139450bcf64d290781609643c4622b 

8cb5

*apache-brooklyn-0.10.0-rc3-client-cli-linux.zip
375eb6a4944a7758dc790a75bd4a0abc782a4ba66af754b4162bbb66a339 

02d8

*apache-brooklyn-0.10.0-rc3-client-cli-macosx.tar.gz
9d95e0679e603a9184ada88b63b0c4e8f8503eb51677e04ae59f4aa05f45 

4860

*apache-brooklyn-0.10.0-rc3-client-cli-macosx.zip
502e0999d3612f0eb4027d24312da67a67183fc127074ec815b3c72f5de4 

1d87

*apache-brooklyn-0.10.0-rc3-client-cli-windows.tar.gz
1e1ec0210e53faaf9ef1d8907275535a197b341df80e0832067967f4075d 

b191

*apache-brooklyn-0.10.0-rc3-client-cli-windows.zip
a32c65628bf897c66ed8bd820a2ce5268bba52815b46f805795e9b51ac83 

6912

*apache-brooklyn-0.10.0-rc3-karaf.tar.gz
4b840de7ab986ba64ffba4b98748ca87818718ac3e386bb2d978533e4fa3 

5f62

*apache-brooklyn-0.10.0-rc3-karaf.zip
7906b3dba2278c9648f2b340a532d3030536a6a8f1c920311bdcd585a421 

11e8

*apache-brooklyn-0.10.0-rc3-src.tar.gz
4579e5e27c2751b5e8b57bab126ae683aa34aa42d6e13eb6c922951df694 

7b4c

*apache-brooklyn-0.10.0-rc3-src.zip
b361ada2d4cc1334e72b44986b96e6302aaa24464252c3782f7b679bfa5d 

a858

*apache-brooklyn-0.10.0-rc3-vagrant.tar.gz
393df963f6b952ec8c05c1aa0ab0d459037e0972798f17ba7d0221f7e357 

0440

*apache-brooklyn-0.10.0-rc3-vagrant.zip
The Nexus staging repository for the Maven artifacts is 
located
at:
https://repository.apache.org/content/repositories/
orgapachebrooklyn-1034
All release artifacts are signed with the following key:
https://people.apache.org/keys/committer/svet.asc

KEYS file available here:

https://dist.apache.org/repos/dist/release/brooklyn/KEYS


The artifacts were built from git commit IDs:

brooklyn: 1d7ab19ffba2c87e9ad1b037fb217dc7d39506c9
brooklyn-client: 0e870fa15714c5a050bb67d6fa0429ff90a8fa4c
brooklyn-dist: dda655c45abab34601a5e62250431ea93fdf1755
brooklyn-docs: d75752725ef6683a0f52e3059ed475dc69872f9e
brooklyn-library: 9fbc78cb4a9d1dccdbdbb70a1ee48f2afddfe067
brooklyn-server: 41561635e3bbb0524dbeaae516a26c793174fd93
brooklyn-ui: a6e2e8bccfdd98b4f7155b5be86f5b85149e0f33
All of the above have been tagged as
"apache-brooklyn-0.10.0-rc3"
Please vote on releasing this package as Apache Brooklyn
0.10.0.
The vote will be open for at least 72 hours.
[ ] +1 Release this package as Apache Brooklyn 0.10.0
[ ] +0 no opinion
[ ] -1 Do not release this package because ...


Thanks!
Svet.


CHECKLIST for reference

[ ] Download links work.
[ ] Binaries work.
[ ] Checksums and PGP signatures are valid.
[ ] Expanded source archive matches contents of RC tag.
[ ] Expanded source archive builds and passes tests.
[ ] LICENSE is present and correct.
[ ] NOTICE is present and correct, including copyright 
date.
[ ] All files have license headers where appropriate.
[ ] All dependencies have compatible licenses.
[ ] No compiled archives bundled in source archive.
[ ] I follow this project’s commits list.