Re: August 2017 Release

[Mars Hall]

Yes that is the PR. When I checked out develop yesterday, I though it was 
already merged. Apologies for my confusion.

I'd like to see if I can get that merged for the release. Looking into it now.

*Mars

( <> .. <> )

> On Aug 4, 2017, at 12:46, Donald Szeto  wrote:
> 
> Hey Mars,
> 
> Is this the PR in question?
> https://github.com/apache/incubator-predictionio/pull/372
> 
> Regards,
> Donald
> 
> On Thu, Aug 3, 2017 at 11:49 AM, Mars Hall  wrote:
> 
>> Hit an Authenticated Elasticsearch 5.x problem on the current develop
>> branch.
>> 
>> I just tested the HEAD of develop by performing:
>> 
>>  ./make-distribution.sh \
>>-Dscala.version=2.11.8 \
>>-Dspark.version=2.1.0 \
>>-Dhadoop.version=2.7.3 \
>>-Delasticsearch.version=5.1.1
>> 
>> Then, tried build/train/deploy of our Universal Recommender template.
>> 
>> Locally, it makes it through train to the point when it saves to
>> Elasticsearch, failing with:
>> 
>>> Exception in thread "main" java.lang.NoSuchMethodError:
>> org.elasticsearch.client.RestClient.performRequest(
>> Ljava/lang/String;Ljava/lang/String;Ljava/util/Map;[Lorg/
>> apache/http/Header;)Lorg/elasticsearch/client/Response;
>>>  at org.template.EsClient$.createIndex(EsClient.scala:132)
>>>  at org.template.EsClient$.hotSwap(EsClient.scala:218)
>>>  at org.template.URModel.save(URModel.scala:86)
>> 
>> I tried deploying it to Heroku as well, and it fails much earlier when
>> simply connecting to Elasticsearch:
>> 
>>> remote: Exception in thread "main" 
>>> org.elasticsearch.client.ResponseException:
>> HEAD https://xx.us-east-1.bonsaisearch.net:443/pio_meta: HTTP/1.1 401
>> Unauthorized
>>> remote:   at org.elasticsearch.client.RestClient$1.completed(
>> RestClient.java:311)
>>> remote:   at org.elasticsearch.client.RestClient$1.completed(
>> RestClient.java:300)
>>> remote:   at shadeio.data.http.concurrent.BasicFuture.completed(
>> BasicFuture.java:119)
>>> remote:   at shadeio.data.http.impl.nio.client.
>> DefaultClientExchangeHandlerImpl.responseCompleted(
>> DefaultClientExchangeHandlerImpl.java:177)
>>> remote:   at shadeio.data.http.nio.protocol.
>> HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:
>> 436)
>>> remote:   at shadeio.data.http.nio.protocol.
>> HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:
>> 309)
>>> remote:   at shadeio.data.http.impl.nio.
>> DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.
>> java:255)
>> 
>> 
>> These issues were previously found to be caused by this shade rule:
>>  https://github.com/apache/incubator-predictionio/blob/
>> develop/storage/elasticsearch/build.sbt#L42
>> 
>> It looks like the shaded package does not actually use the new
>> authentication code.
>> 
>> Chan Lee mentioned to me that he was only able to make the TravisCI build
>> pass by adding this shade rule, but it is clearly breaking the authenicated
>> Elasticsearch functionality.
>> 
>> Any ideas how to solve this?
>> 
>> *Mars
>> 
>> ( <> .. <> )
>> 
>>> On Aug 3, 2017, at 11:02, Donald Szeto  wrote:
>>> 
>>> On Thu, Aug 3, 2017 at 10:07 AM, Mars Hall  wrote:
>>> 
 I just opened a PR to add docs for batch predict.
 
 Moving forward with the 0.12.0 release sounds great. Today, I will pull
 develop and see how it's working with the Heroku buildpack.
 
>>> 
>>> Awesome. Thanks!
>>> 
>>> 
> On Aug 3, 2017, at 00:37, takako shimamoto 
>> wrote:
> 
> I think it's almost ready, and now we just have to update the current
> documentation.
> The deadline of several unresolved issues for Target Version/s:
> 0.12.0-incubating is extended, right?
 
>>> 
>>> Yes. Let's extend those that have not started working if there's no
>>> objection.
>> 
>> 

Re: August 2017 Release

[Donald Szeto]

Hey Mars,

Is this the PR in question?
https://github.com/apache/incubator-predictionio/pull/372

Regards,
Donald

On Thu, Aug 3, 2017 at 11:49 AM, Mars Hall  wrote:

> Hit an Authenticated Elasticsearch 5.x problem on the current develop
> branch.
>
> I just tested the HEAD of develop by performing:
>
>   ./make-distribution.sh \
> -Dscala.version=2.11.8 \
> -Dspark.version=2.1.0 \
> -Dhadoop.version=2.7.3 \
> -Delasticsearch.version=5.1.1
>
> Then, tried build/train/deploy of our Universal Recommender template.
>
> Locally, it makes it through train to the point when it saves to
> Elasticsearch, failing with:
>
> > Exception in thread "main" java.lang.NoSuchMethodError:
> org.elasticsearch.client.RestClient.performRequest(
> Ljava/lang/String;Ljava/lang/String;Ljava/util/Map;[Lorg/
> apache/http/Header;)Lorg/elasticsearch/client/Response;
> >   at org.template.EsClient$.createIndex(EsClient.scala:132)
> >   at org.template.EsClient$.hotSwap(EsClient.scala:218)
> >   at org.template.URModel.save(URModel.scala:86)
>
> I tried deploying it to Heroku as well, and it fails much earlier when
> simply connecting to Elasticsearch:
>
> > remote: Exception in thread "main" 
> > org.elasticsearch.client.ResponseException:
> HEAD https://xx.us-east-1.bonsaisearch.net:443/pio_meta: HTTP/1.1 401
> Unauthorized
> > remote:   at org.elasticsearch.client.RestClient$1.completed(
> RestClient.java:311)
> > remote:   at org.elasticsearch.client.RestClient$1.completed(
> RestClient.java:300)
> > remote:   at shadeio.data.http.concurrent.BasicFuture.completed(
> BasicFuture.java:119)
> > remote:   at shadeio.data.http.impl.nio.client.
> DefaultClientExchangeHandlerImpl.responseCompleted(
> DefaultClientExchangeHandlerImpl.java:177)
> > remote:   at shadeio.data.http.nio.protocol.
> HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:
> 436)
> > remote:   at shadeio.data.http.nio.protocol.
> HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:
> 309)
> > remote:   at shadeio.data.http.impl.nio.
> DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.
> java:255)
>
>
> These issues were previously found to be caused by this shade rule:
>   https://github.com/apache/incubator-predictionio/blob/
> develop/storage/elasticsearch/build.sbt#L42
>
> It looks like the shaded package does not actually use the new
> authentication code.
>
> Chan Lee mentioned to me that he was only able to make the TravisCI build
> pass by adding this shade rule, but it is clearly breaking the authenicated
> Elasticsearch functionality.
>
> Any ideas how to solve this?
>
> *Mars
>
> ( <> .. <> )
>
> > On Aug 3, 2017, at 11:02, Donald Szeto  wrote:
> >
> > On Thu, Aug 3, 2017 at 10:07 AM, Mars Hall  wrote:
> >
> >> I just opened a PR to add docs for batch predict.
> >>
> >> Moving forward with the 0.12.0 release sounds great. Today, I will pull
> >> develop and see how it's working with the Heroku buildpack.
> >>
> >
> > Awesome. Thanks!
> >
> >
> >>> On Aug 3, 2017, at 00:37, takako shimamoto 
> wrote:
> >>>
> >>> I think it's almost ready, and now we just have to update the current
> >>> documentation.
> >>> The deadline of several unresolved issues for Target Version/s:
> >>> 0.12.0-incubating is extended, right?
> >>
> >
> > Yes. Let's extend those that have not started working if there's no
> > objection.
>
>

Re: August 2017 Release

[Mars Hall]

Hit an Authenticated Elasticsearch 5.x problem on the current develop branch.

I just tested the HEAD of develop by performing:

  ./make-distribution.sh \
-Dscala.version=2.11.8 \
-Dspark.version=2.1.0 \
-Dhadoop.version=2.7.3 \
-Delasticsearch.version=5.1.1

Then, tried build/train/deploy of our Universal Recommender template.

Locally, it makes it through train to the point when it saves to Elasticsearch, 
failing with:

> Exception in thread "main" java.lang.NoSuchMethodError: 
> org.elasticsearch.client.RestClient.performRequest(Ljava/lang/String;Ljava/lang/String;Ljava/util/Map;[Lorg/apache/http/Header;)Lorg/elasticsearch/client/Response;
>   at org.template.EsClient$.createIndex(EsClient.scala:132)
>   at org.template.EsClient$.hotSwap(EsClient.scala:218)
>   at org.template.URModel.save(URModel.scala:86)

I tried deploying it to Heroku as well, and it fails much earlier when simply 
connecting to Elasticsearch:

> remote: Exception in thread "main" 
> org.elasticsearch.client.ResponseException: HEAD 
> https://xx.us-east-1.bonsaisearch.net:443/pio_meta: HTTP/1.1 401 
> Unauthorized
> remote:   at 
> org.elasticsearch.client.RestClient$1.completed(RestClient.java:311)
> remote:   at 
> org.elasticsearch.client.RestClient$1.completed(RestClient.java:300)
> remote:   at 
> shadeio.data.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
> remote:   at 
> shadeio.data.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
> remote:   at 
> shadeio.data.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
> remote:   at 
> shadeio.data.http.nio.protocol.HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:309)
> remote:   at 
> shadeio.data.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:255)


These issues were previously found to be caused by this shade rule:
  
https://github.com/apache/incubator-predictionio/blob/develop/storage/elasticsearch/build.sbt#L42

It looks like the shaded package does not actually use the new authentication 
code.

Chan Lee mentioned to me that he was only able to make the TravisCI build pass 
by adding this shade rule, but it is clearly breaking the authenicated 
Elasticsearch functionality.

Any ideas how to solve this?

*Mars

( <> .. <> )

> On Aug 3, 2017, at 11:02, Donald Szeto  wrote:
> 
> On Thu, Aug 3, 2017 at 10:07 AM, Mars Hall  wrote:
> 
>> I just opened a PR to add docs for batch predict.
>> 
>> Moving forward with the 0.12.0 release sounds great. Today, I will pull
>> develop and see how it's working with the Heroku buildpack.
>> 
> 
> Awesome. Thanks!
> 
> 
>>> On Aug 3, 2017, at 00:37, takako shimamoto  wrote:
>>> 
>>> I think it's almost ready, and now we just have to update the current
>>> documentation.
>>> The deadline of several unresolved issues for Target Version/s:
>>> 0.12.0-incubating is extended, right?
>> 
> 
> Yes. Let's extend those that have not started working if there's no
> objection.

Re: August 2017 Release

[Donald Szeto]

On Thu, Aug 3, 2017 at 10:07 AM, Mars Hall  wrote:

> I just opened a PR to add docs for batch predict.
>
> Moving forward with the 0.12.0 release sounds great. Today, I will pull
> develop and see how it's working with the Heroku buildpack.
>

Awesome. Thanks!


> > On Aug 3, 2017, at 00:37, takako shimamoto  wrote:
> >
> > I think it's almost ready, and now we just have to update the current
> > documentation.
> > The deadline of several unresolved issues for Target Version/s:
> > 0.12.0-incubating is extended, right?
>

Yes. Let's extend those that have not started working if there's no
objection.

Re: August 2017 Release

[takako shimamoto]

I think it's almost ready, and now we just have to update the current
documentation.
The deadline of several unresolved issues for Target Version/s:
0.12.0-incubating is extended, right?


2017-08-01 5:59 GMT+09:00 Donald Szeto :
> Hey all,
>
> To adhere to the proposed release cadence by Sara, let's shoot to cut a
> release in early August that addresses all the top level graduation issues.
>
> How does the developer community feel about this? Any major issues that we
> should also resolve with this release? It looks like many TLP cleanup, new
> features and bug fixes have already gone into the current development
> branch.
>
> Regards,
> Donald

August 2017 Release

[Donald Szeto]

Hey all,

To adhere to the proposed release cadence by Sara, let's shoot to cut a
release in early August that addresses all the top level graduation issues.

How does the developer community feel about this? Any major issues that we
should also resolve with this release? It looks like many TLP cleanup, new
features and bug fixes have already gone into the current development
branch.

Regards,
Donald