Re: [ANNOUNCE] Apache PredictionIO 0.12.0-incubating Release

[Chan Lee]

My apologies. The doc site has been updated now.

[ANNOUNCE] Apache PredictionIO 0.12.0-incubating Release

[Chan Lee]

The Apache PredictionIO team would like to announce the release of Apache
PredictionIO 0.12.0-incubating.

Release notes are here:
https://github.com/apache/incubator-predictionio/blob/release/0.12.0/RELEASE.md

Apache PredictionIO (incubating) is an open source Machine Learning Server
built on top of state-of-the-art open source stack, that enables developers
to manage and deploy production-ready predictive services for various kinds
of machine learning tasks.

More details regarding Apache PredictionIO (incubating) can be found here:
http://predictionio.incubator.apache.org/

The release artifacts can be downloaded here:
https://dist.apache.org/repos/dist/release/incubator/predictionio/0.12.0-incubating/

All JIRAs completed for this release are tagged with 'FixVersion =
0.12.0-incubating'; the JIRA release notes can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12340591=12320420

Thanks!
The Apache PredictionIO Team

DISCLAIMER
Apache PredictionIO (incubating) is an effort undergoing incubation at the
Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further
review indicates that the infrastructure, communications, and decision
making process have stabilized in a manner consistent with other successful
ASF projects. While incubation status is not necessarily a reflection of
the completeness or stability of the code, it does indicate that the
project has yet to be fully endorsed by the ASF.

Re: [VOTE] Resolution to create a TLP from graduating Incubator podling

[Chan Lee]

+1 binding


On Mon, Sep 25, 2017 at 9:16 PM, Mars Hall <mars.h...@salesforce.com> wrote:

> +1 binding
>
> On Tue, Sep 26, 2017 at 12:57 Andrew Purtell <andrew.purt...@gmail.com>
> wrote:
>
> > +1 (binding)
> >
> > > On Sep 25, 2017, at 8:50 PM, Donald Szeto <don...@apache.org> wrote:
> > >
> > > Hi all,
> > >
> > > Based on previous discussions (
> > >
> > https://lists.apache.org/thread.html/2b4ef7c394584988cf0c99920824af
> aa60ee4c648d5c0069b1bf55c0@%3Cdev.predictionio.apache.org%3E
> > > and
> > >
> > https://lists.apache.org/thread.html/1b06e510773ee1d315728e0ce25f22
> 0c9cf7d9e8ad601ec9dba4fe1d@%3Cdev.predictionio.apache.org%3E
> > ),
> > > I would like to start a formal vote on graduating PredictionIO from an
> > > Incubator podling to a top level project with the following resolution.
> > > This thread will be forwarded to the Incubator general mailing list.
> > >
> > > Once again, Salesforce has already signed and executed an assignment
> > > agreement to assign the PredictionIO mark to ASF.
> > >
> > > The graduation process we are following is described here:
> > > http://incubator.apache.org/guides/graduation.html
> > >
> > > Once this vote passes, a discussion will be started on Incubator
> general,
> > > followed by a vote when a consensus there would be arrived. The vote
> will
> > > run for at least 72 hours before closing at 9PM PST on 9/28/2017.
> > >
> > > Thank you all! Let's graduate.
> > >
> > > +1 (binding) from me.
> > >
> > > Regards,
> > > Donald
> > >
> > > -
> > >
> > >X. Establish the Apache PredictionIO Project
> > >
> > >   WHEREAS, the Board of Directors deems it to be in the best
> > >   interests of the Foundation and consistent with the
> > >   Foundation's purpose to establish a Project Management
> > >   Committee charged with the creation and maintenance of
> > >   open-source software, for distribution at no charge to
> > >   the public, related to a machine learning server built on top of
> > >   state-of-the-art open source stack, that enables developers to
> > manage
> > >   and deploy production-ready predictive services for various kinds
> > of
> > >   machine learning tasks.
> > >
> > >   NOW, THEREFORE, BE IT RESOLVED, that a Project Management
> > >   Committee (PMC), to be known as the "Apache PredictionIO
> Project",
> > >   be and hereby is established pursuant to Bylaws of the
> > >   Foundation; and be it further
> > >
> > >   RESOLVED, that the Apache PredictionIO Project be and hereby is
> > >   responsible for the creation and maintenance of software
> > >   related to a machine learning server built on top of
> > >   state-of-the-art open source stack, that enables developers to
> > manage
> > >   and deploy production-ready predictive services for various kinds
> > of
> > >   machine learning tasks;
> > >   and be it further
> > >
> > >   RESOLVED, that the office of "Vice President, Apache
> PredictionIO"
> > be
> > >   and hereby is created, the person holding such office to
> > >   serve at the direction of the Board of Directors as the chair
> > >   of the Apache PredictionIO Project, and to have primary
> > > responsibility
> > >   for management of the projects within the scope of
> > >   responsibility of the Apache PredictionIO Project; and be it
> > further
> > >
> > >   RESOLVED, that the persons listed immediately below be and
> > >   hereby are appointed to serve as the initial members of the
> > >   Apache PredictionIO Project:
> > >
> > > * Alex Merritt <emergentor...@apache.org>
> > > * Andrew Kyle Purtell <apurt...@apache.org>
> > > * Chan Lee <chan...@apache.org>
> > > * Donald Szeto <don...@apache.org>
> > > * Felipe Oliveira <fel...@apache.org>
> > > * James Taylor <jtay...@apache.org>
> > > * Justin Yip <yipjus...@apache.org>
> > > * Kenneth Chan <kenn...@apache.org>
> > > * Lars Hofhansl <la...@apache.org>
> > > * Lee Moon Soo <m...@apache.

[RESULT][VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC3)

[Chan Lee]

The vote passes, with 6 +1 votes (6 binding) and no -1 votes.

+1 Chan Lee (binding)
+1 Donald Szeto (binding)
+1 Mars Hall (binding)
+1 Naoki Takezoe (binding)
+1 Shinsuke Sugaya (binding)
+1 Takako Shimamoto (binding)

Thanks all for voting.

Apache PredictionIO (incubating) 0.12.0 RC3 has been accepted as the
release. The voting will now proceed to the Incubator PMC.

Chan

Re: [VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC3)

[Chan Lee]

+1 binding


On Mon, Sep 18, 2017 at 11:20 AM, Mars Hall <mars.h...@salesforce.com>
wrote:

> +1 binding
>
> I checked:
> - build, train, deploy, & batchpredict
> - complete Elasticsearch 5.x functionality
> - Binaries work directly for Heroku deployment
>
> Such an exciting release! Thank you Chan,
>
> *Mars
>
>
> On Sun, Sep 17, 2017 at 11:31 AM, Chan Lee <chanlee...@gmail.com> wrote:
> >
> > > This is the vote for 0.12.0 of Apache PredictionIO (incubating).
> > >
> > > The vote will run for at least 72 hours and will close on Sep 20th,
> 2017.
> > >
> > > The release candidate artifacts can be downloaded here:
> > > https://dist.apache.org/repos/dist/dev/incubator/predictionio/0.12.0-
> > > incubating-rc3
> > >
> > > Test results of RC3 can be found here:
> > > https://travis-ci.org/apache/incubator-predictionio/builds/276558626
> > >
> > > Maven artifacts are built from the release candidate artifacts above,
> and
> > > are provided as convenience for testing with engine templates. The
> Maven
> > > artifacts are provided at the Maven staging repo here:
> > > https://repository.apache.org/content/repositories/
> > > orgapachepredictionio-1021/
> > >
> > > All JIRAs completed for this release are tagged with 'FixVersion =
> > > 0.12.0-incubating'. You can view them here: https://issues.apache.or
> > > g/jira/secure/ReleaseNote.jspa?version=12340591=12320420
> > >
> > > The artifacts have been signed with Key: ytX8GpWv
> > >
> > > Please vote accordingly:
> > >
> > > [ ] +1, accept RC as the official 0.12.0 release
> > > [ ] -1, do not accept RC as the official 0.12.0 release because...
> > >
> >
>

[VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC3)

[Chan Lee]

This is the vote for 0.12.0 of Apache PredictionIO (incubating).

The vote will run for at least 72 hours and will close on Sep 20th, 2017.

The release candidate artifacts can be downloaded here:
https://dist.apache.org/repos/dist/dev/incubator/predictionio/0.12.0-incubating-rc3

Test results of RC3 can be found here:
https://travis-ci.org/apache/incubator-predictionio/builds/276558626

Maven artifacts are built from the release candidate artifacts above, and
are provided as convenience for testing with engine templates. The Maven
artifacts are provided at the Maven staging repo here:
https://repository.apache.org/content/repositories/orgapachepredictionio-1021/

All JIRAs completed for this release are tagged with 'FixVersion =
0.12.0-incubating'. You can view them here: https://issues.apache.or
g/jira/secure/ReleaseNote.jspa?version=12340591=12320420

The artifacts have been signed with Key: ytX8GpWv

Please vote accordingly:

[ ] +1, accept RC as the official 0.12.0 release
[ ] -1, do not accept RC as the official 0.12.0 release because...

Re: [VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC2)

[Chan Lee]

Actually, I found out that there are complications with having to document
difference in two versions of binary release.

Since getting the release is a bigger priority, I'll go with the default
(2.11 dependencies) and push back providing two versions to next release,
if deemed necessary then.

On Sun, Sep 17, 2017 at 10:17 AM, Chan Lee <chanlee...@gmail.com> wrote:

> That sounds great. I'll prepare a new RC with updates and binary
> distribution.
>
> I think it would be reasonable to provide two binary distributions: (Scala
> 2.10.6 / Spark 1.6.3 / ES 1.7.3), (Scala 2.11.8 / Spark 2.1.1 / ES 5.5.2).
> Does anyone have objections?
>
>
> On Sat, Sep 16, 2017 at 2:24 PM, Mars Hall <mars.h...@salesforce.com>
> wrote:
>
>> > On Sat, Sep 16, 2017 at 5:13 AM Naoki Takezoe <take...@gmail.com>
>> wrote:
>> >
>> > > Can we publish the binary distribution which is built by
>> > > make-distribution.sh together?
>>
>>
>> Fantastic  This will improve the experience for so many people.
>> Thank
>> you Naoki!
>>
>> > --
>> *Mars Hall
>> 415-818-7039
>> Customer Facing Architect
>> Salesforce Platform / Heroku
>> San Francisco, California
>>
>
>

Re: [VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC2)

[Chan Lee]

That sounds great. I'll prepare a new RC with updates and binary
distribution.

I think it would be reasonable to provide two binary distributions: (Scala
2.10.6 / Spark 1.6.3 / ES 1.7.3), (Scala 2.11.8 / Spark 2.1.1 / ES 5.5.2).
Does anyone have objections?


On Sat, Sep 16, 2017 at 2:24 PM, Mars Hall  wrote:

> > On Sat, Sep 16, 2017 at 5:13 AM Naoki Takezoe  wrote:
> >
> > > Can we publish the binary distribution which is built by
> > > make-distribution.sh together?
>
>
> Fantastic  This will improve the experience for so many people. Thank
> you Naoki!
>
> > --
> *Mars Hall
> 415-818-7039
> Customer Facing Architect
> Salesforce Platform / Heroku
> San Francisco, California
>

Re: [VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC2)

[Chan Lee]

Mars, this seems like a necessary change, so I'll create RC3 with the PR
and additional updates here: http://predictionio.incubator.apache.org/
resources/upgrade/

If there are changes anyone would like to add, please let me know by today.
I'll patch up another release tonight and send out a new email.

Thanks,
Chan


On Fri, Sep 15, 2017 at 11:54 AM, Donald Szeto <don...@apache.org> wrote:

> Votes are tied to tag/commit by ASF convention, so a new RC and vote will
> be required.
>
> On Fri, Sep 15, 2017 at 10:57 AM Mars Hall <mars.h...@salesforce.com>
> wrote:
>
> > I just opened a release notes PR against apache:release/0.12.0, because
> > that seems to be the right place.
> >
> > Chan, will that work okay with the release process?
> >
> > On Fri, Sep 15, 2017 at 10:20 AM, Mars Hall <mars.h...@salesforce.com>
> > wrote:
> >
> > > Also, I'd love to directly link the PIO-* issue numbers to JIRA.
> > >
> > > On Fri, Sep 15, 2017 at 10:19 AM, Mars Hall <mars.h...@salesforce.com>
> > > wrote:
> > >
> > >> RC2 is working perfectly.
> > >>
> > >> I see a few issues with the releases notes:
> > >>
> > >>
> > >>- PIO-95 should be "Raised request timeout for REST API to
> > 35-seconds"
> > >>- PIO-102, PIO-106, PIO-117, PIO-118, PIO-120 actually includes a
> > >>breaking change to Elasticsearch 5.x StorageClient interface. I
> > think these
> > >>should be enumerated more explicitly with one of them called out
> in a
> > >>"Breaking changes" section.
> > >>
> > >> May I revise RELEASE.md on develop to fix these issues? Does that
> > require
> > >> restarting vote for an RC3?
> > >>
> > >>
> > >> On Thu, Sep 14, 2017 at 11:49 PM, Donald Szeto <don...@apache.org>
> > wrote:
> > >>
> > >>> I believe those are fixed by PIO-60, PIO-62 and PIO-63 in the release
> > >>> notes.
> > >>>
> > >>> +1 binding from me
> > >>>
> > >>> On Thu, Sep 14, 2017 at 2:13 PM Pat Ferrel <p...@occamsmachete.com>
> > >>> wrote:
> > >>>
> > >>> > The last release was hung up by the IPMC regarding content
> licensing
> > >>> > issues and libraries used by the doc site, which we promised to
> > >>> address in
> > >>> > this release. Have these been resolved, don’t recall the specifics?
> > It
> > >>> > would be great to fly through the IPMC vote without issue.
> > >>> >
> > >>> >
> > >>> > On Sep 14, 2017, at 2:06 PM, Chan Lee <chanlee...@gmail.com>
> wrote:
> > >>> >
> > >>> > This is the vote for 0.12.0 of Apache PredictionIO (incubating).
> > >>> >
> > >>> > The vote will run for at least 72 hours and will close on Sep 17th,
> > >>> 2017.
> > >>> >
> > >>> > The release candidate artifacts can be downloaded here:
> > >>> > https://dist.apache.org/repos/dist/dev/incubator/predi
> > >>> > ctionio/0.12.0-incubating-rc2
> > >>> >
> > >>> > Test results of RC1 can be found here: https://travis-ci.org/ap
> > >>> > ache/incubator-predictionio/builds/275634960
> > >>> >
> > >>> > Maven artifacts are built from the release candidate artifacts
> above,
> > >>> and
> > >>> > are provided as convenience for testing with engine templates. The
> > >>> Maven
> > >>> > artifacts are provided at the Maven staging repo here:
> > >>> >
> > >>> > https://repository.apache.org/content/repositories/orgapache
> > >>> predictionio-1020
> > >>> >
> > >>> > All JIRAs completed for this release are tagged with 'FixVersion =
> > >>> > 0.12.0-incubating'. You can view them here:
> https://issues.apache.or
> > >>> > g/jira/secure/ReleaseNote.jspa?version=12340591=12320420
> > >>> >
> > >>> > The artifacts have been signed with Key: ytX8GpWv
> > >>> >
> > >>> > Please vote accordingly:
> > >>> >
> > >>> > [ ] +1, accept RC as the official 0.12.0 release
> > >>> > [ ] -1, do not accept RC as the official 0.12.0 release because...
> > >>> >
> > >>> >
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> *Mars Hall
> > >> 415-818-7039 <(415)%20818-7039>
> > >> Customer Facing Architect
> > >> Salesforce Platform / Heroku
> > >> San Francisco, California
> > >>
> > >
> > >
> > >
> > > --
> > > *Mars Hall
> > > 415-818-7039 <(415)%20818-7039>
> > > Customer Facing Architect
> > > Salesforce Platform / Heroku
> > > San Francisco, California
> > >
> >
> >
> >
> > --
> > *Mars Hall
> > 415-818-7039
> > Customer Facing Architect
> > Salesforce Platform / Heroku
> > San Francisco, California
> >
>

[CANCEL][VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC1)

[Chan Lee]

Cancelling the vote because we have found missing changes in RELEASE.md.

[VOTE] Apache PredictionIO (incubating) 0.12.0 Release (RC1)

[Chan Lee]

This is the vote for 0.12.0 of Apache PredictionIO (incubating).

The vote will run for at least 72 hours and will close on Sep 16th, 2017.

The release candidate artifacts can be downloaded here:
https://dist.apache.org/repos/dist/dev/incubator/predictionio/0.12.0-
incubating-rc1

Test results of RC1 can be found here: https://travis-ci.org/
apache/incubator-predictionio/builds/275220461

Maven artifacts are built from the release candidate artifacts above, and
are provided as convenience for testing with engine templates. The Maven
artifacts are provided at the Maven staging repo here: https://repository.
apache.org/content/repositories/orgapachepredictionio-1018

All JIRAs completed for this release are tagged with 'FixVersion =
0.12.0-incubating'. You can view them here: https://issues.apache.
org/jira/secure/ReleaseNote.jspa?version=12340591=12320420

The artifacts have been signed with Key: ytX8GpWv

Please vote accordingly:

[ ] +1, accept RC as the official 0.12.0 release
[ ] -1, do not accept RC as the official 0.12.0 release because...

JIRAs to include in 0.12 Release

[Chan Lee]

Hi all,

These are the current JIRA issues to include in 0.12 release. The ones
marked "(pending)" are waiting for PR review and merge. Please let me know
if there are any missing issues that you'd like to include in the 0.12
release. Thanks!


Fixes
[PIO-56] Unit tests in module core should not require real setup
[PIO-59] "pio app new" sometimes takes long time (pending)
[PIO-60] Fix minor issues with author information
[PIO-62] Investigate RAT issues and other authorship information
[PIO-63] Fix incubator branding issues
[PIO-64] JBDCPEvents unit test fail intermittently
[PIO-72] In `pio-shell` jdbc.StorageClient cannot be loaded
[PIO-91] Hadoop-HDFS artifact missing when creating binary version of
PredictionIO
[PIO-94] Query parsing may throw intractable errors
[PIO-97] Fixes examples of the official templates
[PIO-99] `pio build` exits successfully even when compilation fails
[PIO-100] Remove SLF4J warning messages on pio command
[PIO-102] ESEngineInstances `getAll` results out of order (Elasticsearch
5.x)
[PIO-104] Make proper implementation of plugins
[PIO-107] Removal of examples under examples/experimental
[PIO-117] Cannot delete event data on ESLEvents
[PIO-118] ClassCastException from NullWritable to Text in ESEventsUtil
[PIO-120] Process hangs if Elasticsearch is not available during train
(pending)

New features / Improvements / Refactoring
[PIO-61] Add S3 Modeldata Repository
[PIO-65] Cache downloaded jars in Travis build
[PIO-66] Document JIRA processes and add to public documentation
[PIO-69] Create binary version of PredictionIO as part of release process
[PIO-83] Move to Spark 2.x and Scala 2.11 as the default env with
deprecating (not deprecated) support for previous tools chain
[PIO-90] Improve /batch/events.json endpoint performance
[PIO-95] Configurable request timeout for REST API
[PIO-105] Batch predictions
[PIO-106] Elasticsearch 5.x StorageClient should reuse RestClient
[PIO-110] Refactor common code shared by CreateServer and BatchPredict
[PIO-111] Document pio batchpredict
[PIO-114] Elasticsearch 5.x StorageClient basic HTTP authentication
[PIO-115] Cache name-to-ID lookups for Storage app & channel
[PIO-116] PySpark Support (pending)
[PIO-119] Bump up Elasticsearch to 5.5.2 (pending)

0.12 Release and Rotation on Release Management

[Chan Lee]

Hi all,

It seems we are ready to push for 0.12 release. Since we have two
additional volunteers (Shinsuke and I) for release management, I think it
would be nice to rotate between releases depending on availability.

I can start the process for 0.12 release if no one objects.

Best,
Chan

Re: Graduation to TLP

[Chan Lee]

Agree with everyone else. +1 for graduation.

I was away for a while due to family issues, but I'd be happy to volunteer
for release management.


On Wed, Aug 30, 2017 at 4:30 PM, Pat Ferrel  wrote:

> Along with the link Donald gave, check this out:
> http://community.apache.org/apache-way/apache-project-maturity-model.html
>  >
>
> We will need at least 2 release managers before graduation, any volunteers?
>
>
> On Aug 30, 2017, at 3:51 PM, Mars Hall  wrote:
>
> Thank you Donald for leading the charge here,
>
> From my perspective PredictionIO is already Apache in process & title.
> Graduation seems quite natural to reach top-level recognition.
>
> I'm interested in helping with PMC duties. Would be great to understand
> what the VP vs Member responsibilities look like.
>
> Let's graduate. +1
>
> *Mars
>
>
> On Wed, Aug 30, 2017 at 15:21 Pat Ferrel  wrote:
>
> > I have had several people tell me they want to wait until PIO is not
> > incubating before using it. This even after explaining that “incubating”
> > has more to do with getting into the Apache Way of doing things and has
> no
> > direct link to quality or community. I can only conclude from this that
> > “incubating” is holding back adoption.
> >
> > And yet we have absorbed the Apache Way and will have at least 3 releases
> > (including 12) a incubating. We have brought in a fair number of new
> > committers and seem to have a healthy community of users.
> >
> > +1 for a push to graduate.
> >
> >
> > On Aug 28, 2017, at 10:20 PM, Donald Szeto  wrote:
> >
> > Hi all,
> >
> > Since the ASF Board meeting in May (
> >
> > http://apache.org/foundation/records/minutes/2017/board_
> minutes_2017_05_17.txt
> > ),
> > PredictionIO has been considered nearing graduation and I think we are
> > almost there. I am kickstarting this thread so that we can discuss on
> these
> > 3 things:
> >
> > 1. Does the development community feel ready to graduate?
> > 2. If we are to graduate, who should we include in the list of the
> initial
> > PMC?
> > 3. If we are to graduate, who should be the VP of the initial PMC?
> >
> > These points are relevant for graduation. Please take a look at the
> > official graduation guide:
> > http://incubator.apache.org/guides/graduation.html.
> >
> > In addition, Sara and I have been working to transfer the PredictionIO
> > trademark to the ASF. We will keep you updated with our progress.
> >
> > I would also like to propose to cut a 0.12.0 release by merging JIRAs
> that
> > have a target version set to 0.12.0-incubating for graduation. 0.12.0
> will
> > contain cleanups for minor license and copyright issues that were pointed
> > out in previous releases by IPMC.
> >
> > Let me know what you think.
> >
> > Regards,
> > Donald
> >
> > --
> *Mars Hall
> 415-818-7039
> Customer Facing Architect
> Salesforce Platform / Heroku
> San Francisco, California
>
>

Re: JIRA and release process for PredictionIO

[Chan Lee]

+1

On Thu, May 25, 2017 at 9:20 PM, Sara Asher 
wrote:

> Hi everyone,
>
> It looks like everyone is content with the JIRA/release process doc.  If I
> don't hear from anyone else in the next day or so, I will assume it's good
> and will add the details to the public documentation for PredictionIO.
>
> Thanks,
> Sara
>
> On Thu, May 18, 2017 at 1:30 PM, Donald Szeto  wrote:
>
> > +1. Thanks Sara!
> >
> > On Thu, May 18, 2017 at 8:48 AM Sara Asher  wrote:
> >
> > > Hi everyone,
> > >
> > > Thank you for allowing me to join the PredictionIO team!
> > >
> > > To start with, I would like to document a process for using JIRA and to
> > > suggest a standard release cadence for PredictionIO.
> > >
> > > I've worked with Donald to come up with the following:
> > >
> > > https://docs.google.com/document/d/1nQpENncXZq72KeI3WMe_
> > > X8Xz8HKkYO2QC12GD3ZKP9g/edit?usp=sharing
> > >
> > > Please comment and give suggestions!  When you are all content, I will
> > add
> > > this to our public documentation.
> > >
> > > Thanks,
> > > Sara
> > >
> >
>

Re: [VOTE] Apache PredictionIO (incubating) 0.11.0 Release (RC2)

[Chan Lee]

[+1] binding accept

On Mon, Apr 10, 2017 at 1:22 PM, Donald Szeto  wrote:

> Thanks! Just a reminder that PMC members should indicate whether the vote
> is binding or not.
>
> On Mon, Apr 10, 2017 at 1:15 PM Mars Hall  wrote:
>
> > My non-binding vote for rc2!
> >
> > [X] +1, accept RC as the official 0.11.0 release
> > [ ] -1, do not accept RC as the official 0.11.0 release because...
> >
> > *Mars
> >
> > ( <> .. <> )
> >
> > > On Apr 9, 2017, at 21:29, Steven Yan 
> wrote:
> > >
> > > [X] +1, accept RC as the official 0.11.0 release
> > > [ ] -1, do not accept RC as the official 0.11.0 release because...
> > >
> > > Thanks,
> > > Steven
> > >
> > > On Sun, Apr 9, 2017 at 5:21 PM, Donald Szeto 
> wrote:
> > >
> > >> This is the vote for 0.11.0 of Apache PredictionIO (incubating).
> > >>
> > >> The vote will run for at least 72 hours and will close on Apr 12th,
> > 2017.
> > >>
> > >> RC2 fixes a usage bug where the pio command does not pass through
> > >> --driver-java-options to spark-submit.
> > >>
> > >> The release candidate artifacts can be downloaded here:
> > >> https://dist.apache.org/repos/dist/dev/incubator/predictionio/0.11.0-
> > >> incubating-rc2/
> > >>
> > >> Test results of RC2 can be found here:
> > >> https://travis-ci.org/apache/incubator-predictionio/builds/220381611
> > >>
> > >> Maven artifacts are built from the release candidate artifacts above,
> > and
> > >> are provided as convenience for testing with engine templates. The
> Maven
> > >> artifacts are provided at the Maven staging repo here:
> > >> https://repository.apache.org/content/repositories/
> > >> orgapachepredictionio-1016/
> > >>
> > >> All JIRAs completed for this release are tagged with 'FixVersion =
> > >> 0.11.0-incubating'. You can view them here:
> > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > >> projectId=12320420=12338381
> > >>
> > >> The artifacts have been signed with Key : 8BF4ABEB
> > >>
> > >> Please vote accordingly:
> > >>
> > >> [ ] +1, accept RC as the official 0.11.0 release
> > >> [ ] -1, do not accept RC as the official 0.11.0 release because...
> > >>
> >
> >
>

Re: [VOTE] Apache PredictionIO (incubating) 0.11.0 Release (RC1)

[Chan Lee]

[+1]


On Tue, Apr 4, 2017 at 11:55 AM, Donald Szeto  wrote:

> This is the vote for 0.11.0 of Apache PredictionIO (incubating).
>
> The vote will run for at least 72 hours and will close on Apr 7th, 2017.
>
> The release candidate artifacts can be downloaded here:
> https://dist.apache.org/repos/dist/dev/incubator/predictionio/0.11.0-
> incubating-rc1/
>
> Test results of RC1 can be found here:
> https://travis-ci.org/apache/incubator-predictionio/builds/218542558
>
> Maven artifacts are built from the release candidate artifacts above, and
> are provided as convenience for testing with engine templates. The Maven
> artifacts are provided at the Maven staging repo here:
> https://repository.apache.org/content/repositories/
> orgapachepredictionio-1015/
>
> All JIRAs completed for this release are tagged with 'FixVersion =
> 0.11.0-incubating'. You can view them here:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12320420=12338381
>
> The artifacts have been signed with Key : 8BF4ABEB
>
> Please vote accordingly:
>
> [ ] +1, accept RC as the official 0.11.0 release
> [ ] -1, do not accept RC as the official 0.11.0 release because...
>

Doc site build now running on Apache Jenkins

[Chan Lee]

Job for building the PredictionIO doc site (
http://predictionio.incubator.apache.org) including the developer api (
http://predictionio.incubator.apache.org/api/current/#package) has been
moved from a private Jenkins to the official Apache Jenkins .

The job consists of an upstream task that generates docs based on poll SCM
of the livedoc branch every 5 minutes. When this has successfully
completed, the downstream task commits the artifacts to asf-site repo.
Details can be found here:
https://builds.apache.org/job/PredictionIO-build-site
https://builds.apache.org/job/PredictionIO-publish-site

Re: Stateless Builds

[Chan Lee]

I'm making additional changes to the stateless build(
https://issues.apache.org/jira/browse/PIO-47), particularly to ensure that
the change does not affect running production servers using PIO.

First, with regards to @Pat's concerns: You'd still be able to run training
on a separate (possibly ephemeral) machine and use the trained ModelData by
providing engine-instance-id as CLI argument. PIO would automatically look
for the latest trained model with the same filepath (engineVersion). I will
add some tests to ensure that this process works.

Also @Pat, I'd appreciate it if you could briefly explain the setup and
common tasks for your production servers to make sure I don't miss anything.
Some immediate questions:
1. When you copy the engine directory on multiple machines, do you include
the .jar files (autogenerated during `pio build` in target/) for all the
machines?
2. Are there any immediate needs/tasks running pio commands outside of the
engine directory?

Thanks,
Chan

On Sun, Dec 11, 2016 at 1:35 PM, Mars Hall  wrote:

> Pat,
>
> Since I represent one of the containerized platforms (Heroku) that needs
> stateless builds (specifically the ability to run a build without a
> database attached) to deploy PIO at its full potential, I would love to be
> able to contribute more to this discussion. Unfortunately I do not
> understand most of the technicalities described here.
>
> How does someone like me learn about this aspect of PIO? Would you wise
> folks step back and describe what the metadata does today?
>
> Are the three PIO_STORAGE types (data, model, & metadata) documented
> clearly anywhere? What are the metadata options, what does it store, & how
> does it effect the engine lifecycle today?
>
> One of the main sources of confusion I've seen from people trying to work
> with PIO is the multiple storage aspects (data, model, & metadata) combined
> with multiple services types (eventserver & engine) and how these all
> interconnect. Plus some engines have a requirement like Elasticsearch, but
> it's not clear where that's required in the grand scheme.
>
> Thanks for all the efforts to move forward with these changes,
>
> *Mars
>
> On Mon, Dec 12, 2016 at 03:13 Pat Ferrel  wrote:
>
>> OK, so that was too much detail. My immediate question is how to train on
>> one machine and deploy on several others—all referencing the same instance
>> data (model)? Before it was by copying the manifest, now there is no
>> manifest.
>>
>>
>>
>> On Dec 7, 2016, at 5:43 PM, Pat Ferrel  wrote:
>>
>> My first question is how to train on an ephemeral machine to swap models
>> into an already deployed prediction server, because this is what i do all
>> the time. The only way to do this now is train first on dummy data then
>> deploy and re-train as data comes in, but there are other issues and
>> questions below. Some may be slightly off topic of this specific PR.
>>
>>
>> > On Dec 5, 2016, at 10:00 AM, Donald Szeto  wrote:
>> >
>> > Hi all,
>> >
>> > I am moving the discussion of stateless build (
>> > https://github.com/apache/incubator-predictionio/pull/328) here.
>> Replying
>> > Pat:
>> >
>> >> BTW @chanlee514 @dszeto Are we thinking of a new command, something
>> like
>> > pio register that would add metadata to the metastore? This would need
>> to
>> > be run every time the engine.json changed for instance? It would also do
>> > not compile? Is there an alternative? What state does this leave us in?
>> >
>> > I imagine we would need pio register after this. Something like what
>> docker
>> > push would do for you today. Changes of engine.json will not require
>> > registration because it is consumed during runtime by pio train and pio
>> > deploy. We are phasing out pio build so that engine templates will be
>> more
>> > friendly with different IDEs.
>>
>> I’m all for removing the manifest and stateless build but I’m not sure we
>> mean the same thing by stateless. My issue is more with stateless commands,
>> or put differently as a fully flexible workflow. Which means all commands
>> read metadata from the metastore, and only one, very explicitly sets
>> metadata into the metastore. Doing the write in train doesn't consider the
>> the deploy before train and multi-tenancy use case.
>>
>> Deploy then train:
>> 1) pio eventserver to start ES on any machine
>> 2) pio deploy to get the query server (prediction server) on any machine
>> 3) pio train at any time on any machine and have a mechanism for deployed
>> engines to discover the metadata they need when they need it or have it
>> automatically updated when changed (pick a method push for deployed engines
>> and pull for train)
>> 4) send input an any time
>>
>> Multi-tenancy:
>> This seems to imply a user visible id for an engine instance id in
>> today’s nomenclature. For multi-tenancy, the user is going to want to set
>> this instance id somewhere and should have stateless 

Re: regarding installation of Middleman

[Chan Lee]

Hi Steven,

This seems to be a general problem with mac os using newer version of gcc,
as described here:
http://stackoverflow.com/questions/19577759/installing-libv8-gem-on-os-x-10-9
.

I'm also using mac os 10.11, and simple `bundle update` solved the problem
for me. Hope this helps.

Chan

On Sun, Oct 16, 2016 at 2:33 PM, Steven Yan 
wrote:

> To whom it may concern,
>
> I am interested in contributing to documentation, so I am following
> http://predictionio.incubator.apache.org/community/
> contribute-documentation/
> on macbook pro, *osx 10.11.6*.
>
> $ cd docs/manual$ gem install bundler$ bundle install$ npm install -g
> bower$ bower install
>
>
> at `bundle install`, it reports
> `An error occurred while installing libv8 (3.16.14.13), and Bundler cannot
> continue.
> Make sure that `gem install libv8 -v '3.16.14.13'` succeeds before
> bundling.
> `
>
> running `gem install libv8 -v '3.16.14.13'`
> will give error
> ../src/objects.h:5252:44: error: shifting a negative signed value is
> undefined [-Werror,-Wshift-negative-value]
>   static const int kElementsKindMask = (-1 << kElementsKindShift) &
> ~~ ^
> ../src/objects.h:7386:36: error: shifting a negative signed value is
> undefined [-Werror,-Wshift-negative-value]
>   (~kMaxCachedArrayIndexLength << kArrayIndexHashLengthShift) |
>~~~ ^
>
> however, running `gem install libv8` will not give error.
>
> after running `gem install libv8`, `bundle install` still gives error,
> insisting I make sure `gem install libv8 -v '3.16.14.13'` works.
>
> Any solution for this problem? Thanks a lot.
>
> Thanks,
> Steven
>

Re: PIO Build Error

[Chan Lee]

Hi Jim,

Could you try removing pio.sbt in the template directory?

Chan

On Wed, Oct 12, 2016 at 8:24 PM, Jim Miller  wrote:

> vagrant:tapster-episode-similar2/ $ pio build --verbose
>
>
>[20:46:39]
> [INFO] [Console$] Using existing engine manifest JSON at
> /home/vagrant/apache-predictionio-0.10.0-incubating/tapster-episode-
> similar2/manifest.json
> [INFO] [Console$] Using command 
> '/home/vagrant/apache-predictionio-0.10.0-incubating/sbt/sbt'
> at the current working directory to build.
> [INFO] [Console$] If the path above is incorrect, this process will fail.
> [INFO] [Console$] Uber JAR disabled. Making sure 
> lib/pio-assembly-0.10.0-incubating.jar
> is absent.
> [INFO] [Console$] Going to run: 
> /home/vagrant/apache-predictionio-0.10.0-incubating/sbt/sbt
>  package assemblyPackageDependency
> [INFO] [Console$] [info] Loading project definition from
> /home/vagrant/apache-predictionio-0.10.0-incubating/tapster-episode-
> similar2/project
> [ERROR] [Console$] /home/vagrant/apache-predictionio-0.10.0-
> incubating/tapster-episode-similar2/pio.sbt:4: error: not found: value
> pioVersion
> [ERROR] [Console$] pioVersion := "0.10.0-incubating"
> [ERROR] [Console$] ^
> [INFO] [Console$] [error] Type error in expression
> [INFO] [Console$] Project loading failed: (r)etry, (q)uit, (l)ast, or
> (i)gnore?
> [ERROR] [Console$] Return code of previous step is 1. Aborting.
>
> --
> Jim Miller
>
> On October 12, 2016 at 11:09:48 PM, Donald Szeto (donald.sz...@gmail.com)
> wrote:
>
> Can you try "pio build --verbose" for detail error?
>
> On Wednesday, October 12, 2016, Jim Miller  wrote:
>
> > I think we’re closer, Donald.
> >
> > 
> > 
> > build.sbt
> > import AssemblyKeys._
> >
> > assemblySettings
> >
> > name := "template-scala-parallel-similarproduct"
> >
> > organization := "org.apache.predictionio"
> >
> > parallelExecution in Test := false
> >
> > test in assembly := {}
> >
> > libraryDependencies ++= Seq(
> > "org.apache.predictionio" %% "apache-predictionio-core" %
> > "0.10.0-incubating" % "provided",
> > "org.apache.spark" %% "spark-core" % "1.3.0" % "provided",
> > "org.apache.spark" %% "spark-mllib" % "1.3.0" % "provided",
> > "org.scalatest" %% "scalatest" % "2.2.1" % "test”)
> > 
> > ***
> > ERROR
> > vagrant:tapster-episode-similar2/ $ pio build
> >
> >
> > [19:46:57]
> > [INFO] [Console$] Using existing engine manifest JSON at
> > /home/vagrant/apache-predictionio-0.10.0-incubating/tapster-episode-
> > similar2/manifest.json
> > [INFO] [Console$] Using command '/home/vagrant/apache-
> predictionio-0.10.0-incubating/sbt/sbt'
> > at the current working directory to build.
> > [INFO] [Console$] If the path above is incorrect, this process will fail.
> > [INFO] [Console$] Uber JAR disabled. Making sure lib/pio-assembly-0.10.0-
> incubating.jar
> > is absent.
> > [INFO] [Console$] Going to run: /home/vagrant/apache-
> predictionio-0.10.0-incubating/sbt/sbt
> > package assemblyPackageDependency
> > [ERROR] [Console$] [error] Type error in expression
> > [ERROR] [Console$] Return code of previous step is 1. Aborting.
> >
> > --
> > Jim Miller
> >
> > On October 12, 2016 at 9:19:32 PM, Donald Szeto (don...@apache.org
> > ) wrote:
> >
> > I think the build plugin is really confusing. Sorry but let's do this:
> >
> > 1. Remove project/pio-build.sbt completely.
> > 2. Replace pioVersion.value with 0.10.0-incubating in build.sbt.
> >
> > That should be it. We are in the process of updating all templates post
> > Apache transition.
> >
> > On Wednesday, October 12, 2016, Jim Miller  > > wrote:
> >
> > > Tom,
> > >
> > > I only found one spot in the pio-build.sbt with 0.9.0 and changed to
> > > 0.10.0 but with the following errors:
> > >
> > > vagrant:tapster-episode-similar2/ $ pio build
> > >
> > >
> > > [18:20:21]
> > > [INFO] [Console$] Using existing engine manifest JSON at
> > > /home/vagrant/apache-predictionio-0.10.0-incubating/tapster-episode-
> > > similar2/manifest.json
> > > [INFO] [Console$] Using command '/home/vagrant/apache-
> > predictionio-0.10.0-incubating/sbt/sbt'
> > > at the current working directory to build.
> > > [INFO] [Console$] If the path above is incorrect, this process will
> fail.
> > > [INFO] [Console$] Uber JAR disabled. Making sure
> lib/pio-assembly-0.10.0-
> > incubating.jar
> > > is absent.
> > > [INFO] [Console$] Going to run: /home/vagrant/apache-
> > predictionio-0.10.0-incubating/sbt/sbt
> > > package assemblyPackageDependency
> > > [ERROR] [Console$] [error] (*:update) sbt.ResolveException: unresolved
> > > dependency: org.apache.predictionio#pio-build;0.10.0: not found
> > > [ERROR] [Console$] Return code of previous step is 1. Aborting.
> > 

Re: templates

[Chan Lee]

e have clarification it's ok.
> >>>
> >>>> On Aug 19, 2016, at 10:09 PM, Pat Ferrel <p...@occamsmachete.com>
> wrote:
> >>>>
> >>>> No misunderstanding—so far at least :-)
> >>>>
> >>>> What we want to do it host the templates on Github as non-Apache
> projects until the donation paper-work is done. This will remove a blocking
> issue—templates that work with the new org namespace and have some (even if
> incomplete) integration tests. We are thinking about the 7 mentioned in
> this Jira: https://issues.apache.org/jira/browse/PIO-24
> >>>>
> >>>> If you want to push them to Github we can work from that. This is
> completely non-Apache but with Apache licenses we should be on sound legal
> ground. PIO was designed with template forking as part of the usage model.
> >>>>
> >>>> If you have any problem with this, that’s fine. I’ll start making the
> same changes you’ve done. I have no problem assuming responsibility since I
> already maintain a couple external templates. I just want to get a release
> of PIO done.
> >>>>
> >>>>
> >>>> On Aug 18, 2016, at 3:31 PM, Chan Lee <chanlee...@gmail.com> wrote:
> >>>>
> >>>> Hi Pat,
> >>>>
> >>>> I think there may have been a misunderstanding. The work I've done is
> only local:
> >>>> 1) I've cloned the 10 official PredictionIO templates and changed the
> org namespace locally.
> >>>> 2) I've added some tests in the main repo to make sure the templates
> are compatible with the latest release.
> >>>>
> >>>> As of now, I don't have write access to PredictionIO org repos, and
> the gallery certainly shouldn't list my fork as the "official" template. So
> if the release must happen before template donation, I would make a PR to
> each of the template repos to change org namespace and update minimum PIO
> version as 0.10. I will leave out the tests for now.
> >>>>
> >>>> But personally, I think it would be better if we wait for the legal
> grant issue to be resolved, so that it is clearer how template code should
> be managed.
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Aug 18, 2016 at 1:13 PM, Pat Ferrel <p...@occamsmachete.com
> <mailto:p...@occamsmachete.com>> wrote:
> >>>> I think we are not waiting for the official template donation to
> release PIO, can you point me to the templates you have working? I’ll make
> sure they get added to the new gallery. We can push them to Apache once the
> grant is done. Thanks for the help.
> >
>

Re: templates

[Chan Lee]

Hi Pat,

I think there may have been a misunderstanding. The work I've done is only
local:
1) I've cloned the 10 official PredictionIO templates and changed the org
namespace locally.
2) I've added some tests in the main repo to make sure the templates are
compatible with the latest release.

As of now, I don't have write access to PredictionIO org repos, and the
gallery certainly shouldn't list my fork as the "official" template. So if
the release must happen before template donation, I would make a PR to each
of the template repos to change org namespace and update minimum PIO
version as 0.10. I will leave out the tests for now.

But personally, I think it would be better if we wait for the legal grant
issue to be resolved, so that it is clearer how template code should be
managed.



On Thu, Aug 18, 2016 at 1:13 PM, Pat Ferrel  wrote:

> I think we are not waiting for the official template donation to release
> PIO, can you point me to the templates you have working? I’ll make sure
> they get added to the new gallery. We can push them to Apache once the
> grant is done. Thanks for the help.

Re: 0.10.0 release

[Chan Lee]

I've changed the namespace for all repos marked "official" except for
Universal Recommender and is running integration tests on them.

Also, I believe the sbt plugin for pio-build needs to be updated.

On Tue, Aug 16, 2016 at 12:50 PM, Pat Ferrel  wrote:

> Let me know if anything is missing: https://issues.
> apache.org/jira/browse/PIO-24
>
>
> On Aug 16, 2016, at 12:26 PM, Pat Ferrel  wrote:
>
> +1
>
>
> On Aug 16, 2016, at 11:35 AM, Donald Szeto  wrote:
>
> 1) I’ll volunteer to take a couple templates and put them in their own
>> repos if we can get some created.
>>
> Sure. I believe Chan might have some converted repos as well.
>
>
>
>> 2) do we have a gallery page yet? we had a good proposal, is it
>> implemented?
>>
> It's already been merged in develop. Once we release the old template
> gallery will become a redirection to the new static page.
>
>
> This is needed only for historical reasons, right? The big blue Template
> button will go there directly after release, I assume.
>
>
>
>> 3) What templates are worth inclusion? Seems like some of the ALS
>> recommender ones are supersets of others do we need them all?
>>
> We should start with a core set of templates and one or a couple that
> focuses on a use case (since they are popular): vanilla, recommendation
> (both Scala and Java), similar product, classification, e-commerce, and
> text classification.
>
>
> sounds good, that is 7. @chan have you scrubbed any of these?
>
>
>
>> 4) do we need “release” of these except through Apache git and Github?
>> Since they are all built by users I suspect the normal Apache release
>> process can be avoided for them, the full push to maven, binary hosting,
>> etc.
>>
> I vote for maintaining the above set of templates to be at least working
> and passing integration tests for the main release. They do not necessarily
> go into the final binary distribution, but they should all be tagged and
> tested against with every release of PredictionIO.
>
>
> +1
>
>
>
>> 5) We need someone to file Infra Jira’s to get repos. If someone can
>> coach me on this I will do it.
>>
> My guess is we can just file an infrastructure ticket to spawn some
> discussion.
>
>
>
> Who did the original request for repo? Can someone point me to the Jira,
> I’ll take it from there
>
>
>> This might only add a week or so to the release date. If we don’t do it I
>> bet it will be much more time spent in support and fixing later.
>>
> Agree. If we all agree let's make this the final major thing to do before
> 0.10.0 release to avoid feature creep.
>
>
> +1, with the possible exception of fixing install.sh https://issues.
> apache.org/jira/browse/PIO-22 One option is to remove it from the
> release, which I favor, discussion on the jira.
>
>
>
>

Re: 0.10.0 release

[Chan Lee]

Regarding 1), I agree with Pat's points for the need to maintain a separate
repository and release cycle for templates. But for the purpose of making
the donation process simple and ensuring consistency of templates with each
new release of PIO, I think having the code reside inside PIO repo would be
the cleanest solution.

How about creating a separate "apache" branch on each official template
repo that is consistent with PIO release? So before every PIO release, we
will merge changes into this "apache" branch, merge them into the main PIO
repo, and run rigorous tests for each template to ensure that they're
compatible.

That way people can continue to clone/fork templates as before, it will be
easy to tag/track template versions compatible with each PIO release, and
we can ensure that all official engines work with each new PIO release
which I feel is most important. Also, as Marcin mentioned, we can expand
tests to keep track of performance changes as well.

Also, I think we should consider merging the current examples/ directory
with the templates/ directory.

Thanks,
Chan


On Sun, Aug 14, 2016 at 8:30 AM, Pat Ferrel <p...@occamsmachete.com> wrote:

> Important things unresolved for release:
>
> 1) What to do with Apache templates—my suggestion is separate repos for
> the 3 reasons below.
> 2) We need a Gallery page listing at least a couple converted tested
> templates—couldn’t find this in the site but maybe I missed it. The UR is
> now converted and tested as are the examples.
> 3) AML fork healed, should be pushed this afternoon.
> 4) the rest of the release magic
>
> PIO is rather moot without templates but if they are on a separate release
> schedule like the doc site (which also needs work) we could release without
> 1 & 2. I’d favor that rather than releasing with templates in a templates/
> directory.
>
> Anything else? How are people feeling about release?
>
>
> On Aug 12, 2016, at 10:18 AM, Pat Ferrel <p...@occamsmachete.com> wrote:
>
> Independent of Apache I’d suggest each template have their own git repo as
> they do now. Is this practically possible in ASF (another example of my
> infrastructure rant but leave that for now). The semantics of this make
> sense. The requirements for all apache or non-apache templates seem to be:
>
> 1) They need to be allowed to have separate release cycles.
> 2) There should be only one way to install templates apache or non-apache
> 3) This method should support tags and branches, and separate PRs
>
> Practically speaking for independent ones (the Universal Recommender will
> remain independent until the issues above are resolved) the templates
> *will* be a separate repo and `git pull` *will* be the method of download.
> To me this implies the Apache ones should use the same pattern.
>
> The separate issue of rules for inclusion in Apache should include a test
> requirement IMO. But inclusion in the Gallery seems a looser attachment to
> the project—just one persons opinion
>
>
> On Aug 11, 2016, at 3:42 PM, Chan Lee <chanlee...@gmail.com> wrote:
>
> I've begun working on migrating official PredictionIO templates into the
> main apache repository, and there are a few points I'd like to bring to
> attention.
>
> 1) Template code will be merged under templates/ directory, and all commit
> history and authorship will be preserved.
>
> 2) Organization namespace will change in the template code from
> "io.prediction" to "org.apache.predictionio".
>
> 3) In order to download an engine template, users could (1) git clone the
> entire repo and copy the files in templates/, or (2) use
> something like svn to download only the necessary subdirectory. I'd like to
> ask what everyone thinks should be the recommended approach. The
> documentation should be updated accordingly (instead of `pio template get`).
>
> 4) I'd like to ask if there are any ActionML/outside templates that should
> be included or updated. For instance, 
> template-scala-parallel-universal-recommendation
> in PredictionIO repo seems outdate compared to one in actionml repo.
>
>
> Thanks,
> Chan
>
>
>
> On Mon, Aug 8, 2016 at 9:16 AM, Marcin Ziemiński <ziem...@gmail.com>
> wrote:
>
>> I think that this is a very good idea. Obviously it would require
>> configuring more concurrent builds to finish in reasonable time, what
>> should not be a problem. Besides, it would make the official templates be
>> consistent with every next release of PredictionIO and what is very
>> important - it would provide many different tests for the repository.
>> Having working templates of different types could give us more material to
>> test not only the reliability of PredictionIO,

Re: transition from "official" PredictionIO templates to Apache

[Chan Lee]

I've begun working on migrating official PredictionIO templates into the
main apache repository, and there are a few points I'd like to bring to
attention.

1) Template code will be merged under templates/ directory, and all commit
history and authorship will be preserved.

2) Organization namespace will change in the template code from
"io.prediction" to "org.apache.predictionio".

3) In order to download an engine template, users could (1) git clone the
entire repo and copy the files in templates/, or (2) use
something like svn to download only the necessary subdirectory. I'd like to
ask what everyone thinks should be the recommended approach. The
documentation should be updated accordingly (instead of `pio template get`).

4) I'd like to ask if there are any ActionML/outside templates that should
be included or updated. For instance,
template-scala-parallel-universal-recommendation
in PredictionIO repo seems outdate compared to one in actionml repo.


Thanks,
Chan



On Mon, Aug 8, 2016 at 9:16 AM, Marcin Ziemiński  wrote:

> I think that this is a very good idea. Obviously it would require
> configuring more concurrent builds to finish in reasonable time, what
> should not be a problem. Besides, it would make the official templates be
> consistent with every next release of PredictionIO and what is very
> important - it would provide many different tests for the repository.
> Having working templates of different types could give us more material to
> test not only the reliability of PredictionIO, but also its performance
> with every change. It would require adding different types of tests and
> tools to gather runtime statistics, but this is something I would like to
> take a look at in the future.
>
> niedz., 7.08.2016 o 10:21 użytkownik Donald Szeto 
> napisał:
>
> > We can also make these templates part of the integration test that Chan
> and
> > Marcin just submitted (
> > https://github.com/apache/incubator-predictionio/pull/267). That way we
> > can
> > make commitment to included templates be part of every committer's
> effort.
> > Just my 2c.
> >
> > On Sun, Aug 7, 2016 at 9:47 AM, Pat Ferrel 
> wrote:
> >
> > > If this sound ok, I propose the process be:
> > > 1) inclusion in the gallery is a PR to the yaml gallery file. This is
> > > reviewable by all but takes only one committer to approve and merge.
> > > 2) submission for inclusion in the project can be a PR to the new
> > > templates directory as Simon suggests. But this may require a more
> formal
> > > vote, and come with some commitment for support and possibly
> > consideration
> > > of the author for committer status.
> > >
> > > Someone who knows Apache rules better than me may know the type of vote
> > to
> > > have for #2, maybe this is only informal review and single committer
> > > acceptance too as long as the committer is willing to support the
> > template.
> > >
> > > As far as the Salesforce templates marked official, I’d hope that most
> > > would be donated. According to the proposed rules all we need is a PR
> for
> > > each. And again a big thanks to Salesforce!
> > >
> > >
> > > On Aug 4, 2016, at 2:55 PM, Pat Ferrel  wrote:
> > >
> > > Actually this is mostly a fine idea but I think the bigger question is
> > how
> > > do we treat templates in general.
> > >
> > > IMO the maintaining author can decide to contribute them or not and the
> > > committers can decide to accept or not. For example in the case of the
> > UR I
> > > may decide to support and maintain it outside of Apache while some from
> > > Salesforce might be submitted as donations.  Donation comes at some
> > > obligation. There are lots of examples in Mahout of algorithms whose
> > > authors no longer support them. These are slowly being deprecated and
> > > removed so I’d like to see a method that avoids this issue.
> > >
> > > If we allow maintainers to submit templates to the Gallery with a link
> to
> > > code as well as support then donating the template code to Apache is
> > > independent of acceptance to the Gallery. Any template donated to
> Apache
> > > should come with some commitment by the author to support it there
> > > indefinitely and perhaps even acceptance as a PIO committer—and if they
> > > disappear or don’t support the template it is easy to deprecate/remove.
> > >
> > > That means if Salesforce wants to donate the templates it
> > maintains—great.
> > > Personally I'd would vote for acceptance of most of them. Then the
> > support
> > > link can be to the Apache user list or instructions for joining it. If
> a
> > > maintainer wants to stay out of the Apache repo and support separately
> > then
> > > this is possible also. With all templates treated equally—official or
> > > not—and there is a route to becoming official for non-official ones.
> > >
> > > BTW it might be good to remove the “official” designation in favor of
> > > “Apache 

Re: PIO-20 problems

[Chan Lee]

I want to clarify some points:

2) Deploying templates do not require SSL. If you execute
./make-distribution.sh using the current develop branch (provided you
change the namespace in the template from io.prediction to
org.apache.predictionio), you can deploy without SSL on localhost. The
travis tests also do not use SSL and pass as can be seen in
https://travis-ci.org/Ziemin/incubator-predictionio.

3) You can run the tests locally the same way it is run on travis ('python3
${PIO_DIR}/tests/pio_tests/tests.py'). It is documented in the README in
the same directory, but maybe this was not clear enough. If you have any
additional questions or run into issues executing the tests, please let me
know.

Thanks,
Chan

On Thu, Aug 11, 2016 at 6:07 AM, Pat Ferrel  wrote:

> I’m always disinclined to commit code that has not been tested. However
> many things in the current state of the develop branch seem broken or at
> least too big to address in a single commit.
>
> 1) Style test fail in Travis that are not runnable or at least have not
> been documented for local build. I have fixed these.
> 2) SSL seems to still be required for deploying an engine, PIO-1 may not
> have addressed this. Has anyone tried to build and test a template yet
> without SSL? Unless someone can state otherwise I’m inclined to ignore this
> failure for now since it is out of scope for this commit.
> 3) the test framework has been integrated into Travis but again no
> documented way to run it locally and some of the travis errors seem
> spurious. I am also not inclined to wait for this to be fixed unless
> someone can point out real problems it is discovering and send instructions
> for how to run it locally.
>
> Before the extra travis tests I was able to build and train a template,
> which fails on deploy with an SSL config error. Since the PR does not
> strictly touch any templates I am going to have to ignore this serious
> issue and address it in Jira.
>
> Unless someone vetos this I plan to reluctantly push the PR with these
> non-trivial issues remaining.
>
>
>