I think the problem with templates in examples is that they are already modified and it will be hard to merge any upstream template changes back to these examples. We need a method to consistently test and support official templates. We can probably use separate repositories, but I am not sure if we have enough time if we are planning to release 0.10.0 this week. We probably want GitHub integration for all these official template repos as well.
I do not have a strong opinion in mandating one engine one repo, but I think otherwise it would be hard for customized engines to merge upstream official changes. On Mon, Aug 15, 2016 at 3:05 PM, Pat Ferrel <p...@occamsmachete.com> wrote: > Possible but we have template in examples/ for testing and that’s ok imo. > If we think separate repos is the way to go why not ask? > > @mentors how hard would it be to get say 5 new repos tied to github > mirrors? > > > On Aug 15, 2016, at 1:47 PM, Chan Lee <chanlee...@gmail.com> wrote: > > Regarding 1), I agree with Pat's points for the need to maintain a > separate repository and release cycle for templates. But for the purpose of > making the donation process simple and ensuring consistency of templates > with each new release of PIO, I think having the code reside inside PIO > repo would be the cleanest solution. > > How about creating a separate "apache" branch on each official template > repo that is consistent with PIO release? So before every PIO release, we > will merge changes into this "apache" branch, merge them into the main PIO > repo, and run rigorous tests for each template to ensure that they're > compatible. > > That way people can continue to clone/fork templates as before, it will be > easy to tag/track template versions compatible with each PIO release, and > we can ensure that all official engines work with each new PIO release > which I feel is most important. Also, as Marcin mentioned, we can expand > tests to keep track of performance changes as well. > > Also, I think we should consider merging the current examples/ directory > with the templates/ directory. > > Thanks, > Chan > > > On Sun, Aug 14, 2016 at 8:30 AM, Pat Ferrel <p...@occamsmachete.com > <mailto:p...@occamsmachete.com>> wrote: > Important things unresolved for release: > > 1) What to do with Apache templates—my suggestion is separate repos for > the 3 reasons below. > 2) We need a Gallery page listing at least a couple converted tested > templates—couldn’t find this in the site but maybe I missed it. The UR is > now converted and tested as are the examples. > 3) AML fork healed, should be pushed this afternoon. > 4) the rest of the release magic > > PIO is rather moot without templates but if they are on a separate release > schedule like the doc site (which also needs work) we could release without > 1 & 2. I’d favor that rather than releasing with templates in a templates/ > directory. > > Anything else? How are people feeling about release? > > > On Aug 12, 2016, at 10:18 AM, Pat Ferrel <p...@occamsmachete.com <mailto: > p...@occamsmachete.com>> wrote: > > Independent of Apache I’d suggest each template have their own git repo as > they do now. Is this practically possible in ASF (another example of my > infrastructure rant but leave that for now). The semantics of this make > sense. The requirements for all apache or non-apache templates seem to be: > > 1) They need to be allowed to have separate release cycles. > 2) There should be only one way to install templates apache or non-apache > 3) This method should support tags and branches, and separate PRs > > Practically speaking for independent ones (the Universal Recommender will > remain independent until the issues above are resolved) the templates > *will* be a separate repo and `git pull` *will* be the method of download. > To me this implies the Apache ones should use the same pattern. > > The separate issue of rules for inclusion in Apache should include a test > requirement IMO. But inclusion in the Gallery seems a looser attachment to > the project—just one persons opinion > > > On Aug 11, 2016, at 3:42 PM, Chan Lee <chanlee...@gmail.com <mailto: > chanlee...@gmail.com>> wrote: > > I've begun working on migrating official PredictionIO templates into the > main apache repository, and there are a few points I'd like to bring to > attention. > > 1) Template code will be merged under templates/ directory, and all commit > history and authorship will be preserved. > > 2) Organization namespace will change in the template code from > "io.prediction" to "org.apache.predictionio". > > 3) In order to download an engine template, users could (1) git clone the > entire repo and copy the files in templates/<some-template>, or (2) use > something like svn to download only the necessary subdirectory. I'd like to > ask what everyone thinks should be the recommended approach. The > documentation should be updated accordingly (instead of `pio template get`). > > 4) I'd like to ask if there are any ActionML/outside templates that should > be included or updated. For instance, > template-scala-parallel-universal-recommendation > in PredictionIO repo seems outdate compared to one in actionml repo. > > > Thanks, > Chan > > > > On Mon, Aug 8, 2016 at 9:16 AM, Marcin Ziemiński <ziem...@gmail.com > <mailto:ziem...@gmail.com>> wrote: > I think that this is a very good idea. Obviously it would require > configuring more concurrent builds to finish in reasonable time, what > should not be a problem. Besides, it would make the official templates be > consistent with every next release of PredictionIO and what is very > important - it would provide many different tests for the repository. > Having working templates of different types could give us more material to > test not only the reliability of PredictionIO, but also its performance > with every change. It would require adding different types of tests and > tools to gather runtime statistics, but this is something I would like to > take a look at in the future. > > niedz., 7.08.2016 o 10:21 użytkownik Donald Szeto <don...@apache.org > <mailto:don...@apache.org>> > napisał: > > > We can also make these templates part of the integration test that Chan > and > > Marcin just submitted ( > > https://github.com/apache/incubator-predictionio/pull/267 < > https://github.com/apache/incubator-predictionio/pull/267>). That way we > > can > > make commitment to included templates be part of every committer's > effort. > > Just my 2c. > > > > On Sun, Aug 7, 2016 at 9:47 AM, Pat Ferrel <p...@occamsmachete.com > <mailto:p...@occamsmachete.com>> wrote: > > > > > If this sound ok, I propose the process be: > > > 1) inclusion in the gallery is a PR to the yaml gallery file. This is > > > reviewable by all but takes only one committer to approve and merge. > > > 2) submission for inclusion in the project can be a PR to the new > > > templates directory as Simon suggests. But this may require a more > formal > > > vote, and come with some commitment for support and possibly > > consideration > > > of the author for committer status. > > > > > > Someone who knows Apache rules better than me may know the type of vote > > to > > > have for #2, maybe this is only informal review and single committer > > > acceptance too as long as the committer is willing to support the > > template. > > > > > > As far as the Salesforce templates marked official, I’d hope that most > > > would be donated. According to the proposed rules all we need is a PR > for > > > each. And again a big thanks to Salesforce! > > > > > > > > > On Aug 4, 2016, at 2:55 PM, Pat Ferrel <p...@occamsmachete.com <mailto: > p...@occamsmachete.com>> wrote: > > > > > > Actually this is mostly a fine idea but I think the bigger question is > > how > > > do we treat templates in general. > > > > > > IMO the maintaining author can decide to contribute them or not and the > > > committers can decide to accept or not. For example in the case of the > > UR I > > > may decide to support and maintain it outside of Apache while some from > > > Salesforce might be submitted as donations. Donation comes at some > > > obligation. There are lots of examples in Mahout of algorithms whose > > > authors no longer support them. These are slowly being deprecated and > > > removed so I’d like to see a method that avoids this issue. > > > > > > If we allow maintainers to submit templates to the Gallery with a link > to > > > code as well as support then donating the template code to Apache is > > > independent of acceptance to the Gallery. Any template donated to > Apache > > > should come with some commitment by the author to support it there > > > indefinitely and perhaps even acceptance as a PIO committer—and if they > > > disappear or don’t support the template it is easy to deprecate/remove. > > > > > > That means if Salesforce wants to donate the templates it > > maintains—great. > > > Personally I'd would vote for acceptance of most of them. Then the > > support > > > link can be to the Apache user list or instructions for joining it. If > a > > > maintainer wants to stay out of the Apache repo and support separately > > then > > > this is possible also. With all templates treated equally—official or > > > not—and there is a route to becoming official for non-official ones. > > > > > > BTW it might be good to remove the “official” designation in favor of > > > “Apache supported” or the PredictionIO logo or something like that. We > > > really need to encourage template development even if they are not code > > > contributions. > > > > > > > > > On Aug 3, 2016, at 3:32 PM, Simon Chan <si...@salesforce.com <mailto: > si...@salesforce.com>> wrote: > > > > > > Hey guys, > > > > > > I just want to start the discussion about moving previously "official" > > > PredictionIO engine templates into Apache. > > > > > > Official templates are those marked "Official" here: > > > http://templates.prediction.io/ <http://templates.prediction.io/> > > > > > > In order to move them to ASF Git, would the best way be creating > > > sub-directory under PredictionIO repo? > > > > > > Simon > > > > > > > > > > > > > > > > >
