PredictionIO is completely useless without a Template yet we seem as a group
too focused on releasing PIO without regard for Templates. This IMO must
change. 90% of users will never touch the code of a template and only 1% will
actually create a template. These guesses come from list questions. If this is
true we need to switch our mindset to "templates first” not “pio first”. Before
any upgrade vote, every committer should make sure their favorite template
works with the new build. I will be doing so from now on.
We have one fairly significant problem that I see from a template supporter's
side. PIO has several new build directives that change dependencies like Spark
version and tools like Scala version. These are unknown to templates and there
is no PIO supported way to communicate these to the template's build.sbt. This
leaves us with templates that will not work with most combinations of PIO
builds. If we are lucky they may be updated to work with the *default* pio
config. But this did not happen when PIO-0.12.0 was released, only shortly
afterwards. This must change, the Apache templates at least must have some
support for PIO before release and here is one idea that might help...
How do we solve this?
1) .make-distribution modifies or creates a script that can be imported by the
templates build.sbt. This might be pio-env if we use `pio build` to build
templates because it is available to the template’s build.sbt, or something
else when we move to using sbt to build templates directly. This script defines
values used to build PIO.
2) update some or all of the Apache templates to use this mechanism to build
with the right scala version, etc. taken from the PIO build.
I had a user do this for the UR to support many different pio build directives,
and some that are new. The result was a build.sbt that includes such things as
val pioVersion = sys.env.getOrElse("PIO_VERSION","0.12.0-incubating”)
val scalaVersion = sys.env.getOrElse(“PIO_SCALA_VERSION”, “2.10.0”)
val elasticsearch1Version = sys.env.getOrElse("PIO_ELASTIC_VERSION","1.7.5")
val sparkVersion = sys.env.getOrElse("PIO_SPARK_VERSION","1.4.0”)
these are then used in the lib dependencies lists to pull in the right versions
of artifacts.
This in some form would allow templates to move along in lock step with changes
in the way pio is built on any given machine. Without something like this,
users even less expert at sbt than myself (hard to imagine) will have a
significant problem dumped on them.
Since this is only partially baked it may not be ready for a Jira and so
warrants discussion.
