[
https://issues.apache.org/jira/browse/PIO-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716175#comment-15716175
]
ASF GitHub Bot commented on PIO-47:
-----------------------------------
Github user pferrel commented on the issue:
https://github.com/apache/incubator-predictionio/pull/328
@dszeto asked: "Do you currently have any production deployments that rely
heavily on engine IDs and versions? That would be a bigger immediate concern
regarding this change."
Yes. That is why so many questions above. We create a pio driver machine
that runs training and is then taken down. This is in sync with bringing up
Spark and taking it down. The pio driver machine is really part of the cluster
and for the UR (and other PIO templates we have) the predict part runs without
Spark. So this requires we use some engine instance id across machines to find
data in the metastore. We also have deployments with 2 EventServers and 2
PredictionServers behind a load balancer. This requires that the same metadata
is available to several different machines running, what I've come to see as 3
independent processes, ES, PS, train. They are independent except for the
metadata.
> Remove engine manifest for stateless build
> ------------------------------------------
>
> Key: PIO-47
> URL: https://issues.apache.org/jira/browse/PIO-47
> Project: PredictionIO
> Issue Type: New Feature
> Reporter: Chan
>
> As discussed in the dev mailing list, removing engine manifest would be the
> first step in improving the workflow towards a more modular design.
> - Remove manifest.json completely. `pio build` will be stateless, and will
> not write anything to the database. This will make it easier to compile/build
> on PaaS platforms such as Heroku. Later, we can remove `pio build` command
> entirely, so that PIO is independent of the build tool (sbt).
> - An immediate major disadvantage would be not being able to run pio commands
> outside of the engine directory. This can be resolved in the next step of
> creating a general metadata registry.
> - Meanwhile, we can use engineFactory as *engineId* , and SHA-1 hash of
> engine filepath as *engineVersion* (as before). We can improve this when
> designing a metadata registry,
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
