This is how the Flink guys are doing it -
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65870673
They call them FLIP (Flink Improvement Process) similar to KLIP in Kafka.
You could use a shared google doc as well as the Wiki and Mailing lists.
Make sure that the Wiki is the
What do you think about using a general purpose registry, that can also be used
to discover cluster machines, or microservices?
Something like consul.io or docker swarm with and ASF compatible license? This
would be a real step into the future and since some work is needed anyway…
I think
I think that getting rid of the manifest.json and introducing a new kind of
resourse-id for an engine to be registered is a good idea.
Currently in the repository there are three important keys:
* engine id
* engine version - depends only on the path the engine was built at to
distinguish copies
This sounds like a good case for Donald’s suggestion.
What I was trying to add to the discussion is a way to make all commands rely
on state in the megastore, rather than any file on any machine in a cluster or
on ordering of execution or execution from a location in a directory structure.
Hello folks,
Great to hear about this possibility. I've been working on running PredictionIO
on Heroku https://www.heroku.com
Heroku's 12-factor architecture https://12factor.net prefers "stateless builds"
to ensure that compiled artifacts result in processes which may be cheaply
restarted,
Yes, a new thing though it might serve some of the same purposes. The idea is
to only use engine instance information from the metadata store so the template
commands will work from anywhere and mostly in any order. On a cluster machine
if the engine instance data is in the metastore and the
Pat, would you explain more about the 'instanceId' as in
`pio register --variant path/to/some-engine.json --instanceId
some-REST-compatible-resource-id` ?
Currently PIO also has a concept of engineInstanceId, which is output of
train. I think you are referring to different thing, right?
Kenneth
This is a great discussion topic and a great idea.
However the cons must also be addressed, we will need to do this before
multi-tenant deploys can happen and the benefits are just as large as removing
`pio build`
It would be great to get rid of manifest.json and put all metadata in the store
Hi all,
I want to start the discussion of removing engine registration. How many
people actually take advantage of being able to run pio commands everywhere
outside of an engine template directory? This will be a nontrivial change
on the operational side so I want to gauge the potential impact to
