On 01/20/2014 12:50 PM, Andrey Lazarev wrote:
Inlined.
On Mon, Jan 20, 2014 at 8:15 AM, Matthew Farrellee <m...@redhat.com
<mailto:m...@redhat.com>> wrote:
(inline, trying to make this readable by a text-only mail client
that doesn't use tabs to indicate quoting)
On 01/20/2014 02:50 AM, Andrey Lazarev wrote:
------
FIX - @rest.get('/jobs/config-hints/____<job_type>') -
should move to
GET /plugins/<plugin_name>/<____plugin_version>, similar to
get_node_processes
and get_required_image_tags
------
Not sure if it should be plugin specific right now. EDP
uses it
to show some
configs to users in the dashboard. it's just a cosmetic
thing.
Also when user
starts define some configs for some job he might not define
cluster yet and
thus plugin to run this job. I think we should leave it
as is
and leave only
abstract configs like Mapper/Reducer class and allow
users to
apply any
key/value configs if needed.
FYI, the code contains comments suggesting it should be
plugin specific.
https://github.com/openstack/____savanna/blob/master/savanna/____service/edp/workflow_creator/____workflow_factory.py#L179
<https://github.com/openstack/__savanna/blob/master/savanna/__service/edp/workflow_creator/__workflow_factory.py#L179>
<https://github.com/openstack/__savanna/blob/master/savanna/__service/edp/workflow_creator/__workflow_factory.py#L179
<https://github.com/openstack/savanna/blob/master/savanna/service/edp/workflow_creator/workflow_factory.py#L179>>
IMHO, the EDP should have no plugin specific dependencies.
If it currently does, we should look into why and see if we
can't
eliminate this entirely.
[AL] EDP uses plugins in two ways:
1. for HDFS user
2. for config hints
I think both items should not be plugin specific on EDP API
level. But
implementation should go to plugin and call plugin API for result.
In fact they are both plugin specific. The user is forced to click
through a plugin selection (when launching a job on transient
cluster) or the plugin selection has already occurred (when
launching a job on an existing cluster).
Since the config is something that is plugin specific, you might not
have hbase hints from vanilla but you would from hdp, and you
already have plugin information whenever you ask for a hint, my view
that this be under the /plugins namespace is growing stronger.
[AL] Disagree. They are plugin specific, but EDP itself could have
additional plugin-independent logic inside. Now config hints return EDP
properties (like mapred.input.dir) as well as plugin-specific
properties. Placing it under /plugins namespace will give a vision that
it is fully plugin specific.
I like to see EDP API fully plugin independent and in one workspace. If
core side needs some information internally it can easily go into the
plugin.
I'm not sure if we're disagreeing. We may, in fact, be in violent agreement.
The EDP API is fully plugin independent, and should stay that way as a
project goal. config-hints is extra data that the horizon app can use to
help give users suggestions about what config they may want to
optionally add to their job. Those config options are independent of the
job and specific to the cluster where the job will run, which is the
purview of the plugin.
Moving config-hints out of the EDP API will make this even more clear.
Best,
matt
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
