[
https://issues.apache.org/jira/browse/OOZIE-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purshotam Shah updated OOZIE-1825:
----------------------------------
Description:
Currently protoconf stores all Hadoop configuration properties + libpath + ...
Hadoop configuration properties are lot and take huge space, I don't think we
need to store Hadoop configuration properties in protoconf.
We can load it every time wf action runs.
Some statistics on table storage.
|| Table || Size(gb) ||
|WFJOBS_PROTO_ACTION_CONF|55.54|
|WFJOBS_WF_INSTANCE|11.89|
|WFJOBS_CONF|2.43|
If we can remove Hadoop configuration form proto, size will be < 1gb.
was:
Currently protoconf stores all Hadoop configuration properties + libpath + ...
Hadoop configuration properties are lot and take huge space, I don't think we
need to store Hadoop configuration properties in protoconf.
We can load it every time wf action runs.
Some statistics on table storage.
Table size(gb)
WFJOBS_PROTO_ACTION_CONF 55.54
WFJOBS_WF_INSTANCE 11.89
WFJOBS_CONF 2.43
If we can remove Hadoop configuration form proto, size will be < 1gb.
> Optimize wf_jobs protoconf storage
> ----------------------------------
>
> Key: OOZIE-1825
> URL: https://issues.apache.org/jira/browse/OOZIE-1825
> Project: Oozie
> Issue Type: Bug
> Reporter: Purshotam Shah
>
> Currently protoconf stores all Hadoop configuration properties + libpath + ...
> Hadoop configuration properties are lot and take huge space, I don't think we
> need to store Hadoop configuration properties in protoconf.
> We can load it every time wf action runs.
> Some statistics on table storage.
> || Table || Size(gb) ||
> |WFJOBS_PROTO_ACTION_CONF|55.54|
> |WFJOBS_WF_INSTANCE|11.89|
> |WFJOBS_CONF|2.43|
> If we can remove Hadoop configuration form proto, size will be < 1gb.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
