Technically, you can have the value of hdp_version in your
spark-defaults.xml. But I hope you might have already considered the
impacts due to upgrade, etc.
Case1:
if you are not exposing it on UI then you can use template with variable
substitution..
step1: put a template for spark-default.xml under package/templates with
the required label.
ex spark-default.xml.j2
<property>hdp_version</property>
<value>{{hdp_version_derived}}</value>
step2: derive the value of hdp_version_derived in params.py. (let's say
after derivation)
hdp_version_derived = 2.3
step3: Write it to the filesystem using File Resource.
ex: File(target_file, template(your_template), owner=required_owner,
group=required_group)
Case2:
if you are exposing it UI, template's with variable substitution cannot be
used. You will have to perform the below
step1. expose a configuration file under configurations directory (with
some default for hdp_version)
step2. read the properties when service is loaded / started
step3. Propagate the value to filesystem. (XmlConfig can be used to write
Xml files with required values)
Be advised for case2, if you update the value at backend by some logic in
your code, your UI config display may not be in sync with the actual value
written on the filesystem and you will need to update the correct value on
UI if you want them to be in sync. This is kind of hack and not recommended.
Let me know if there are any questions.
Thanks,
Bhuvnesh Chaudhary
Email: bchau <[email protected]>[email protected]
Desk: +1-650-846-1696 | Mobile: +1-973-906-6976
On Mon, Jan 4, 2016 at 8:11 PM, Jeff Zhang <[email protected]> wrote:
> I am trying to create configuration file (spark-defaults.xml) for my
> component
>
> This configuration is very similar to hadoop configuration file. It seems
> support variable substitution. and can I use hdp_version variable (or any
> other variable represent hdp_version) in this configuration file ? Thanks
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
