[Puppet-dev] Re: Hiera Merge
On 10/08/17 20:48, ggun wrote:
Thanks, but the main point to merge the hash is that the sub hash header
value is not fixed in the example I have mentioned as
esa-user-profile-service or esa-group-service. It can be anything. I
need to make the code free from using direct Key value .
So 'was_dmgr_data' is a hash of unknown hashes. Did we understand
correctly that the "common part" is some kind of default?
Can you change the original structure, or are you stuck with this design?
In puppet 5.0.0 there is a function named 'tree_each' that can be used
together with 'map()' and the result of that fed into a Hash.new with an
option of 'tree', or 'tree_hash'. This is a very powerful function but
requires a little bit of setup.
I also included a simpler variant using the reduce() function.
That works well here since the task is to map one level of data in a
known way, so using tree_each is a bit of an overkill.
Using tree_each
===
-example.pp
# Sample data for illustration
$hash = {
common => { x => 10, y => 20},
was_data => {
thing => {a => 'in thing'},
other_thing => {a => 'in other thing', x => 1000 }
}
}
# The common to part to merge as defaults
#
$common = $hash[common]
# Map the tree to a flat structure. Skip the root
# (we want data one level down),
# and skip all values as they are included in the hash containers (and
# we do not need to transform them)
#
$flat_tree = $hash[was_data]
.tree_each({include_root => false, include_values => false} )
.map |$entry| {
# assign the path and value in $entry to separate variables
[$path, $val] = $entry
# only map top level keys (the path has length == 1) to a new value
# and make all other entries undef.
# The mapped entries is the common part with the specific data
# merged ($common + $val)
#
if $path =~ Array[1,1] {
[ $path, $common + $val ]
}
}.filter |$x| { $x != undef } # filter out the undef entries
# create a new hash and notice it
notice Hash($flat_tree, 'hash_tree')
--
When running this the output would be:
{thing => {x => 10, y => 20, a => in thing},
other_thing => {x => 1000, y => 20, a => in other thing}}
As you can see the result is a hash with the keys under 'was_data'
with the expected merged data.
Using reduce
If you are on a version before Puppet 5.0.0 it can be done with the
reduce function.
-example.pp
# Sample data for illustration
$hash = {
common => { x => 10, y => 20},
was_data => {
thing => {a => 'in thing'},
other_thing => {a => 'in other thing', x => 1000 }
}
}
$common = $hash[common]
$mapped = $hash[was_data].reduce({}) |$memo, $entry| {
[$k, $v] = $entry
$memo + { $k => $common + $v}
}
notice $mapped
-
I hope these two examples help.
Regards
- henrik
On Wednesday, August 9, 2017 at 2:24:34 PM UTC-4, Reid Vandewiele wrote:
If you're just trying to transform the data in Puppet code and
assuming (as Henrik was) that you can't change how the data is
stored, something like this might work.
# Assuming $was_data is the hash of data from Hiera
$common_data = $was_data.filter |$pair| { $pair[0] != 'was_dmgr_data' }
$hash1 = {'esa-group-service'=> $common_data +
$was_data['was_dmgr_data']['esa-group-service'] }
$hash2 = {'esa-user-profile-service' => $common_data +
$was_data['was_dmgr_data']['esa-user-profile-service'] }
~Reid
On Wednesday, August 9, 2017 at 6:15:41 AM UTC-7, ggun wrote:
Thanks
On Tuesday, August 8, 2017 at 7:10:13 PM UTC-4, ggun wrote:
Hi Experts,
I have a requirement as below.
I need to create a Hash from below hiera data.
was_data:
hs3sourcepath: 'glic.binaries/websphere'
hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
hibmagentpath:
/opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
hbase_dir: '/opt/was/was855'
hinstance_name: WebSphere
was_dmgr_data:
esa-group-service:
hgroup: websph
hdmgr_profile: TST
hdmgr_cell: CELL
hcluster_name: CLUSTER
hpptdmgrsrvport: 8080
esa-user-profile-service:
hdmgr_profile: ABC
hdmgr_cell: PQS
hcluster_name: IOP
hpptdmgrsrvport:
I need a hash of above data as
Hash 1 :
esa-group-service:
hgroup: websph
hdmgr_profile: TST
hdmgr_cell: CELL
hcluster_name: CLUSTER
hpptdmgrsrvport: 8080
hs3sourcepath: 'glic.binaries/websphere'
hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
[Puppet-dev] Re: Hiera Merge
Thanks, but the main point to merge the hash is that the sub hash header
value is not fixed in the example I have mentioned as
esa-user-profile-service or esa-group-service. It can be anything. I need
to make the code free from using direct Key value .
On Wednesday, August 9, 2017 at 2:24:34 PM UTC-4, Reid Vandewiele wrote:
>
> If you're just trying to transform the data in Puppet code and assuming
> (as Henrik was) that you can't change how the data is stored, something
> like this might work.
>
> # Assuming $was_data is the hash of data from Hiera
> $common_data = $was_data.filter |$pair| { $pair[0] != 'was_dmgr_data' }
> $hash1 = {'esa-group-service'=> $common_data +
> $was_data['was_dmgr_data']['esa-group-service'] }
> $hash2 = {'esa-user-profile-service' => $common_data +
> $was_data['was_dmgr_data']['esa-user-profile-service'] }
>
> ~Reid
>
> On Wednesday, August 9, 2017 at 6:15:41 AM UTC-7, ggun wrote:
>>
>> Thanks
>>
>> On Tuesday, August 8, 2017 at 7:10:13 PM UTC-4, ggun wrote:
>>>
>>> Hi Experts,
>>>
>>> I have a requirement as below.
>>> I need to create a Hash from below hiera data.
>>>
>>> was_data:
>>> hs3sourcepath: 'glic.binaries/websphere'
>>> hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
>>> hibmagentpath:
>>> /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
>>> hbase_dir: '/opt/was/was855'
>>> hinstance_name: WebSphere
>>> was_dmgr_data:
>>> esa-group-service:
>>> hgroup: websph
>>> hdmgr_profile: TST
>>> hdmgr_cell: CELL
>>> hcluster_name: CLUSTER
>>> hpptdmgrsrvport: 8080
>>> esa-user-profile-service:
>>> hdmgr_profile: ABC
>>> hdmgr_cell: PQS
>>> hcluster_name: IOP
>>> hpptdmgrsrvport:
>>>
>>>
>>> I need a hash of above data as
>>> Hash 1 :
>>> esa-group-service:
>>> hgroup: websph
>>> hdmgr_profile: TST
>>> hdmgr_cell: CELL
>>> hcluster_name: CLUSTER
>>> hpptdmgrsrvport: 8080
>>> hs3sourcepath: 'glic.binaries/websphere'
>>> hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
>>> hibmagentpath:
>>> /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
>>> hbase_dir: '/opt/was/was855'
>>> hinstance_name: WebSphere
>>>
>>> Hash 2
>>> esa-user-profile-service:
>>> hdmgr_profile: ABC
>>> hdmgr_cell: PQS
>>> hcluster_name: IOP
>>> hpptdmgrsrvport:
>>> hs3sourcepath: 'glic.binaries/websphere'
>>> hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
>>> hibmagentpath:
>>> /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
>>> hbase_dir: '/opt/was/was855'
>>> hinstance_name: WebSphere
>>>
>>> So I trying to merge the has of esa-group-service to was_data and
>>> esa-user-profile-service to was_data
>>>
>>> Please let me know if there is a way
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/a7e5b925-2c85-4cb6-beaa-892ca1c1c80e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Hiera Merge
If you're just trying to transform the data in Puppet code and assuming (as
Henrik was) that you can't change how the data is stored, something like
this might work.
# Assuming $was_data is the hash of data from Hiera
$common_data = $was_data.filter |$pair| { $pair[0] != 'was_dmgr_data' }
$hash1 = {'esa-group-service'=> $common_data +
$was_data['was_dmgr_data']['esa-group-service'] }
$hash2 = {'esa-user-profile-service' => $common_data +
$was_data['was_dmgr_data']['esa-user-profile-service'] }
~Reid
On Wednesday, August 9, 2017 at 6:15:41 AM UTC-7, ggun wrote:
>
> Thanks
>
> On Tuesday, August 8, 2017 at 7:10:13 PM UTC-4, ggun wrote:
>>
>> Hi Experts,
>>
>> I have a requirement as below.
>> I need to create a Hash from below hiera data.
>>
>> was_data:
>> hs3sourcepath: 'glic.binaries/websphere'
>> hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
>> hibmagentpath:
>> /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
>> hbase_dir: '/opt/was/was855'
>> hinstance_name: WebSphere
>> was_dmgr_data:
>> esa-group-service:
>> hgroup: websph
>> hdmgr_profile: TST
>> hdmgr_cell: CELL
>> hcluster_name: CLUSTER
>> hpptdmgrsrvport: 8080
>> esa-user-profile-service:
>> hdmgr_profile: ABC
>> hdmgr_cell: PQS
>> hcluster_name: IOP
>> hpptdmgrsrvport:
>>
>>
>> I need a hash of above data as
>> Hash 1 :
>> esa-group-service:
>> hgroup: websph
>> hdmgr_profile: TST
>> hdmgr_cell: CELL
>> hcluster_name: CLUSTER
>> hpptdmgrsrvport: 8080
>> hs3sourcepath: 'glic.binaries/websphere'
>> hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
>> hibmagentpath:
>> /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
>> hbase_dir: '/opt/was/was855'
>> hinstance_name: WebSphere
>>
>> Hash 2
>> esa-user-profile-service:
>> hdmgr_profile: ABC
>> hdmgr_cell: PQS
>> hcluster_name: IOP
>> hpptdmgrsrvport:
>> hs3sourcepath: 'glic.binaries/websphere'
>> hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
>> hibmagentpath:
>> /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
>> hbase_dir: '/opt/was/was855'
>> hinstance_name: WebSphere
>>
>> So I trying to merge the has of esa-group-service to was_data and
>> esa-user-profile-service to was_data
>>
>> Please let me know if there is a way
>>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/be56ce3b-77e2-41a8-b9a5-75c42293a15c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Hiera Merge
Thanks On Tuesday, August 8, 2017 at 7:10:13 PM UTC-4, ggun wrote: > > Hi Experts, > > I have a requirement as below. > I need to create a Hash from below hiera data. > > was_data: > hs3sourcepath: 'glic.binaries/websphere' > hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml > hibmagentpath: > /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip > hbase_dir: '/opt/was/was855' > hinstance_name: WebSphere > was_dmgr_data: > esa-group-service: > hgroup: websph > hdmgr_profile: TST > hdmgr_cell: CELL > hcluster_name: CLUSTER > hpptdmgrsrvport: 8080 > esa-user-profile-service: > hdmgr_profile: ABC > hdmgr_cell: PQS > hcluster_name: IOP > hpptdmgrsrvport: > > > I need a hash of above data as > Hash 1 : > esa-group-service: > hgroup: websph > hdmgr_profile: TST > hdmgr_cell: CELL > hcluster_name: CLUSTER > hpptdmgrsrvport: 8080 > hs3sourcepath: 'glic.binaries/websphere' > hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml > hibmagentpath: > /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip > hbase_dir: '/opt/was/was855' > hinstance_name: WebSphere > > Hash 2 > esa-user-profile-service: > hdmgr_profile: ABC > hdmgr_cell: PQS > hcluster_name: IOP > hpptdmgrsrvport: > hs3sourcepath: 'glic.binaries/websphere' > hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml > hibmagentpath: > /opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip > hbase_dir: '/opt/was/was855' > hinstance_name: WebSphere > > So I trying to merge the has of esa-group-service to was_data and > esa-user-profile-service to was_data > > Please let me know if there is a way > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/db787a3a-de76-43cc-840a-ed8905510aa0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[Puppet-dev] Re: Hiera Merge
On 09/08/17 01:10, ggun wrote:
Hi Experts,
I have a requirement as below.
I need to create a Hash from below hiera data.
was_data:
hs3sourcepath: 'glic.binaries/websphere'
hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
hibmagentpath:
/opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
hbase_dir: '/opt/was/was855'
hinstance_name: WebSphere
was_dmgr_data:
esa-group-service:
hgroup: websph
hdmgr_profile: TST
hdmgr_cell: CELL
hcluster_name: CLUSTER
hpptdmgrsrvport: 8080
esa-user-profile-service:
hdmgr_profile: ABC
hdmgr_cell: PQS
hcluster_name: IOP
hpptdmgrsrvport:
I need a hash of above data as
Hash 1 :
esa-group-service:
hgroup: websph
hdmgr_profile: TST
hdmgr_cell: CELL
hcluster_name: CLUSTER
hpptdmgrsrvport: 8080
hs3sourcepath: 'glic.binaries/websphere'
hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
hibmagentpath:
/opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
hbase_dir: '/opt/was/was855'
hinstance_name: WebSphere
Hash 2
esa-user-profile-service:
hdmgr_profile: ABC
hdmgr_cell: PQS
hcluster_name: IOP
hpptdmgrsrvport:
hs3sourcepath: 'glic.binaries/websphere'
hdaresponse_file: /opt/software/WAS8.5.5.10_Install.xml
hibmagentpath:
/opt/software/agent.installer.linux.gtk.x86_64_1.8.2000.20150303_1526.zip
hbase_dir: '/opt/was/was855'
hinstance_name: WebSphere
So I trying to merge the has of esa-group-service to was_data and
esa-user-profile-service to was_data
Please let me know if there is a way
"There is always a way..." - question is if there is a good way ;-)
I take it that it is not feasible to rearrange how the data is organized
in hiera in the first place - that would otherwise be the best.
The issue here is that you cannot simply both merge and split up the
hash at the same time - you have one part that looks like a common part;
defaults for the other two parts if you like.
With hiera 5, if you change the data structure to consist of a defaults
part - under one key, say "esa_default", and then store the other two
under their respective keys 'esa_group_part', 'esa_user_part', then you
have several options:
* write a hiera 5 backend function that returns the merged result for
the two full keys for the group and user structures
* write a class where you use APL to get the three parts, and then merge
in that class, binding to a variable that you then use to get the merged
result.
I would do the hiera backend function.
If you also need the original data structure (as shown in your example)
then also merge a result that looks like that in your backend function.
The function would be simple - basically do something like:
lookup('esa_default') + lookup('esa_group_part')
or
lookup('esa_default') + lookup('esa_user_part')
to override defaults with more specific values (if that is what you want
to do - or some variation on that if it is not defaults in the "common"
part).
I choose the names "group_part" and "user_part" simply for illustration.
Hope this helps you.
You can read more about how to create a backend function here:
https://docs.puppet.com/puppet/5.0/hiera_custom_lookup_key.html
- henrik
--
Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/
--
You received this message because you are subscribed to the Google Groups "Puppet
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/omejnq%24j0t%241%40blaine.gmane.org.
For more options, visit https://groups.google.com/d/optout.
