For posterity, here is how you do it:
jq -s '[.[] | to_entries] | flatten | reduce .[] as $dot ({}; .[$dot.key]
+= $dot.value)' file1.json file2.json fileN.json | packer build -
On Tue, Sep 10, 2019 at 10:37 AM Nicholas Bayle <
[email protected]> wrote:
> So everybody just does copy/paste and manages multiple files by hand? A
> nightmare indeed.
>
> On Thu, Sep 5, 2019 at 4:13 PM Nicholas Bayle <
> [email protected]> wrote:
>
>> So I'm trying to merge multiple packer json files together using the
>> following json files:
>> - variables used by all platforms
>> - platform-specific requirements (usually a builder & sometimes a
>> provisioner and/or post-processor)
>> - the provisioners used for all platforms
>> - the post-processors used by all platforms
>>
>> So, generating the json and running packer to build a VM would look
>> roughly like:
>> PACKER_LOG=1 jq -s 'reduce .[] as $item ({}; . * $item)'
>> packer-variables.json packer-vm.json packer-provisioners.json
>> packer-post-processors.json | packer build -force -
>>
>> The json output at first glance looks ok, but on closer inspection I have
>> an issue. If packer-vm.json contains this snippet (among other things):
>> "provisioners": [
>> {
>> "type": "shell",
>> "inline": ["echo 'vagrant' | sudo -S -- bash -c 'dd if=/dev/zero
>> of=/EMPTY bs=1M; rm -f /EMPTY; sync'"]
>> }
>> ],
>>
>> *and* packer-provsioners.json contains:
>> {
>> "provisioners": [
>> {
>> "type": "shell",
>> "inline": ["echo 'vagrant' | sudo -S bash -c 'echo {
>> \\\"platform_type\\\": \\\"{{user `platform`}}\\\" } > /.platform_info' &&
>> echo 'vagrant' | sudo -S chmod 666 /.platform_info"]
>> },
>> {
>> "type": "chef-solo",
>> "cookbook_paths": ["cookbooks"],
>> "run_list": ["things"],
>> "version": "14.13",
>> "chef_environment": "{{user `chef_env_type`}}",
>> "environments_path": "./environments"
>> },
>> {
>> "type": "shell",
>> "inline": ["echo 'vagrant' | sudo -S apt-get --yes --purge
>> autoremove chef"]
>> },
>> {
>> "type": "shell",
>> "inline": ["dpkg-query -f '${Package} ${Version}\n' -W >
>> /tmp/manifest; sleep 2"]
>> },
>> {
>> "type": "file",
>> "source": "/tmp/manifest",
>> "destination": "manifest",
>> "direction": "download"
>> },
>> {
>> "type": "shell",
>> "inline": ["rm /tmp/manifest"]
>> }
>> ]
>> }
>>
>> Then jq will just wipe out the provisioners from packer-vm.json (i.e. no
>> dd command in this example). Does anyone know the magic jq sauce to
>> preserve that during a merge?
>>
>> Nick
>>
>>
>>
>>
>> On Tue, Jun 11, 2019 at 10:15 AM Rickard von Essen <
>> [email protected]> wrote:
>>
>>> or you can pipe it like this:
>>>
>>> cat template.json | packer build -
>>>
>>>
>>> On Tue, Jun 11, 2019 at 3:13 PM Nasir Amin <[email protected]> wrote:
>>>
>>>> Hi Richard,
>>>>
>>>> Thanks. So that means I'll have to generate files at runtime and then
>>>> delete them once the packer build has completed?
>>>>
>>>> Regards,
>>>>
>>>> On Tuesday, June 11, 2019 at 1:30:51 PM UTC+1, Rickard von Essen wrote:
>>>>>
>>>>> You can use jq to join several json files and thus keep snippets that
>>>>> you join together. See
>>>>> https://stackoverflow.com/questions/19529688/how-to-merge-2-json-file-using-jq
>>>>>
>>>>> On Tue, Jun 11, 2019 at 12:42 PM Nasir Amin <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am using Packer version 1.4.1
>>>>>> What is the best practice to manage generic templates.
>>>>>> We have a few packer template files for different windows versions.
>>>>>> The template's builders section is common and we use var files to pass in
>>>>>> any user variables.
>>>>>>
>>>>>> The reason we are having to keep multiple json template files is due
>>>>>> to the Provisioner sections being different for different types of
>>>>>> templates.
>>>>>>
>>>>>> Question:
>>>>>> Is there a way we can use the same template file with just the
>>>>>> builders section.
>>>>>> For provisioners, we can either put them in a separate file and then
>>>>>> somehow inject it into the main tempalte OR
>>>>>> if we can pass in the Provisioners in the form of a json array
>>>>>> through a user variable from a variables file?
>>>>>>
>>>>>> Some of my provisioners are powershell scripts, ansible and windows
>>>>>> restart. Is it possible to pass in provisioners from a variable file like
>>>>>> this?
>>>>>>
>>>>>>
>>>>>> "host_provisioners": [
>>>>>> {
>>>>>> "type": "powershell",
>>>>>> "inline": [
>>>>>> "cmd.exe /c powershell.exe Set-ExecutionPolicy
>>>>>> Unrestricted -force",
>>>>>> "a:\\scripts\\Configure-MinorSettings.ps1",
>>>>>> "a:\\scripts\\win2019-setup-ssh.ps1",
>>>>>> "a:\\scripts\\Setup-Docker.ps1"
>>>>>> ]
>>>>>> },
>>>>>> {
>>>>>> "type": "windows-restart",
>>>>>> "restart_check_command": "powershell -command \"& {Write-Output
>>>>>> 'Guest Restarted Successfully.'}\""
>>>>>> }
>>>>>> ]
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Then in the main template file where my builders are, I do this:
>>>>>>
>>>>>> "provisioners": "{{user host_provisioners }}"
>>>>>>
>>>>>> It seems like you have to put provisioners and builders in the same
>>>>>> file. It would be good if we can do on one of the following and if it is
>>>>>> already possible:
>>>>>>
>>>>>>
>>>>>> - Isolate builders and provisioners into separate files and then
>>>>>> combine them in the packer build command with a flag like
>>>>>> --provisioner-file=<path to file>
>>>>>> - OR Allow passing the provisioners from a variables file.
>>>>>>
>>>>>> The first approach will probably be more flexible.
>>>>>>
>>>>>> What is the best practice on structuring and managing multiple
>>>>>> templates or making your templates generic? As things stand, it seems
>>>>>> impossible to apply DRY principles to Packer.
>>>>>>
>>>>>> I'll really appreciate any guidance on this please as our packer
>>>>>> scripts and templates are becoming a bit hard to scale and maintain.
>>>>>> I see packer will eventually support HCL2 which should make some of
>>>>>> the things easier e.g. conditional provisioners etc
>>>>>> https://github.com/hashicorp/packer/issues/1768
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Nas
>>>>>>
>>>>>>
>>>>>> --
>>>>>> This mailing list is governed under the HashiCorp Community
>>>>>> Guidelines - https://www.hashicorp.com/community-guidelines.html.
>>>>>> Behavior in violation of those guidelines may result in your removal from
>>>>>> this mailing list.
>>>>>>
>>>>>> GitHub Issues: https://github.com/mitchellh/packer/issues
>>>>>> IRC: #packer-tool on Freenode
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Packer" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/packer-tool/ea90623a-2da1-453e-969c-0e7c39a04969%40googlegroups.com
>>>>>> <https://groups.google.com/d/msgid/packer-tool/ea90623a-2da1-453e-969c-0e7c39a04969%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>> --
>>>> This mailing list is governed under the HashiCorp Community Guidelines
>>>> - https://www.hashicorp.com/community-guidelines.html. Behavior in
>>>> violation of those guidelines may result in your removal from this mailing
>>>> list.
>>>>
>>>> GitHub Issues: https://github.com/mitchellh/packer/issues
>>>> IRC: #packer-tool on Freenode
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Packer" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/packer-tool/317528bb-11d0-42f8-ad75-e6fcfa9411e0%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/packer-tool/317528bb-11d0-42f8-ad75-e6fcfa9411e0%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>>> This mailing list is governed under the HashiCorp Community Guidelines -
>>> https://www.hashicorp.com/community-guidelines.html. Behavior in
>>> violation of those guidelines may result in your removal from this mailing
>>> list.
>>>
>>> GitHub Issues: https://github.com/mitchellh/packer/issues
>>> IRC: #packer-tool on Freenode
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Packer" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/packer-tool/CALz9Rt8sZk%2BEQGpFqmaWk_4Tc97n%2BBwwY_EvZ9Yu4LdXBaGvnw%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/packer-tool/CALz9Rt8sZk%2BEQGpFqmaWk_4Tc97n%2BBwwY_EvZ9Yu4LdXBaGvnw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
--
This mailing list is governed under the HashiCorp Community Guidelines -
https://www.hashicorp.com/community-guidelines.html. Behavior in violation of
those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups
"Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/packer-tool/CAGzi4%3D66Tn-KO6P3Tdj6qnV0JNj6Q-e4VDwcH583q8MKQL%3Dksg%40mail.gmail.com.
