If you are on that scale python or ruby would do json manipulation in all imaginary ways.
But frankly nowadays everyone uses Docker and the only need to build one base image to run it on (or none). ;-) On Thu, Sep 12, 2019, 22:13 'Nicholas Bayle' via Packer < [email protected]> wrote: > For posterity, here is how you do it: > > jq -s '[.[] | to_entries] | flatten | reduce .[] as $dot ({}; .[$dot.key] > += $dot.value)' file1.json file2.json fileN.json | packer build - > > On Tue, Sep 10, 2019 at 10:37 AM Nicholas Bayle < > [email protected]> wrote: > >> So everybody just does copy/paste and manages multiple files by hand? A >> nightmare indeed. >> >> On Thu, Sep 5, 2019 at 4:13 PM Nicholas Bayle < >> [email protected]> wrote: >> >>> So I'm trying to merge multiple packer json files together using the >>> following json files: >>> - variables used by all platforms >>> - platform-specific requirements (usually a builder & sometimes a >>> provisioner and/or post-processor) >>> - the provisioners used for all platforms >>> - the post-processors used by all platforms >>> >>> So, generating the json and running packer to build a VM would look >>> roughly like: >>> PACKER_LOG=1 jq -s 'reduce .[] as $item ({}; . * $item)' >>> packer-variables.json packer-vm.json packer-provisioners.json >>> packer-post-processors.json | packer build -force - >>> >>> The json output at first glance looks ok, but on closer inspection I >>> have an issue. If packer-vm.json contains this snippet (among other things): >>> "provisioners": [ >>> { >>> "type": "shell", >>> "inline": ["echo 'vagrant' | sudo -S -- bash -c 'dd if=/dev/zero >>> of=/EMPTY bs=1M; rm -f /EMPTY; sync'"] >>> } >>> ], >>> >>> *and* packer-provsioners.json contains: >>> { >>> "provisioners": [ >>> { >>> "type": "shell", >>> "inline": ["echo 'vagrant' | sudo -S bash -c 'echo { >>> \\\"platform_type\\\": \\\"{{user `platform`}}\\\" } > /.platform_info' && >>> echo 'vagrant' | sudo -S chmod 666 /.platform_info"] >>> }, >>> { >>> "type": "chef-solo", >>> "cookbook_paths": ["cookbooks"], >>> "run_list": ["things"], >>> "version": "14.13", >>> "chef_environment": "{{user `chef_env_type`}}", >>> "environments_path": "./environments" >>> }, >>> { >>> "type": "shell", >>> "inline": ["echo 'vagrant' | sudo -S apt-get --yes --purge >>> autoremove chef"] >>> }, >>> { >>> "type": "shell", >>> "inline": ["dpkg-query -f '${Package} ${Version}\n' -W > >>> /tmp/manifest; sleep 2"] >>> }, >>> { >>> "type": "file", >>> "source": "/tmp/manifest", >>> "destination": "manifest", >>> "direction": "download" >>> }, >>> { >>> "type": "shell", >>> "inline": ["rm /tmp/manifest"] >>> } >>> ] >>> } >>> >>> Then jq will just wipe out the provisioners from packer-vm.json (i.e. no >>> dd command in this example). Does anyone know the magic jq sauce to >>> preserve that during a merge? >>> >>> Nick >>> >>> >>> >>> >>> On Tue, Jun 11, 2019 at 10:15 AM Rickard von Essen < >>> [email protected]> wrote: >>> >>>> or you can pipe it like this: >>>> >>>> cat template.json | packer build - >>>> >>>> >>>> On Tue, Jun 11, 2019 at 3:13 PM Nasir Amin <[email protected]> >>>> wrote: >>>> >>>>> Hi Richard, >>>>> >>>>> Thanks. So that means I'll have to generate files at runtime and then >>>>> delete them once the packer build has completed? >>>>> >>>>> Regards, >>>>> >>>>> On Tuesday, June 11, 2019 at 1:30:51 PM UTC+1, Rickard von Essen wrote: >>>>>> >>>>>> You can use jq to join several json files and thus keep snippets that >>>>>> you join together. See >>>>>> https://stackoverflow.com/questions/19529688/how-to-merge-2-json-file-using-jq >>>>>> >>>>>> On Tue, Jun 11, 2019 at 12:42 PM Nasir Amin <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am using Packer version 1.4.1 >>>>>>> What is the best practice to manage generic templates. >>>>>>> We have a few packer template files for different windows versions. >>>>>>> The template's builders section is common and we use var files to pass >>>>>>> in >>>>>>> any user variables. >>>>>>> >>>>>>> The reason we are having to keep multiple json template files is due >>>>>>> to the Provisioner sections being different for different types of >>>>>>> templates. >>>>>>> >>>>>>> Question: >>>>>>> Is there a way we can use the same template file with just the >>>>>>> builders section. >>>>>>> For provisioners, we can either put them in a separate file and then >>>>>>> somehow inject it into the main tempalte OR >>>>>>> if we can pass in the Provisioners in the form of a json array >>>>>>> through a user variable from a variables file? >>>>>>> >>>>>>> Some of my provisioners are powershell scripts, ansible and windows >>>>>>> restart. Is it possible to pass in provisioners from a variable file >>>>>>> like >>>>>>> this? >>>>>>> >>>>>>> >>>>>>> "host_provisioners": [ >>>>>>> { >>>>>>> "type": "powershell", >>>>>>> "inline": [ >>>>>>> "cmd.exe /c powershell.exe Set-ExecutionPolicy >>>>>>> Unrestricted -force", >>>>>>> "a:\\scripts\\Configure-MinorSettings.ps1", >>>>>>> "a:\\scripts\\win2019-setup-ssh.ps1", >>>>>>> "a:\\scripts\\Setup-Docker.ps1" >>>>>>> ] >>>>>>> }, >>>>>>> { >>>>>>> "type": "windows-restart", >>>>>>> "restart_check_command": "powershell -command \"& {Write-Output >>>>>>> 'Guest Restarted Successfully.'}\"" >>>>>>> } >>>>>>> ] >>>>>>> } >>>>>>> >>>>>>> >>>>>>> Then in the main template file where my builders are, I do this: >>>>>>> >>>>>>> "provisioners": "{{user host_provisioners }}" >>>>>>> >>>>>>> It seems like you have to put provisioners and builders in the same >>>>>>> file. It would be good if we can do on one of the following and if it is >>>>>>> already possible: >>>>>>> >>>>>>> >>>>>>> - Isolate builders and provisioners into separate files and then >>>>>>> combine them in the packer build command with a flag like >>>>>>> --provisioner-file=<path to file> >>>>>>> - OR Allow passing the provisioners from a variables file. >>>>>>> >>>>>>> The first approach will probably be more flexible. >>>>>>> >>>>>>> What is the best practice on structuring and managing multiple >>>>>>> templates or making your templates generic? As things stand, it seems >>>>>>> impossible to apply DRY principles to Packer. >>>>>>> >>>>>>> I'll really appreciate any guidance on this please as our packer >>>>>>> scripts and templates are becoming a bit hard to scale and maintain. >>>>>>> I see packer will eventually support HCL2 which should make some of >>>>>>> the things easier e.g. conditional provisioners etc >>>>>>> https://github.com/hashicorp/packer/issues/1768 >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Nas >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> This mailing list is governed under the HashiCorp Community >>>>>>> Guidelines - https://www.hashicorp.com/community-guidelines.html. >>>>>>> Behavior in violation of those guidelines may result in your removal >>>>>>> from >>>>>>> this mailing list. >>>>>>> >>>>>>> GitHub Issues: https://github.com/mitchellh/packer/issues >>>>>>> IRC: #packer-tool on Freenode >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Packer" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/packer-tool/ea90623a-2da1-453e-969c-0e7c39a04969%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/packer-tool/ea90623a-2da1-453e-969c-0e7c39a04969%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>> This mailing list is governed under the HashiCorp Community Guidelines >>>>> - https://www.hashicorp.com/community-guidelines.html. Behavior in >>>>> violation of those guidelines may result in your removal from this mailing >>>>> list. >>>>> >>>>> GitHub Issues: https://github.com/mitchellh/packer/issues >>>>> IRC: #packer-tool on Freenode >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Packer" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/packer-tool/317528bb-11d0-42f8-ad75-e6fcfa9411e0%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/packer-tool/317528bb-11d0-42f8-ad75-e6fcfa9411e0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> This mailing list is governed under the HashiCorp Community Guidelines >>>> - https://www.hashicorp.com/community-guidelines.html. Behavior in >>>> violation of those guidelines may result in your removal from this mailing >>>> list. >>>> >>>> GitHub Issues: https://github.com/mitchellh/packer/issues >>>> IRC: #packer-tool on Freenode >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Packer" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/packer-tool/CALz9Rt8sZk%2BEQGpFqmaWk_4Tc97n%2BBwwY_EvZ9Yu4LdXBaGvnw%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/packer-tool/CALz9Rt8sZk%2BEQGpFqmaWk_4Tc97n%2BBwwY_EvZ9Yu4LdXBaGvnw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- > This mailing list is governed under the HashiCorp Community Guidelines - > https://www.hashicorp.com/community-guidelines.html. Behavior in > violation of those guidelines may result in your removal from this mailing > list. > > GitHub Issues: https://github.com/mitchellh/packer/issues > IRC: #packer-tool on Freenode > --- > You received this message because you are subscribed to the Google Groups > "Packer" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/packer-tool/CAGzi4%3D66Tn-KO6P3Tdj6qnV0JNj6Q-e4VDwcH583q8MKQL%3Dksg%40mail.gmail.com > <https://groups.google.com/d/msgid/packer-tool/CAGzi4%3D66Tn-KO6P3Tdj6qnV0JNj6Q-e4VDwcH583q8MKQL%3Dksg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list. GitHub Issues: https://github.com/mitchellh/packer/issues IRC: #packer-tool on Freenode --- You received this message because you are subscribed to the Google Groups "Packer" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CALz9Rt-yu99VkOhESk3FZgb8wHPxDXRYuCqDaHdiTSbuG0U__g%40mail.gmail.com.
