So I'm trying to merge multiple packer json files together using the
following json files:
- variables used by all platforms
- platform-specific requirements (usually a builder & sometimes a
provisioner and/or post-processor)
- the provisioners used for all platforms
- the post-processors used by all platforms
So, generating the json and running packer to build a VM would look roughly
like:
PACKER_LOG=1 jq -s 'reduce .[] as $item ({}; . * $item)'
packer-variables.json packer-vm.json packer-provisioners.json
packer-post-processors.json | packer build -force -
The json output at first glance looks ok, but on closer inspection I have
an issue. If packer-vm.json contains this snippet (among other things):
"provisioners": [
{
"type": "shell",
"inline": ["echo 'vagrant' | sudo -S -- bash -c 'dd if=/dev/zero
of=/EMPTY bs=1M; rm -f /EMPTY; sync'"]
}
],
*and* packer-provsioners.json contains:
{
"provisioners": [
{
"type": "shell",
"inline": ["echo 'vagrant' | sudo -S bash -c 'echo {
\\\"platform_type\\\": \\\"{{user `platform`}}\\\" } > /.platform_info' &&
echo 'vagrant' | sudo -S chmod 666 /.platform_info"]
},
{
"type": "chef-solo",
"cookbook_paths": ["cookbooks"],
"run_list": ["things"],
"version": "14.13",
"chef_environment": "{{user `chef_env_type`}}",
"environments_path": "./environments"
},
{
"type": "shell",
"inline": ["echo 'vagrant' | sudo -S apt-get --yes --purge autoremove
chef"]
},
{
"type": "shell",
"inline": ["dpkg-query -f '${Package} ${Version}\n' -W >
/tmp/manifest; sleep 2"]
},
{
"type": "file",
"source": "/tmp/manifest",
"destination": "manifest",
"direction": "download"
},
{
"type": "shell",
"inline": ["rm /tmp/manifest"]
}
]
}
Then jq will just wipe out the provisioners from packer-vm.json (i.e. no dd
command in this example). Does anyone know the magic jq sauce to preserve
that during a merge?
Nick
On Tue, Jun 11, 2019 at 10:15 AM Rickard von Essen <
[email protected]> wrote:
> or you can pipe it like this:
>
> cat template.json | packer build -
>
>
> On Tue, Jun 11, 2019 at 3:13 PM Nasir Amin <[email protected]> wrote:
>
>> Hi Richard,
>>
>> Thanks. So that means I'll have to generate files at runtime and then
>> delete them once the packer build has completed?
>>
>> Regards,
>>
>> On Tuesday, June 11, 2019 at 1:30:51 PM UTC+1, Rickard von Essen wrote:
>>>
>>> You can use jq to join several json files and thus keep snippets that
>>> you join together. See
>>> https://stackoverflow.com/questions/19529688/how-to-merge-2-json-file-using-jq
>>>
>>> On Tue, Jun 11, 2019 at 12:42 PM Nasir Amin <[email protected]> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am using Packer version 1.4.1
>>>> What is the best practice to manage generic templates.
>>>> We have a few packer template files for different windows versions. The
>>>> template's builders section is common and we use var files to pass in any
>>>> user variables.
>>>>
>>>> The reason we are having to keep multiple json template files is due to
>>>> the Provisioner sections being different for different types of templates.
>>>>
>>>> Question:
>>>> Is there a way we can use the same template file with just the builders
>>>> section.
>>>> For provisioners, we can either put them in a separate file and then
>>>> somehow inject it into the main tempalte OR
>>>> if we can pass in the Provisioners in the form of a json array through
>>>> a user variable from a variables file?
>>>>
>>>> Some of my provisioners are powershell scripts, ansible and windows
>>>> restart. Is it possible to pass in provisioners from a variable file like
>>>> this?
>>>>
>>>>
>>>> "host_provisioners": [
>>>> {
>>>> "type": "powershell",
>>>> "inline": [
>>>> "cmd.exe /c powershell.exe Set-ExecutionPolicy
>>>> Unrestricted -force",
>>>> "a:\\scripts\\Configure-MinorSettings.ps1",
>>>> "a:\\scripts\\win2019-setup-ssh.ps1",
>>>> "a:\\scripts\\Setup-Docker.ps1"
>>>> ]
>>>> },
>>>> {
>>>> "type": "windows-restart",
>>>> "restart_check_command": "powershell -command \"& {Write-Output
>>>> 'Guest Restarted Successfully.'}\""
>>>> }
>>>> ]
>>>> }
>>>>
>>>>
>>>> Then in the main template file where my builders are, I do this:
>>>>
>>>> "provisioners": "{{user host_provisioners }}"
>>>>
>>>> It seems like you have to put provisioners and builders in the same
>>>> file. It would be good if we can do on one of the following and if it is
>>>> already possible:
>>>>
>>>>
>>>> - Isolate builders and provisioners into separate files and then
>>>> combine them in the packer build command with a flag like
>>>> --provisioner-file=<path to file>
>>>> - OR Allow passing the provisioners from a variables file.
>>>>
>>>> The first approach will probably be more flexible.
>>>>
>>>> What is the best practice on structuring and managing multiple
>>>> templates or making your templates generic? As things stand, it seems
>>>> impossible to apply DRY principles to Packer.
>>>>
>>>> I'll really appreciate any guidance on this please as our packer
>>>> scripts and templates are becoming a bit hard to scale and maintain.
>>>> I see packer will eventually support HCL2 which should make some of the
>>>> things easier e.g. conditional provisioners etc
>>>> https://github.com/hashicorp/packer/issues/1768
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Nas
>>>>
>>>>
>>>> --
>>>> This mailing list is governed under the HashiCorp Community Guidelines
>>>> - https://www.hashicorp.com/community-guidelines.html. Behavior in
>>>> violation of those guidelines may result in your removal from this mailing
>>>> list.
>>>>
>>>> GitHub Issues: https://github.com/mitchellh/packer/issues
>>>> IRC: #packer-tool on Freenode
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Packer" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/packer-tool/ea90623a-2da1-453e-969c-0e7c39a04969%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/packer-tool/ea90623a-2da1-453e-969c-0e7c39a04969%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>> This mailing list is governed under the HashiCorp Community Guidelines -
>> https://www.hashicorp.com/community-guidelines.html. Behavior in
>> violation of those guidelines may result in your removal from this mailing
>> list.
>>
>> GitHub Issues: https://github.com/mitchellh/packer/issues
>> IRC: #packer-tool on Freenode
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Packer" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/packer-tool/317528bb-11d0-42f8-ad75-e6fcfa9411e0%40googlegroups.com
>> <https://groups.google.com/d/msgid/packer-tool/317528bb-11d0-42f8-ad75-e6fcfa9411e0%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in
> violation of those guidelines may result in your removal from this mailing
> list.
>
> GitHub Issues: https://github.com/mitchellh/packer/issues
> IRC: #packer-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Packer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/packer-tool/CALz9Rt8sZk%2BEQGpFqmaWk_4Tc97n%2BBwwY_EvZ9Yu4LdXBaGvnw%40mail.gmail.com
> <https://groups.google.com/d/msgid/packer-tool/CALz9Rt8sZk%2BEQGpFqmaWk_4Tc97n%2BBwwY_EvZ9Yu4LdXBaGvnw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
This mailing list is governed under the HashiCorp Community Guidelines -
https://www.hashicorp.com/community-guidelines.html. Behavior in violation of
those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups
"Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/packer-tool/CAGzi4%3D7AsiEAK040RRm5XsQoYynP7_KKH30-%3DZnhymzBgGSrMw%40mail.gmail.com.
