[
https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14205401#comment-14205401
]
Nicholas Chammas commented on SPARK-3821:
-----------------------------------------
Thanks for taking a look [~danospv]. Looking forward to your feedback.
Keeping the fully-baked AMIs could totally work. The current scripts allow the
image creation to be fully automated. We may need some more tooling around
image management (e.g. like [{{delete-all-registered-spark-amis.py}} |
https://github.com/nchammas/spark-ec2/blob/packer/packer/delete-all-registered-spark-amis.py]),
and we will need to maintain the image library we build up.
So it's probably just a question of whether we are ready to accept the
maintenance / tooling burden at this time, though it's totally feasible.
> Develop an automated way of creating Spark images (AMI, Docker, and others)
> ---------------------------------------------------------------------------
>
> Key: SPARK-3821
> URL: https://issues.apache.org/jira/browse/SPARK-3821
> Project: Spark
> Issue Type: Improvement
> Components: Build, EC2
> Reporter: Nicholas Chammas
> Assignee: Nicholas Chammas
> Attachments: packer-proposal.html
>
>
> Right now the creation of Spark AMIs or Docker containers is done manually.
> With tools like [Packer|http://www.packer.io/], we should be able to automate
> this work, and do so in such a way that multiple types of machine images can
> be created from a single template.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
