Re: vmdebootstrap plans
On Sun, Nov 13, 2016 at 08:58:02PM +0100, Thomas Lange wrote: > I'm really wondering why you want to use YAML for this? Your examples > look like things you could easily implement in shell, if you provide > some functions, subroutines or external programs that implement your > steps. I don't think using the shell language to implement a DSL is acceptable, however easy it may seem. Shell is a really, really bad programming language and I don't want to encourage its use. In addition, I will _not_ use shell to implement anything that requires robustness or code clarity, for it is a lousy language for anything beyond very simple scripts. Life is too short to write, maintain, or debug, complicated software written in shell. I'm too old for this sh. > create_raw_file $size $image I realize that you only wrote the above as part of an example, but to me it's an example of why shell won't do. You didn't quote the variable expansions, which may result in quite nasty bugs. Reading the code I would need to evaluate if it's safe to not quote, or if it is in fact correct to not quote, or if there's a possible bug there. The rules for quoting in shell are too intricate, and it's too easy to accidentally do the wrong thing. > Much easier and more common for sysadmins IMO. Having written this kind of software before, I guarantee it's not easier in the long term. Or the medium term. It's only easier for getting to a quick prototype. Beyond that, one runs into language limitations. If editing a YAML file is too difficult for a sysadmin, then I don't really care. I don't think it will be. YAML files are easy and a very large number of sysadmins use them with, say, Ansible. -- I want to build worthwhile things that might last. --joeyh signature.asc Description: PGP signature
Re: vmdebootstrap plans
On 14 Nov 2016 8:39 am, "Marcin Kulisz"wrote: > > On 2016-11-13 20:58:02, Thomas Lange wrote: > > Hi Lars, > > > > I'm really wondering why you want to use YAML for this? > > I can talk just for my self but my pros would be: cleanliness, machine > parcebility, human redeability, neat python integration, simple templating > integration (jinja2) with programming languages. So I suppose Lars > would agree with those and possibly add something else. > > > Your examples > > look like things you could easily implement in shell, if you provide > > some functions, subroutines or external programs that implement your > > steps. Then your DSL would just be a shell script that the user has to > > write. > > > > > > create_raw_file $size $image > > create_partition_table msdos > > create_partition $bootsize ext2 /boot bootable > > create_partition rest ext4 / > > mount_all > > Above doesn't look so neat to me like well structured schema based yaml. > > > and so on. > > > > > The "step:" key specifies the step to execute. The other keys are > > > parameters for the step. The {{...}} parts are where Jinja2 is used > > > to expand variable values. > > These things are very similar to calling commands with parameters in a > > shell script without the need of YAML and Jinja2. > > Yes, but IMO yaml and jinja2 allow for creating very simple and nice > integration and extending capabilities with minimal effort. > > > Much easier and more common for sysadmins IMO. > > I can agree that this may be the case for some people but it's hard to deny > that there are restrictions on bash which can't be overcome due to bash nature > and which does not exist in python as this is fully blown programming language. > > And yes I know Thomas your approach to object orientated programming :-) > > What Lars is proposing I see as a very reasonable thing to do. Providing well design > and maintained tool with easy extensibility and a way to accommodate eventual > future requirements. > > IMO what Lars is trying to achieve is much easier to do in python then in bash > and with properly abstracted logic there should be no need to anybody using > this tool to even know python (what for me is a bit odd for sysadmins) to create > images with vmdb. > Knowledge of available options and basics of yaml would suffice and I think > this is even lower threshold for entry level then knowing bash. These days golang is the go, excuse the pun and the fact it compiles to a single executable means the language and extra packages such as jinja2 are not required. > -- > > |_|0|_| | > |_|_|0| "Panta rei" | > |0|0|0| kuLa | > > gpg --keyserver pgp.mit.edu --recv-keys 0x686930DD58C338B3 > 3DF1 A4DF C732 4688 38BC F121 6869 30DD 58C3 38B3
Re: vmdebootstrap plans
On 2016-11-13 20:58:02, Thomas Lange wrote: > Hi Lars, > > I'm really wondering why you want to use YAML for this? I can talk just for my self but my pros would be: cleanliness, machine parcebility, human redeability, neat python integration, simple templating integration (jinja2) with programming languages. So I suppose Lars would agree with those and possibly add something else. > Your examples > look like things you could easily implement in shell, if you provide > some functions, subroutines or external programs that implement your > steps. Then your DSL would just be a shell script that the user has to > write. > > > create_raw_file $size $image > create_partition_table msdos > create_partition $bootsize ext2 /boot bootable > create_partition rest ext4 / > mount_all Above doesn't look so neat to me like well structured schema based yaml. > and so on. > > > The "step:" key specifies the step to execute. The other keys are > > parameters for the step. The {{...}} parts are where Jinja2 is used > > to expand variable values. > These things are very similar to calling commands with parameters in a > shell script without the need of YAML and Jinja2. Yes, but IMO yaml and jinja2 allow for creating very simple and nice integration and extending capabilities with minimal effort. > Much easier and more common for sysadmins IMO. I can agree that this may be the case for some people but it's hard to deny that there are restrictions on bash which can't be overcome due to bash nature and which does not exist in python as this is fully blown programming language. And yes I know Thomas your approach to object orientated programming :-) What Lars is proposing I see as a very reasonable thing to do. Providing well design and maintained tool with easy extensibility and a way to accommodate eventual future requirements. IMO what Lars is trying to achieve is much easier to do in python then in bash and with properly abstracted logic there should be no need to anybody using this tool to even know python (what for me is a bit odd for sysadmins) to create images with vmdb. Knowledge of available options and basics of yaml would suffice and I think this is even lower threshold for entry level then knowing bash. -- |_|0|_| | |_|_|0| "Panta rei" | |0|0|0| kuLa | gpg --keyserver pgp.mit.edu --recv-keys 0x686930DD58C338B3 3DF1 A4DF C732 4688 38BC F121 6869 30DD 58C3 38B3 signature.asc Description: PGP signature
Re: my progress
> On Sun, 13 Nov 2016 06:42:37 -0800, Noah Meyerhans> said: > I based my work on jessie initially because we have JEB's high quality > jessie images to use as a baseline for comparison, and I found that > useful. +1 We agreed that we want to use fai for the official Debian images from stretch. But the users may want to use the new tool for their customized jessie images, or give it a try before they have to use it for stretch anyway. IMO it was a good idea to start with jessie and we should keep this config in the official git repository. -- regards Thomas
Re: vmdebootstrap plans
Hi Lars, I'm really wondering why you want to use YAML for this? Your examples look like things you could easily implement in shell, if you provide some functions, subroutines or external programs that implement your steps. Then your DSL would just be a shell script that the user has to write. create_raw_file $size $image create_partition_table msdos create_partition $bootsize ext2 /boot bootable create_partition rest ext4 / mount_all and so on. > The "step:" key specifies the step to execute. The other keys are > parameters for the step. The {{...}} parts are where Jinja2 is used > to expand variable values. These things are very similar to calling commands with parameters in a shell script without the need of YAML and Jinja2. Much easier and more common for sysadmins IMO. Just my two cents. Thomas > On Fri, 11 Nov 2016 14:09:39 +, Lars Wirzeniussaid: > * The YAML might look somethin like this: > - step: create_raw_file > size: {{ size }} > filename: {{ image }} > - step: create_partition_table > partition_table_type: msdos > - step: create_partition > size: {{ bootsize }} > partition_type: ext2 > bootable: yes > mount_point: /boot > - step: create_partition > size: rest > partition_type: ext4 > mount_point: / > - step: mount_all > - step: debootstrap > - step: install_grub > - step: run_customization_script > script: {{ customize }} > - step: cleanup > - step: convert_image_to_output_format > output_format: qcow2
Re: my progress
On Sun, Nov 13, 2016 at 06:17:33AM -0500, Sam Hartman wrote: > Noah> I've modified the class/DEBIAN.var file such that the default > Noah> behavior is to generate images for jessie. We can add a > Noah> STRETCH class in order to generate images for testing. I'd > Noah> rather use "stable", but it's preferable to use the release > Noah> codename in many places and there's no easy way to resolve > Noah> stable to the current codename AFAIK. Maybe there should be? > > I thought we agreed the fai efforts would be for stretch and we would > not be converting our existing jessie work. > I don't mind having jessie support if it's easy, but for example if we > run into situations where it adds extra code to work around bugs fixed > in stretch but present in jessie, I'd like to push back. Yes, I think you're right. I based my work on jessie initially because we have JEB's high quality jessie images to use as a baseline for comparison, and I found that useful. In practice, activating a JESSIE or STRETCH class still needs to happen via the fai-diskimage commandline anyway, and these classes override the release variable, so defining a default value for that variable in class/DEBIAN.var was not actually useful. My inclination is to require the user to activate a specific release class, and for FAI to abort if one isn't provided. noah signature.asc Description: PGP signature
Re: my progress
> "Noah" == Noah Meyerhanswrites: Noah> I've modified the class/DEBIAN.var file such that the default Noah> behavior is to generate images for jessie. We can add a Noah> STRETCH class in order to generate images for testing. I'd Noah> rather use "stable", but it's preferable to use the release Noah> codename in many places and there's no easy way to resolve Noah> stable to the current codename AFAIK. Maybe there should be? I thought we agreed the fai efforts would be for stretch and we would not be converting our existing jessie work. I don't mind having jessie support if it's easy, but for example if we run into situations where it adds extra code to work around bugs fixed in stretch but present in jessie, I'd like to push back. I suspect that long-term find it easier to maintain something where we have the work-arounds for things we've found in stable releases in a class, and the main body of the code is targeted at sid and testing. Thoughts?
Re: ENA kernel driver for Stretch release on AWS cloud
On 2016-11-13 03:37:17, Ben Hutchings wrote: > On Tue, 2016-11-08 at 14:24 +, Marcin Kulisz wrote: > > Hi all, > > > > We need ena kernel driver for AWS cloud images as I'm not sure what kernel > > version we're going to ship with Stretch and this driver is available with > > 4.9rc, I'd like to ask (and fill the bug report if need): > > what kernel version are we going to have in Stretch? > > It will be the next kernel version that has upstream long-term support > (or 4.10 at the latest). This is planned to be 4.9, although I think > that hasn't been finalised yet. Thank you for the info > > If it's going to be 4.8.X are you ok with me creating bug requesting > > including > > ena driver with it? > [...] > > Please do that if we announce that we'll be using 4.8. > > For now, I've enabled this driver for the next upload of 4.9 to > experimental. Thx a lot Ben it's going to be a great help. -- |_|0|_| | |_|_|0| "Panta rei" | |0|0|0| kuLa | gpg --keyserver pgp.mit.edu --recv-keys 0x686930DD58C338B3 3DF1 A4DF C732 4688 38BC F121 6869 30DD 58C3 38B3 signature.asc Description: PGP signature