Re: Devops question: unattended installs of FreeBSD?
Craig Rodrigues rodr...@freebsd.org wrote: I had a devops person who is familiar with setting up hundreds of Linux nodes in cloud environment ask me what is the best way to do unattended installs in a cloud environment. Linux has kickstart installs, which are quite useful and popular. What is the equivalent in FreeBSD? vultr.com has an option to automatically install a freebsd instance (it's an automated install, not simply a binary image etc.) You can even vnc to the console and watch it progress. Though I'm not sure if the free trial requires credit card authoration - they are trying to make sure they aren't exploited by spammers etc. cheers, Jamie ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Connected sanitizer libraries to the build (for x86)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 01/13/15 12:28, Dimitry Andric wrote: Hi, In r277146, I have connected the sanitizer libraries from compiler-rt to the build. Currently, this works for i386 and amd64, and contains Address Sanitizer (ASan) and Undefined Behavior Sanitizer. AddressSanitizer is a fast memory error detector [1]. It consists of a compiler instrumentation module and a run-time library. The tool can detect the following types of bugs: * Out-of-bounds accesses to heap, stack and globals * Use-after-free * Use-after-return (to some extent) * Double-free, invalid free * Memory leaks (experimental) The typical slowdown introduced by AddressSanitizer is 2x. Enable it by compiling and linking with clang, and using the -fsanitize=address flag. Undefined Behavior Sanitizer is a fast and compatible undefined behavior checker, which enables a number of checks that have small runtime cost and no impact on address space layout or ABI. Enable it by using the -fsanitize=undefined flag. [2] Please note that the sanitizers still have some rough edges on FreeBSD, particularly on i386. These will hopefully be smoothed out in the coming time. Reports of problems (and fixes :) are very welcome, but please log them in Bugzilla, so they can be tracked. -Dimitry [1] http://llvm.org/releases/3.5.0/tools/clang/docs/AddressSanitizer.html [2] http://llvm.org/releases/3.5.0/tools/clang/docs/UsersManual.html#opt-fsanitize-undefined Do you want a test run for arm? sean -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQF8BAEBCgBmBQJUtaFkXxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5k5nsH/iHhTW359K0f2BtUDBwH/+ga 9w7MbymRJZvKTh60LABeuN//DJ9BBRHzGHtRd5nYvvLSBN5HfVy2LbmNvz9H1p0B /Gw6N9XL3pVMpLxU4JP6IMJ6c1YIlrapDxfUPOVpEPmdOeZ2xPsgRDB20tDNuKxj AQftpNqf1KJL0FhzfKv0TupxPpCKuffTfO+kYa5tQQU/bDXkgxB1BsuxUD/4HiZU nRAsbhlZV1roEo3l36a2mlRtc6sEPpZTKn4Phv3oNT7cfCd5hnuhyCfcZOWk7yXo HyboVn10ABX8GismKQ0erkxNhcHD4VepY2CCc/0z+AhUV0DztpfLvzDoEE0lDaY= =DqeZ -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Devops question: freebsd-update needs a real tty to run, problem for automation
Hi, Ahmed Kamal, a devops expert, is helping me to script the steps to upgrade a cluster of FreeBSD machines. For certain machines, we want to track the official FreeBSD releases and use freebsd-update to install official updates. We found that when the invocation of freebsd-update was scripted and not run via a real tty, we can into this error: freebsd-update fetch should not be run non-interactively. There are various workarounds mentioned on various web pages. However, should we modify freebsd-update so that it can work better when not run via a real tty? This would make it more devops/automation friendly. The closest thing I have found is freebsd-update cron, which can fetch the updates and run without a real tty. The only problem with freebsd-update cron is that it sleeps a random amount of time between 1 and 3600 seconds before fetching the updates. This is OK when run in a cron job, but not OK when run as part of a devops automation framework. Anybody have ideas as to the best way to proceed in fixing this in freebsd-update? -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Devops question: unattended installs of FreeBSD?
On Mon, Jan 12, 2015 at 12:12 PM, John Nielsen li...@jnielsen.net wrote: I'd be happy to provide more specific suggestions if needed. It really depends on how fully automated you want things to be and how much customization you want to include, as well as what you have available in the install environment. If you're installing on live VMs then you first have to get them booted. A custom ISO or MFS image is probably the simplest for that, though PXE is also an option. (Actually, serving an mfsBSD image via PXE is pretty straightforward.) Thanks! You provided some excellent concrete examples for how to do unattended FreeBSD installs. In the past 6 months, I have had two different people ask me how to: - create a PXE boot server - take the ISO image for FreeBSD 9.2, FreeBSD 10.1, etc. - create a kickstart environment where it is possible to PXE boot a cluster of machines, and have an unattended kickstart install take place of the various FreeBSD versions I have coded this kind of stuff up myself in the past and written my own scripts. However, it would be really nice if we had more straightforward documentation and example scripts for doing this. That way, the average devops engineer experienced with Linux and kickstart can set this up with no problem, instead of having to struggle and figure things out. It looks like all this stuff is possible under FreeBSD. The main problem I see is that the access to the documentation for doing this is not straightforward at all. If I do a web search for Linux kickstart, the search results I get lead me to documentation that is actually quite good. I am not a Linux expert, but I can read that stuff and figure out how to set it up reasonably quickly. If I do a web search for FreeBSD kickstart, the top search results I get lead to a few broken web links, and some private notes from different people on the Internet. The notes are not bad, but not as straightforward to follow as the Linux documentation links. Since you have some good experience with this, can we create a thread on https://forums.freebsd.org with the title FreeBSD kickstart with some step-by-step examples for creating a kickstart environment? That way over time, a web search for FreeBSD kickstart will show the forum post with top-notch examples. Thanks. -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Devops question: unattended installs of FreeBSD?
On 2015-01-13 16:10, Craig Rodrigues wrote: On Mon, Jan 12, 2015 at 12:12 PM, John Nielsen li...@jnielsen.net wrote: I'd be happy to provide more specific suggestions if needed. It really depends on how fully automated you want things to be and how much customization you want to include, as well as what you have available in the install environment. If you're installing on live VMs then you first have to get them booted. A custom ISO or MFS image is probably the simplest for that, though PXE is also an option. (Actually, serving an mfsBSD image via PXE is pretty straightforward.) Thanks! You provided some excellent concrete examples for how to do unattended FreeBSD installs. In the past 6 months, I have had two different people ask me how to: - create a PXE boot server - take the ISO image for FreeBSD 9.2, FreeBSD 10.1, etc. - create a kickstart environment where it is possible to PXE boot a cluster of machines, and have an unattended kickstart install take place of the various FreeBSD versions I have coded this kind of stuff up myself in the past and written my own scripts. However, it would be really nice if we had more straightforward documentation and example scripts for doing this. That way, the average devops engineer experienced with Linux and kickstart can set this up with no problem, instead of having to struggle and figure things out. It looks like all this stuff is possible under FreeBSD. The main problem I see is that the access to the documentation for doing this is not straightforward at all. If I do a web search for Linux kickstart, the search results I get lead me to documentation that is actually quite good. I am not a Linux expert, but I can read that stuff and figure out how to set it up reasonably quickly. If I do a web search for FreeBSD kickstart, the top search results I get lead to a few broken web links, and some private notes from different people on the Internet. The notes are not bad, but not as straightforward to follow as the Linux documentation links. Since you have some good experience with this, can we create a thread on https://forums.freebsd.org with the title FreeBSD kickstart with some step-by-step examples for creating a kickstart environment? That way over time, a web search for FreeBSD kickstart will show the forum post with top-notch examples. Thanks. -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Martin Matuška gave a talk at AsiaBSDCon 2014 about using 'Foreman' in addition to Puppet to manage FreeBSD instances. Slides: http://blog.vx.sk/pdf/foreman-asiabsdcon2014.pdf Video: https://www.youtube.com/watch?v=nb8jB5x0OX4 So foreman does the inventory management (for physical hosts etc), and manages network booting mfsBSD and running the scripted installation, and boot strapping puppet, which then takes over the config management side of things. My self, I've looked at using zfs send|receive to deploy new hosts, since I am deploying servers in remote data centers where I don't have another host to PXE Boot off of. -- Allan Jude signature.asc Description: OpenPGP digital signature
Connected sanitizer libraries to the build (for x86)
Hi, In r277146, I have connected the sanitizer libraries from compiler-rt to the build. Currently, this works for i386 and amd64, and contains Address Sanitizer (ASan) and Undefined Behavior Sanitizer. AddressSanitizer is a fast memory error detector [1]. It consists of a compiler instrumentation module and a run-time library. The tool can detect the following types of bugs: * Out-of-bounds accesses to heap, stack and globals * Use-after-free * Use-after-return (to some extent) * Double-free, invalid free * Memory leaks (experimental) The typical slowdown introduced by AddressSanitizer is 2x. Enable it by compiling and linking with clang, and using the -fsanitize=address flag. Undefined Behavior Sanitizer is a fast and compatible undefined behavior checker, which enables a number of checks that have small runtime cost and no impact on address space layout or ABI. Enable it by using the -fsanitize=undefined flag. [2] Please note that the sanitizers still have some rough edges on FreeBSD, particularly on i386. These will hopefully be smoothed out in the coming time. Reports of problems (and fixes :) are very welcome, but please log them in Bugzilla, so they can be tracked. -Dimitry [1] http://llvm.org/releases/3.5.0/tools/clang/docs/AddressSanitizer.html [2] http://llvm.org/releases/3.5.0/tools/clang/docs/UsersManual.html#opt-fsanitize-undefined signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Connected sanitizer libraries to the build (for x86)
On Tue, 13 Jan 2015 21:28:08 +0100 Dimitry Andric d...@freebsd.org wrote Hi, In r277146, I have connected the sanitizer libraries from compiler-rt to the build. Currently, this works for i386 and amd64, and contains Address Sanitizer (ASan) and Undefined Behavior Sanitizer. AddressSanitizer is a fast memory error detector [1]. It consists of a compiler instrumentation module and a run-time library. The tool can detect the following types of bugs: * Out-of-bounds accesses to heap, stack and globals * Use-after-free * Use-after-return (to some extent) * Double-free, invalid free * Memory leaks (experimental) The typical slowdown introduced by AddressSanitizer is 2x. Enable it by compiling and linking with clang, and using the -fsanitize=address flag. Undefined Behavior Sanitizer is a fast and compatible undefined behavior checker, which enables a number of checks that have small runtime cost and no impact on address space layout or ABI. Enable it by using the -fsanitize=undefined flag. [2] Please note that the sanitizers still have some rough edges on FreeBSD, particularly on i386. These will hopefully be smoothed out in the coming time. Reports of problems (and fixes :) are very welcome, but please log them in Bugzilla, so they can be tracked. These looks like welcomed addition(s). Thank you, Dimitry! --Chris -Dimitry [1] http://llvm.org/releases/3.5.0/tools/clang/docs/AddressSanitizer.html [2] http://llvm.org/releases/3.5.0/tools/clang/docs/UsersManual.html#opt-fsanitiz ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Devops question: freebsd-update needs a real tty to run, problem for automation
On 1/13/2015 5:11 PM, Craig Rodrigues wrote: Hi, Ahmed Kamal, a devops expert, is helping me to script the steps to upgrade a cluster of FreeBSD machines. For certain machines, we want to track the official FreeBSD releases and use freebsd-update to install official updates. We found that when the invocation of freebsd-update was scripted and not run via a real tty, we can into this error: freebsd-update fetch should not be run non-interactively. There are various workarounds mentioned on various web pages. However, should we modify freebsd-update so that it can work better when not run via a real tty? This would make it more devops/automation friendly. The closest thing I have found is freebsd-update cron, which can fetch the updates and run without a real tty. The only problem with freebsd-update cron is that it sleeps a random amount of time between 1 and 3600 seconds before fetching the updates. This is OK when run in a cron job, but not OK when run as part of a devops automation framework. Anybody have ideas as to the best way to proceed in fixing this in freebsd-update? -- Craig sed -i '' -e 's,-t 0 ];,-t 0 ] \\ [ 0 -eq 1 ],' /usr/sbin/freebsd-update This is untested. We'll likely put it in Poudriere as well. IMHO the check should be removed in the official version. -- Regards, Bryan Drewery signature.asc Description: OpenPGP digital signature
Re: Devops question: freebsd-update needs a real tty to run, problem for automation
On Tue, Jan 13, 2015 at 05:29:16PM -0600, Bryan Drewery wrote: On 1/13/2015 5:11 PM, Craig Rodrigues wrote: Hi, Ahmed Kamal, a devops expert, is helping me to script the steps to upgrade a cluster of FreeBSD machines. For certain machines, we want to track the official FreeBSD releases and use freebsd-update to install official updates. We found that when the invocation of freebsd-update was scripted and not run via a real tty, we can into this error: freebsd-update fetch should not be run non-interactively. There are various workarounds mentioned on various web pages. However, should we modify freebsd-update so that it can work better when not run via a real tty? This would make it more devops/automation friendly. The closest thing I have found is freebsd-update cron, which can fetch the updates and run without a real tty. The only problem with freebsd-update cron is that it sleeps a random amount of time between 1 and 3600 seconds before fetching the updates. This is OK when run in a cron job, but not OK when run as part of a devops automation framework. Anybody have ideas as to the best way to proceed in fixing this in freebsd-update? -- Craig sed -i '' -e 's,-t 0 ];,-t 0 ] \\ [ 0 -eq 1 ],' /usr/sbin/freebsd-update This is untested. We'll likely put it in Poudriere as well. IMHO the check should be removed in the official version. You do not need it in poudriere as the rexec binary emulates a tty to workaround this. Best regards, Bapt pgpTkV4xcLG1l.pgp Description: PGP signature
Re: Devops question: freebsd-update needs a real tty to run, problem for automation
On Tue, Jan 13, 2015 at 3:29 PM, Bryan Drewery bdrew...@freebsd.org wrote:
On 1/13/2015 5:11 PM, Craig Rodrigues wrote:
Hi,
Ahmed Kamal, a devops expert, is helping me to script the steps to
upgrade a cluster of FreeBSD machines. For certain machines,
we want to track the official FreeBSD releases and use freebsd-update
to install official updates.
We found that when the invocation of freebsd-update was scripted
and not run via a real tty, we can into this error:
freebsd-update fetch should not be run non-interactively.
There are various workarounds mentioned on various web pages.
However, should we modify freebsd-update so that it can work better
when not run via a real tty? This would make it more devops/automation
friendly.
The closest thing I have found is freebsd-update cron, which can fetch
the updates and run without a real tty. The only problem with
freebsd-update cron
is that it sleeps a random amount of time between 1 and 3600 seconds before
fetching the updates. This is OK when run in a cron job,
but not OK when run as part of a devops automation framework.
Anybody have ideas as to the best way to proceed in fixing this in
freebsd-update?
--
Craig
sed -i '' -e 's,-t 0 ];,-t 0 ] \\ [ 0 -eq 1 ],' /usr/sbin/freebsd-update
This is untested. We'll likely put it in Poudriere as well.
freebsd-update needs to grow a non-interactive option probably:
2375 read dummy /dev/tty
2376 ${EDITOR} `pwd`/merge/new/${F} /dev/tty
2377 done failed.merges
2378 rm failed.merges
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Devops question: freebsd-update needs a real tty to run, problem for automation
On Tue, Jan 13, 2015 at 4:09 PM, NGie Cooper yaneurab...@gmail.com wrote:
On Tue, Jan 13, 2015 at 3:29 PM, Bryan Drewery bdrew...@freebsd.org wrote:
On 1/13/2015 5:11 PM, Craig Rodrigues wrote:
Hi,
Ahmed Kamal, a devops expert, is helping me to script the steps to
upgrade a cluster of FreeBSD machines. For certain machines,
we want to track the official FreeBSD releases and use freebsd-update
to install official updates.
We found that when the invocation of freebsd-update was scripted
and not run via a real tty, we can into this error:
freebsd-update fetch should not be run non-interactively.
There are various workarounds mentioned on various web pages.
However, should we modify freebsd-update so that it can work better
when not run via a real tty? This would make it more devops/automation
friendly.
The closest thing I have found is freebsd-update cron, which can fetch
the updates and run without a real tty. The only problem with
freebsd-update cron
is that it sleeps a random amount of time between 1 and 3600 seconds before
fetching the updates. This is OK when run in a cron job,
but not OK when run as part of a devops automation framework.
Anybody have ideas as to the best way to proceed in fixing this in
freebsd-update?
--
Craig
sed -i '' -e 's,-t 0 ];,-t 0 ] \\ [ 0 -eq 1 ],' /usr/sbin/freebsd-update
This is untested. We'll likely put it in Poudriere as well.
freebsd-update needs to grow a non-interactive option probably:
2375 read dummy /dev/tty
2376 ${EDITOR} `pwd`/merge/new/${F} /dev/tty
2377 done failed.merges
2378 rm failed.merges
$ grep -nr /dev/tty `which freebsd-update`
2375:read dummy /dev/tty
2376:${EDITOR} `pwd`/merge/new/${F} /dev/tty
2405:continuep /dev/tty || return 1
2417:continuep /dev/tty || return 1
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Devops question: freebsd-update needs a real tty to run, problem for automation
On 2015-01-13 18:11, Craig Rodrigues wrote: Hi, Ahmed Kamal, a devops expert, is helping me to script the steps to upgrade a cluster of FreeBSD machines. For certain machines, we want to track the official FreeBSD releases and use freebsd-update to install official updates. We found that when the invocation of freebsd-update was scripted and not run via a real tty, we can into this error: freebsd-update fetch should not be run non-interactively. There are various workarounds mentioned on various web pages. However, should we modify freebsd-update so that it can work better when not run via a real tty? This would make it more devops/automation friendly. The closest thing I have found is freebsd-update cron, which can fetch the updates and run without a real tty. The only problem with freebsd-update cron is that it sleeps a random amount of time between 1 and 3600 seconds before fetching the updates. This is OK when run in a cron job, but not OK when run as part of a devops automation framework. Anybody have ideas as to the best way to proceed in fixing this in freebsd-update? -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org I think this requirement was originally added when Colin hosted the mirrors for FreeBSD update himself, and was worried about everyone scripting it to run via crontab at midnight every night. It is likely a false requirement, and can be safely removed. Dealing with the merges, only really affects version upgrades, and is less of an issue compared to being able to automate security fixes. -- Allan Jude signature.asc Description: OpenPGP digital signature
