Re: Using the Hetzner Cloud
Hi Christopher, Christopher Baines skribis: > I've been using Hetzner for hosting data.guix.gnu.org for at least the > last few months now. I've been paying for it so far, but ideally it > wouldn't depend on me personally paying for it in the long term. > > Coupled with this, I think there are some services (hpc.guix.info, ...) > it would be nice to move off bayfront (a machine in Bordeaux) to > somewhere that's more reliable/remotely accessible, mostly to avoid the > risk of me breaking them again when attempting to reconfigure bayfront. > > Maybe Guix Europe (a legal entity in Europe) could have a Hetzner > account that could be used at the discretion of those involved in Guix > Europe? I think Hetzner support billing by SEPA direct debit, which > might work, otherwise I can probably pay the bill and then be > reimbursed. Yes, we should get that done. Alternately, if Guix Europe cannot do it, we can at least get you reimbursed via our funds at the Free Software Foundation. Andreas, Manolis: thoughts? Ludo’.
Re: Using the Hetzner Cloud
Heya,
Giovanni Biscuolo writes:
> Hello Alex,
>
> Alex Sassmannshausen writes:
>
>> Giovanni Biscuolo writes:
>
> [...]
>
>>> could you please share that "guix-infect" script?
>>
>> Sure, please see attached. This one here is a bash script that works
>> with a system config that is specified as part of a here-doc in the bash
>> script.
>
> Cool, thanks!
>
> [...]
>
>> — primarily to encourage myself to keep automating further by
>> integrating this in Guix deploy. Where does all the time go!!!
>
> I'd like to help but still need to enhance my guile-foo
>
>> Happy to help if you run into problems or have questions.
>
> maybe when guix will be in Debian [1] your script could be simpified in
> the first steps... but I hope we'll have a proper guix-deploy way for
> many many providers before :-)
>
> just one more question:
>
> [...]
>
>> #!/bin/bash
>>
>> e2label /dev/sda1 root
>
> [...]
>
>> guix system build /etc/bootstrap-config.scm
>> guix system reconfigure /etc/bootstrap-config.scm
>> mv /etc /old-etc
>> mkdir /etc
>> cp -r /old-etc/{passwd,group,shadow,gshadow,mtab,guix,bootstrap-config.scm}
>> /etc/
>> guix system reconfigure /etc/bootstrap-config.scm
>
> why two "guix system reconfigure"?
Tbh, not 100% sure. This script is heavily copypasta. It looks like
there is an element of transferring debian files away over the course of
the 2, so perhaps it's to fully transition.
But, tbh, not really investigated it, as the second reconfigure is dirt
cheap, and the script just works (tm).
Feel free to experiment! If you find it's unnecessary, I'd be happy to
hear about it!
Cheers,
Alex
Re: Using the Hetzner Cloud
Hello Alex,
Alex Sassmannshausen writes:
> Giovanni Biscuolo writes:
[...]
>> could you please share that "guix-infect" script?
>
> Sure, please see attached. This one here is a bash script that works
> with a system config that is specified as part of a here-doc in the bash
> script.
Cool, thanks!
[...]
> — primarily to encourage myself to keep automating further by
> integrating this in Guix deploy. Where does all the time go!!!
I'd like to help but still need to enhance my guile-foo
> Happy to help if you run into problems or have questions.
maybe when guix will be in Debian [1] your script could be simpified in
the first steps... but I hope we'll have a proper guix-deploy way for
many many providers before :-)
just one more question:
[...]
> #!/bin/bash
>
> e2label /dev/sda1 root
[...]
> guix system build /etc/bootstrap-config.scm
> guix system reconfigure /etc/bootstrap-config.scm
> mv /etc /old-etc
> mkdir /etc
> cp -r /old-etc/{passwd,group,shadow,gshadow,mtab,guix,bootstrap-config.scm}
> /etc/
> guix system reconfigure /etc/bootstrap-config.scm
why two "guix system reconfigure"?
[...]
Thanks! Gio'
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644
--
Giovanni Biscuolo
Xelera IT Infrastructures
signature.asc
Description: PGP signature
Re: Using the Hetzner Cloud
Heya,
Giovanni Biscuolo writes:
> Hello Alex,
>
> Alex Sassmannshausen writes:
>
> [...]
>
>> Now I use a different approach: deploy a debian server then use a
>> guix-infect style script (gleaned from the guix deploy code for digital
>> ocean).
>
> could you please share that "guix-infect" script?
Sure, please see attached. This one here is a bash script that works
with a system config that is specified as part of a here-doc in the bash
script.
It's all a bit gaffer tape & macgyver — primarily to encourage myself to
keep automating further by integrating this in Guix deploy. Where does
all the time go!!!
Happy to help if you run into problems or have questions.
> I think this could/should become an entry in our cookbook, in a similar
> way NixOS does here:
> https://nixos.wiki/wiki/NixOS_friendly_hosters#Hoster-agnostic_means_of_installation
Agreed, a cookbook recipe that summarises the existing ways of deploying
would be cool. I'd be happy to read over and give feedback on any
proposed articles.
Ellen Papsch writes:
> […]
> mv var/guix /var/ && mv gnu /
>
> there seems to be a complete takeover, even better than a FrankenDebian
> :-)
Agreed — it's testament to the versatility of Guix that it can literally
do this hostile take-over. Very cool :-)
Cheers,
Alex
#!/bin/bash
e2label /dev/sda1 root
apt-get update
apt-get install xz-utils -y
wget https://ftp.gnu.org/gnu/guix/guix-binary-1.0.1.x86_64-linux.tar.xz
cd /tmp
tar --warning=no-timestamp -xf ~/guix-binary-1.0.1.x86_64-linux.tar.xz
mv var/guix /var/ && mv gnu /
mkdir -p ~root/.config/guix
ln -sf /var/guix/profiles/per-user/root/current-guix ~root/.config/guix/current
export GUIX_PROFILE="`echo ~root`/.config/guix/current" ;
source $GUIX_PROFILE/etc/profile
groupadd --system guixbuild
for i in `seq -w 1 10`; do
useradd -g guixbuild -G guixbuild-d /var/empty -s `which nologin` -c "Guix build user $i" --system guixbuilder$i;
done;
cp ~root/.config/guix/current/lib/systemd/system/guix-daemon.service /etc/systemd/system/
systemctl start guix-daemon && systemctl enable guix-daemon
mkdir -p /usr/local/bin
cd /usr/local/bin
ln -s /var/guix/profiles/per-user/root/current-guix/bin/guix
mkdir -p /usr/local/share/info
cd /usr/local/share/info
for i in /var/guix/profiles/per-user/root/current-guix/share/info/*; do
ln -s $i;
done
guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub
# FIXME: I'm pulling a commit that fixes some issues. When there is a new
# guix release this can be removed.
guix pull --commit=3a695c01d7ee18f30f22df53f3c44dfac04017f1
guix package -i openssl
# FIXME: Just loading the default example from the guix manual here. This can
# be adapted to whatever base guix deployment you want.
cat > /etc/bootstrap-config.scm << EOF
;; This is an operating system configuration template
;; for a "bare bones" setup, with no X11 display server.
(use-modules (gnu))
(use-service-modules networking ssh)
(use-package-modules screen)
(operating-system
(host-name "komputilo")
(timezone "Europe/Berlin")
(locale "en_US.utf8")
;; Boot in "legacy" BIOS mode, assuming /dev/sdX is the
;; target hard disk, and "my-root" is the label of the target
;; root file system.
(bootloader (bootloader-configuration
(bootloader grub-bootloader)
(target "/dev/sdX")))
(file-systems (cons (file-system
(device (file-system-label "my-root"))
(mount-point "/")
(type "ext4"))
%base-file-systems))
;; This is where user accounts are specified. The "root"
;; account is implicit, and is initially created with the
;; empty password.
(users (cons (user-account
(name "alice")
(comment "Bob's sister")
(group "users")
;; Adding the account to the "wheel" group
;; makes it a sudoer. Adding it to "audio"
;; and "video" allows the user to play sound
;; and access the webcam.
(supplementary-groups '("wheel"
"audio" "video")))
%base-user-accounts))
;; Globally-installed packages.
(packages (cons screen %base-packages))
;; Add services to the baseline: a DHCP client and
;; an SSH server.
(services (append (list (service dhcp-client-service-type)
(service openssh-service-type
(openssh-configuration
(port-number
%base-services)))
EOF
guix system build /etc/bootstrap-config.scm
guix system reconfigure /etc/bootstrap-config.scm
Re: Using the Hetzner Cloud
On 17.02.20 18:15, Alex Sassmannshausen wrote:> This is by far the fastest way of deploying to Hetzner VMs I've found > yet. > > It also means you can use Hetzner's CLI for creating and managing your > VMs. > > It's on my to do list to create a solid guix deploy server creation > module for guix. That would be pretty neat!
Re: Using the Hetzner Cloud
On 17.02.20 17:55, Ellen Papsch wrote:> a few days ago I requested the Guix installer from Hetzner as well. I > gave them the ISO URL from Guix website. They mounted it without > modification, which prevented boot due to the image being compressed. I > had to supply the uncompressed image, which booted fine. > > The installer was not able to install right away. I could complete all > the GUI steps. When it came to guix system init, it bailed with the > message that an initrd-module may be missing. System init (and the > subsequent boot) worked fine with: > > (initrd-modules (append (list "virtio_pci" > "virtio_scsi") > %base-initrd-modules)) AFAIK it's fixed in Guix master. But 1.0.1 is missing the patch. It's time for new release :) > BTW, the VGA console is very limited, you can't select text or send > special key strokes, e.g. control keys. It suffices for activating SSH > though :-) The console is limited but was always good enough for my use cases :) > Hetzner added the image only for me, which is a hindrance for Guix > adoption. Another limitation is that I can't create new servers right > away without going through installation. I can dance around it by > creating a snapshot of a template server and creating the new server > from that snapshot, then reconfiguring the system (or doing a deploy, I > want to try that yet). > > So, doing Guix is a little more inconvenient than a simple, plain > Debian. I can still recommend it, though you may find Hetzner not > meeting your demands in general: The cloud offerings have a rather low > cpu/memory ratio (0.25 cpu/1G RAM) and it's not configurable. Going through the installer all the time if you have more servers is annoying. But for smaller setups with one or two servers pretty okay.
Re: Using the Hetzner Cloud
Hello Alex, Alex Sassmannshausen writes: [...] > Now I use a different approach: deploy a debian server then use a > guix-infect style script (gleaned from the guix deploy code for digital > ocean). could you please share that "guix-infect" script? I think this could/should become an entry in our cookbook, in a similar way NixOS does here: https://nixos.wiki/wiki/NixOS_friendly_hosters#Hoster-agnostic_means_of_installation [...] > It's on my to do list to create a solid guix deploy server creation > module for guix. that's the long term solution and it'll be fantastic, IMHO it's still interesting to show a "hoster agnostic means of installation" for Guix Thanks! Gio' [...] -- Giovanni Biscuolo Xelera IT Infrastructures signature.asc Description: PGP signature
Re: Using the Hetzner Cloud
Am Montag, den 17.02.2020, 18:15 +0100 schrieb Alex Sassmannshausen: > > > So I deploy debian, then copy across a script and run that. This > takes > care of turning the debian machine into a guix machine and deploys my > sys config immediately. > > This is by far the fastest way of deploying to Hetzner VMs I've found > yet. > > It also means you can use Hetzner's CLI for creating and managing > your > VMs. That sounds nice. I had wondered if there's some sort of FrankenDebian way but did not pursue it. Given this line in the script: mv var/guix /var/ && mv gnu / there seems to be a complete takeover, even better than a FrankenDebian :-) > It's on my to do list to create a solid guix deploy server creation > module for guix. That would certainly be appreciated. I'm currently in the process of cementing Guix usage here (for databases at least). If I succeed and get some time, I might be able to do a first stab. Though it would be more useful for applications, which come and go more often than databases. We currently manage apps with ansible, I already have started lobbying at low rate. But first things first -- databases! Best regards
Re: Using the Hetzner Cloud
Ricardo Wurmus writes:
> Christopher Baines writes:
>
>> Coupled with this, I think there are some services (hpc.guix.info, ...)
>> it would be nice to move off bayfront (a machine in Bordeaux) to
>> somewhere that's more reliable/remotely accessible, mostly to avoid the
>> risk of me breaking them again when attempting to reconfigure bayfront.
>
> That’s unfortunate. Why is bayfront less reliable than we want it to
> be? Surely that’s not because it’s using Guix System, because
> {berlin,ci}.guix.gnu.org also runs Guix System and has been reliably
> hosting services for us.
I guess having access to recover from issues is the primary concern, and
reliability relates to that.
Around FOSDEM, I think I caused Bayfront to restart while attempting to
restart cuirass. I'm not sure exactly what happened, as my SSH
connection just dropped, and the machine became inaccessible (including
Cuirass, hpc.guix.info, ...).
Luckily, some time later Ludovic and Andreas were available to remotely
access the power supply, and turn the machine off and on, this did seem
to bring the machine back.
> I would not mind hosting services on one of the 30 new servers the MDC
> bought for ci.guix.gnu.org, but I’d be happier if we figured out what’s
> wrong with bayfront instead of centralizing all services in Berlin…
I'm definitely not saying anything is wrong with Bayfront, just that it
would be good to maybe check what the expectations are with the services
it hosts. Given that the remote access only allows power cycling the
machine, if that doesn't work, there's the potential for downtime for
those services to last until someone physically goes to the machine to
fix it, if SSH access can't be restored by turning it off and on again.
signature.asc
Description: PGP signature
Re: Using the Hetzner Cloud
Christopher Baines writes:
> Coupled with this, I think there are some services (hpc.guix.info, ...)
> it would be nice to move off bayfront (a machine in Bordeaux) to
> somewhere that's more reliable/remotely accessible, mostly to avoid the
> risk of me breaking them again when attempting to reconfigure bayfront.
That’s unfortunate. Why is bayfront less reliable than we want it to
be? Surely that’s not because it’s using Guix System, because
{berlin,ci}.guix.gnu.org also runs Guix System and has been reliably
hosting services for us.
I would not mind hosting services on one of the 30 new servers the MDC
bought for ci.guix.gnu.org, but I’d be happier if we figured out what’s
wrong with bayfront instead of centralizing all services in Berlin…
--
Ricardo
Re: Using the Hetzner Cloud
Ellen Papsch writes: > a few days ago I requested the Guix installer from Hetzner as well. I > gave them the ISO URL from Guix website. They mounted it without > modification, which prevented boot due to the image being compressed. I > had to supply the uncompressed image, which booted fine. I had exactly the same experience, I've had this with other services as well. For the next release at least, providing there are no big technical blockers, it would be good to publish an uncompressed ISO image, as well as a compressed one. > Hetzner added the image only for me, which is a hindrance for Guix > adoption. Another limitation is that I can't create new servers right > away without going through installation. I can dance around it by > creating a snapshot of a template server and creating the new server > from that snapshot, then reconfiguring the system (or doing a deploy, I > want to try that yet). Hopefully if enough people ask them, they might include it in the ISO images they provide to everyone. Thanks for sharing your experiences, Chris signature.asc Description: PGP signature
Re: Using the Hetzner Cloud
Jonathan Brielmaier writes: > as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they > could provide us some free VMs in their cloud[1]. > > A few days ago they came back go to me. Sadly they can't provide us free > VMs, but they will give us 30 EUR credit for the first month. > > As they bill hour-wise and offer a serial console over HTTPS, you can do > tests for stuff like the installer for 2 or 3 cents :) > > What do you think about it? Thanks for looking in to the Jonathan, that's really useful. I've been using Hetzner for hosting data.guix.gnu.org for at least the last few months now. I've been paying for it so far, but ideally it wouldn't depend on me personally paying for it in the long term. Coupled with this, I think there are some services (hpc.guix.info, ...) it would be nice to move off bayfront (a machine in Bordeaux) to somewhere that's more reliable/remotely accessible, mostly to avoid the risk of me breaking them again when attempting to reconfigure bayfront. Maybe Guix Europe (a legal entity in Europe) could have a Hetzner account that could be used at the discretion of those involved in Guix Europe? I think Hetzner support billing by SEPA direct debit, which might work, otherwise I can probably pay the bill and then be reimbursed. Thanks again for following up talking to Hetzner Jonathan! Chris signature.asc Description: PGP signature
Re: Using the Hetzner Cloud
Hello, Ellen Papsch writes: > Hi, > > Am Montag, den 17.02.2020, 14:47 +0100 schrieb Jonathan Brielmaier: >> Hi folks, >> >> as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they >> could provide us some free VMs in their cloud[1]. >> >> […] >> >> What do you think about it? >> >> ~Jonathan >> > > a few days ago I requested the Guix installer from Hetzner as well. I > […] > cpu/memory ratio (0.25 cpu/1G RAM) and it's not configurable. > > Best regards > Ellen Figured I might as well add my experiences. I originally have done the same thing as you did Ellen. This was a bit of a slow process for deploying Guix to Hetzner servers though. Now I use a different approach: deploy a debian server then use a guix-infect style script (gleaned from the guix deploy code for digital ocean). So I deploy debian, then copy across a script and run that. This takes care of turning the debian machine into a guix machine and deploys my sys config immediately. This is by far the fastest way of deploying to Hetzner VMs I've found yet. It also means you can use Hetzner's CLI for creating and managing your VMs. It's on my to do list to create a solid guix deploy server creation module for guix. Hth, Alex
Re: Using the Hetzner Cloud
Hi, Am Montag, den 17.02.2020, 14:47 +0100 schrieb Jonathan Brielmaier: > Hi folks, > > as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they > could provide us some free VMs in their cloud[1]. > > A few days ago they came back go to me. Sadly they can't provide us > free > VMs, but they will give us 30 EUR credit for the first month. > > As they bill hour-wise and offer a serial console over HTTPS, you can > do > tests for stuff like the installer for 2 or 3 cents :) > > What do you think about it? > > ~Jonathan > a few days ago I requested the Guix installer from Hetzner as well. I gave them the ISO URL from Guix website. They mounted it without modification, which prevented boot due to the image being compressed. I had to supply the uncompressed image, which booted fine. The installer was not able to install right away. I could complete all the GUI steps. When it came to guix system init, it bailed with the message that an initrd-module may be missing. System init (and the subsequent boot) worked fine with: (initrd-modules (append (list "virtio_pci" "virtio_scsi") %base-initrd-modules)) BTW, the VGA console is very limited, you can't select text or send special key strokes, e.g. control keys. It suffices for activating SSH though :-) Hetzner added the image only for me, which is a hindrance for Guix adoption. Another limitation is that I can't create new servers right away without going through installation. I can dance around it by creating a snapshot of a template server and creating the new server from that snapshot, then reconfiguring the system (or doing a deploy, I want to try that yet). So, doing Guix is a little more inconvenient than a simple, plain Debian. I can still recommend it, though you may find Hetzner not meeting your demands in general: The cloud offerings have a rather low cpu/memory ratio (0.25 cpu/1G RAM) and it's not configurable. Best regards Ellen
Using the Hetzner Cloud
Hi folks, as promised on the Guix Days in Bruxelles I asked Hetzner[0] if they could provide us some free VMs in their cloud[1]. A few days ago they came back go to me. Sadly they can't provide us free VMs, but they will give us 30 EUR credit for the first month. As they bill hour-wise and offer a serial console over HTTPS, you can do tests for stuff like the installer for 2 or 3 cents :) What do you think about it? ~Jonathan P.S: I encourage you to ask your local/well-known hosting provider for some support in any way for Guix. It's not so much effort. [0] https://www.hetzner.de/ big German hosting company [1] https://www.hetzner.de/cloud
