Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:  closed
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:  fixed
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by anarcat):

 * status:  merge_ready => closed
 * resolution:   => fixed


Comment:

 kvm4 gone from hetzner too. all done here.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  merge_ready
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 hum, okay... it failed with an error, and when i tried to open a new
 script to test, everything collapse in a flaming heap. now the server
 doesn't respond to pings, so hopefully it's really dead now.

 tomorrow hetzner should retire it completely and this will be done.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  merge_ready
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 hum. the first wipe didn't automatically exit, so it probably got hung
 there for a few hours. i hit "enter" and it started the second round. eta
 45 minutes to apocalypse.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  merge_ready
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by anarcat):

 * status:  accepted => merge_ready


Comment:

 I scheduled deletion with hetzner for now + 2days:

 >Please note that this server will be cancelled on 28/05/2020 and all
 data will be deleted.
 >
 > Confirmation* I have read and understood the above message. I confirm
 that I want to cancel my server. The cancellation will take effect on
 28/05/2020.

 i'll keep this opened until the server is canceled, but this is all but
 done.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 started this final wipe:

 {{{
 nwipe --autonuke --method=random --verify=off /dev/sda ; \
 nwipe --autonuke --method=random --rounds=2 /dev/nvme0n1 ; \
 echo "SHUTTING DOWN FOREVER IN ONE MINUTE" ; \
 sleep 60 ; \
 echo o > /proc/sysrq-trigger
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 interestingly, the nvme wipe completed now. so i'm now entering the last
 stage of the wiping process, which should logically complete in another
 24h.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 sdb wipe completed. 12h remaining for nvme1n1.

 since the machine has been removed from puppet/ldap, its public key is not
 available from the servers anymore. if you need to connect, you can use
 the following known_hosts:

 {{{
 94.130.38.33 ssh-ed25519
 C3NzaC1lZDI1NTE5IMB96RR18dP0BQ9ySEcpWuBcF4z5rET4KpvzOLKx4m9T
 root@kvm4
 }}}

 ... and connect directly to the IP address.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 in progress:

 {{{
 nwipe --autonuke --method=random --verify=off /dev/sdb
 nwipe --autonuke --method=random --rounds=2 /dev/nvme1n1
 }}}

 next up is the final self-destruct procedure and cancelation.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by anarcat):

 started retirement procedure:

  1. N/A
  2. removed from nagios
  3. N/A
  4. removed from puppet, backups (fabric)
  5. removed from LDAP:
 {{{
 380 host=kvm4,ou=hosts,dc=torproject,dc=org
 objectClass: top
 objectClass: debianServer
 host: kvm4
 hostname: kvm4.torproject.org
 architecture: amd64
 admin: torproject-ad...@torproject.org
 sshRSAHostKey: ssh-rsa
 
B3NzaC1yc2EDAQABAAABAQCjqP9zljgAnZj666cVW00GaF4rII8t1gYCUvCHX8+OIxffowAuNfxVAfk1AFqCMZCkP8ULLJCevAfrcbc86uVnzQ9J0bND4gFJE4dTDQeCgVrPqR+bt48V2U/KbYC93q2QTRa/UjHitBxO7Z1ryvYh0J0HluJew9ZIBXZ21/uqkqxQ4GWLXo7fXHOHTzKEtP6wwpWiYc9IOEfe4+93vmNX0ubPfgnsAh+2+2/SPUQairOc4c7XmSQM/fyyadoIit/lYANuXfPNidXQtgie1jggyD+Ti72mtHI7pRlVSXKRhXOiuganboENv0Hb9KszStLYnkk/3jJnCAxcGP6VVdpx
 root@kvm4
 sshRSAHostKey: ssh-ed25519
 C3NzaC1lZDI1NTE5IMB96RR18dP0BQ9ySEcpWuBcF4z5rET4KpvzOLKx4m9T
 root@kvm4
 description: KVM host
 purpose: KVM host
 access: restricted
 distribution: Debian
 ipHostNumber: 94.130.38.33
 ipHostNumber: 2a01:4f8:10b:239f::2
 l: Falkenstein, Saxony, Germany
 rebootPolicy: manual
 }}}
  6. removed from source code (puppet, auto-dns, domains, wiki)
  7. removed from tor-passwords
  8. N/A (dnswl)
  9. removed from spreadsheet

 last steps: disk wipes and cancelation with hetzner.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [x] alberti.torproject.org (LDAP) #33908
>  * [/] build-x86-09.torproject.org (build server) - RETIRE - replaced
> with build-x86-11 on gnt-fsn
>  * [x] eugeni.torproject.org (email) #32803
>  * [x] meronense.torproject.org (metrics) #33909
>  * [x] neriniflorum.torproject.org (DNS) #33910
>  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
>  * [x] pauli.torproject.org (puppet) #33912
>  * [x] rouyi.torproject.org (jenkins) #33913
>  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
>  * [x] weissii.torproject.org (windows build box) #33914
>  * [ ] winklerianum.torproject.org (windows build box) turned off,
> RETIRE?
>  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [x] alberti.torproject.org (LDAP) #33908
  * [x] build-x86-09.torproject.org (build server) - RETIRE - replaced with
 build-x86-11 on gnt-fsn
  * [x] eugeni.torproject.org (email) #32803
  * [x] meronense.torproject.org (metrics) #33909
  * [x] neriniflorum.torproject.org (DNS) #33910
  * [x] oo-hetzner-03.torproject.org (metrics) #33911
  * [x] pauli.torproject.org (puppet) #33912
  * [x] rouyi.torproject.org (jenkins) #33913
  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
  * [x] weissii.torproject.org (windows build box) #33914
  * [x] winklerianum.torproject.org (windows build box) turned off,
 migrated along weissii, but turned off
  * [x] woronowii.torproject.org (windows build box) turned off, migrated
 along weissii, but turned off

--

Comment (by anarcat):

 i just retired oo-hetzner-03, the next step here is to retire kvm4 itself,
 it seems.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by weasel):

 we set up new onionoo infra (cf. #31659), so oo-hetzner-03.torproject.org
 can be shut down and retired in a day or two

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Description changed by weasel:

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [x] alberti.torproject.org (LDAP) #33908
>  * [ ] build-x86-09.torproject.org (build server) - RETIRE - replaced
> with build-x86-11 on gnt-fsn
>  * [ ] eugeni.torproject.org (email) #32803
>  * [x] meronense.torproject.org (metrics) #33909
>  * [x] neriniflorum.torproject.org (DNS) #33910
>  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
>  * [x] pauli.torproject.org (puppet) #33912
>  * [x] rouyi.torproject.org (jenkins) #33913
>  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
>  * [x] weissii.torproject.org (windows build box) #33914
>  * [ ] winklerianum.torproject.org (windows build box) turned off,
> RETIRE?
>  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [x] alberti.torproject.org (LDAP) #33908
  * [/] build-x86-09.torproject.org (build server) - RETIRE - replaced with
 build-x86-11 on gnt-fsn
  * [x] eugeni.torproject.org (email) #32803
  * [x] meronense.torproject.org (metrics) #33909
  * [x] neriniflorum.torproject.org (DNS) #33910
  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
  * [x] pauli.torproject.org (puppet) #33912
  * [x] rouyi.torproject.org (jenkins) #33913
  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
  * [x] weissii.torproject.org (windows build box) #33914
  * [ ] winklerianum.torproject.org (windows build box) turned off, RETIRE?
  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Description changed by weasel:

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [x] alberti.torproject.org (LDAP) #33908
>  * [ ] build-x86-09.torproject.org (build server) - RETIRE - replaced
> with build-x86-11 on gnt-fsn
>  * [ ] eugeni.torproject.org (email) #32803
>  * [ ] meronense.torproject.org (metrics) #33909
>  * [ ] neriniflorum.torproject.org (DNS) #33910
>  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
>  * [x] pauli.torproject.org (puppet) #33912
>  * [x] rouyi.torproject.org (jenkins) #33913
>  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
>  * [x] weissii.torproject.org (windows build box) #33914
>  * [ ] winklerianum.torproject.org (windows build box) turned off,
> RETIRE?
>  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [x] alberti.torproject.org (LDAP) #33908
  * [ ] build-x86-09.torproject.org (build server) - RETIRE - replaced with
 build-x86-11 on gnt-fsn
  * [ ] eugeni.torproject.org (email) #32803
  * [x] meronense.torproject.org (metrics) #33909
  * [x] neriniflorum.torproject.org (DNS) #33910
  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
  * [x] pauli.torproject.org (puppet) #33912
  * [x] rouyi.torproject.org (jenkins) #33913
  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
  * [x] weissii.torproject.org (windows build box) #33914
  * [ ] winklerianum.torproject.org (windows build box) turned off, RETIRE?
  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [x] alberti.torproject.org (LDAP) #33908
>  * [ ] build-x86-09.torproject.org (build server) - RETIRE there's a
> build box on kvm4, kvm5 and two on moly, all of which are scheduled for
> retirement, so we have to keep *some* of those resources
>  * [ ] eugeni.torproject.org (email) #32803
>  * [ ] meronense.torproject.org (metrics) #33909
>  * [ ] neriniflorum.torproject.org (DNS) #33910
>  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
>  * [x] pauli.torproject.org (puppet) #33912
>  * [x] rouyi.torproject.org (jenkins) #33913
>  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
>  * [x] weissii.torproject.org (windows build box) #33914
>  * [ ] winklerianum.torproject.org (windows build box) turned off,
> RETIRE?
>  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [x] alberti.torproject.org (LDAP) #33908
  * [ ] build-x86-09.torproject.org (build server) - RETIRE - replaced with
 build-x86-11 on gnt-fsn
  * [ ] eugeni.torproject.org (email) #32803
  * [ ] meronense.torproject.org (metrics) #33909
  * [ ] neriniflorum.torproject.org (DNS) #33910
  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
  * [x] pauli.torproject.org (puppet) #33912
  * [x] rouyi.torproject.org (jenkins) #33913
  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
  * [x] weissii.torproject.org (windows build box) #33914
  * [ ] winklerianum.torproject.org (windows build box) turned off, RETIRE?
  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

--

Comment (by anarcat):

 build 9 is replaced by build 11

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Comment (by weasel):

 Replying to [comment:7 anarcat]:
 > re the build box, weasel says we can retire it, but we will eventually
 need to create build boxes in the gnt-fsn cluster at some point.

 We have at leaste one build box on gnt-fsn.  We can just shut down
 build-x86-09 when we retire kvm4.  I'm leaving it running for now, because
 it doesn't hurt and it helps a little bit at times.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Description changed by weasel:

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [ ] alberti.torproject.org (LDAP) #33908
>  * [ ] build-x86-09.torproject.org (build server) - RETIRE there's a
> build box on kvm4, kvm5 and two on moly, all of which are scheduled for
> retirement, so we have to keep *some* of those resources
>  * [ ] eugeni.torproject.org (email) #32803
>  * [ ] meronense.torproject.org (metrics) #33909
>  * [ ] neriniflorum.torproject.org (DNS) #33910
>  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
>  * [ ] pauli.torproject.org (puppet) #33912
>  * [ ] rouyi.torproject.org (jenkins) #33913
>  * [ ] web-hetzner-01.torproject.org (static mirror) RETIRE
>  * [ ] weissii.torproject.org (windows build box) #33914
>  * [ ] winklerianum.torproject.org (windows build box) turned off,
> RETIRE?
>  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [x] alberti.torproject.org (LDAP) #33908
  * [ ] build-x86-09.torproject.org (build server) - RETIRE there's a build
 box on kvm4, kvm5 and two on moly, all of which are scheduled for
 retirement, so we have to keep *some* of those resources
  * [ ] eugeni.torproject.org (email) #32803
  * [ ] meronense.torproject.org (metrics) #33909
  * [ ] neriniflorum.torproject.org (DNS) #33910
  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
  * [x] pauli.torproject.org (puppet) #33912
  * [x] rouyi.torproject.org (jenkins) #33913
  * [x] web-hetzner-01.torproject.org (static mirror) RETIRE
  * [x] weissii.torproject.org (windows build box) #33914
  * [ ] winklerianum.torproject.org (windows build box) turned off, RETIRE?
  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-may  |  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-
Changes (by anarcat):

 * keywords:  tpa-roadmap-april => tpa-roadmap-may


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-april|  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [ ] alberti.torproject.org (LDAP) #33908
>  * [ ] build-x86-09.torproject.org (build server) - RETIRE? there's a
> build box on kvm4, kvm5 and two on moly, all of which are scheduled for
> retirement, so we have to keep *some* of those resources
>  * [ ] eugeni.torproject.org (email) #32803
>  * [ ] meronense.torproject.org (metrics) #33909
>  * [ ] neriniflorum.torproject.org (DNS) #33910
>  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
>  * [ ] pauli.torproject.org (puppet) #33912
>  * [ ] rouyi.torproject.org (jenkins) #33913
>  * [ ] web-hetzner-01.torproject.org (static mirror) RETIRE?
>  * [ ] weissii.torproject.org (windows build box) #33914
>  * [ ] winklerianum.torproject.org (windows build box) turned off,
> RETIRE?
>  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [ ] alberti.torproject.org (LDAP) #33908
  * [ ] build-x86-09.torproject.org (build server) - RETIRE there's a build
 box on kvm4, kvm5 and two on moly, all of which are scheduled for
 retirement, so we have to keep *some* of those resources
  * [ ] eugeni.torproject.org (email) #32803
  * [ ] meronense.torproject.org (metrics) #33909
  * [ ] neriniflorum.torproject.org (DNS) #33910
  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
  * [ ] pauli.torproject.org (puppet) #33912
  * [ ] rouyi.torproject.org (jenkins) #33913
  * [ ] web-hetzner-01.torproject.org (static mirror) RETIRE
  * [ ] weissii.torproject.org (windows build box) #33914
  * [ ] winklerianum.torproject.org (windows build box) turned off, RETIRE?
  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

--

Comment (by anarcat):

 re the build box, weasel says we can retire it, but we will eventually
 need to create build boxes in the gnt-fsn cluster at some point.

 web-hetzner-01 can be retired.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #32802 [Internal Services/Tor Sysadmin Team]: retire kvm4, 8 VMs to migrate (was: retire kvm4, 12 VMs to migrate)

[Tor Bug Tracker & Wiki]

#32802: retire kvm4, 8 VMs to migrate
-+-
 Reporter:  anarcat  |  Owner:  anarcat
 Type:  project  | Status:
 |  accepted
 Priority:  High |  Milestone:
Component:  Internal Services/Tor Sysadmin Team  |Version:
 Severity:  Major| Resolution:
 Keywords:  tpa-roadmap-april|  Actual Points:
Parent ID:   | Points:
 Reviewer:   |Sponsor:
-+-

Old description:

> kvm4 is getting fairly old. it's been setup in 2015 and is showing sign
> of old age. for example today it freaked us all out by not returning
> after a reboot right before the holidays (#32801). considering how
> critical that server is (email, puppet, ldap, jenkins, dns, web mirror,
> all the windows buildboxes!) we should start considering a
> decomissionning process.
>
> at the very least, we need to get eugeni the heck out of there.
>
> we have budget to provision another ganeti cluster, so let's use it to
> replace this, and hopefully more. the existing cluster has already taken
> more than its share by taking machines from both kvm1/textile and moly,
> so it's time we provision more hardware for this.
>
> this requires a new ganeti node (fsn-node-06, #33907).
>
> machines to be migrated:
>
>  * [ ] alberti.torproject.org (LDAP)
>  * [ ] build-x86-09.torproject.org (build server)
>  * [ ] eugeni.torproject.org (email) #32803
>  * [ ] meronense.torproject.org (metrics)
>  * [ ] neriniflorum.torproject.org (DNS)
>  * [ ] oo-hetzner-03.torproject.org (metrics)
>  * [ ] pauli.torproject.org (puppet)
>  * [ ] rouyi.torproject.org (jenkins)
>  * [ ] web-hetzner-01.torproject.org (static mirror)
>  * [ ] weissii.torproject.org (windows build box)
>  * [ ] winklerianum.torproject.org (windows build box, turned off)
>  * [ ] woronowii.torproject.org (windows build box, turned off)

New description:

 kvm4 is getting fairly old. it's been setup in 2015 and is showing sign of
 old age. for example today it freaked us all out by not returning after a
 reboot right before the holidays (#32801). considering how critical that
 server is (email, puppet, ldap, jenkins, dns, web mirror, all the windows
 buildboxes!) we should start considering a decomissionning process.

 at the very least, we need to get eugeni the heck out of there.

 we have budget to provision another ganeti cluster, so let's use it to
 replace this, and hopefully more. the existing cluster has already taken
 more than its share by taking machines from both kvm1/textile and moly, so
 it's time we provision more hardware for this.

 this requires a new ganeti node (fsn-node-06, #33907).

 machines to be migrated:

  * [ ] alberti.torproject.org (LDAP) #33908
  * [ ] build-x86-09.torproject.org (build server) - RETIRE? there's a
 build box on kvm4, kvm5 and two on moly, all of which are scheduled for
 retirement, so we have to keep *some* of those resources
  * [ ] eugeni.torproject.org (email) #32803
  * [ ] meronense.torproject.org (metrics) #33909
  * [ ] neriniflorum.torproject.org (DNS) #33910
  * [ ] oo-hetzner-03.torproject.org (metrics) #33911
  * [ ] pauli.torproject.org (puppet) #33912
  * [ ] rouyi.torproject.org (jenkins) #33913
  * [ ] web-hetzner-01.torproject.org (static mirror) RETIRE?
  * [ ] weissii.torproject.org (windows build box) #33914
  * [ ] winklerianum.torproject.org (windows build box) turned off, RETIRE?
  * [ ] woronowii.torproject.org (windows build box) turned off, RETIRE?

--

Comment (by anarcat):

 created a ticket for every VM i think should be migrated, which means we
 would retire 4 VMs here:

  * two windows build boxes
  * a static mirror
  * a build box

 does this make sense?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs