Vestibular SENAI 2017/1 | Inscreva-se e garanta sua vaga
___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
On 2016-Nov-22 10:07:49 +, Pete Frenchwrote: >to another machine and trying to import the pools causes an instant panic. Can you provide details of the panic, please. -- Peter Jeremy signature.asc Description: PGP signature
Re: Help! two machines ran out of swap and corrupted their zpools!
Pete French wrote: zpool import -N -O readonly=on -f -R /mnt/somezpoool If that doesn't help try: zpool import -N -O readonly=on -f -R /mnt/somezpoool -Fn I got someone to do this (am still having toruble finding time as am supposed to be off sick) and it causes instant kernel panic on trying to import the pool. Same as it does on boot. Instapanic, huh... Ok, let's put documentation aside and focus on unsupported development features. You'll need a list of available transactions: zdb -ul This one when given a vdev lists all uberblocks with their respective transaction ids. You can take the highest one (it's not the last one) and try to mount pool with: zpool import -N -O readonly=on -f -R /mnt/somezpool -F -T Then check available filesystems. If it breaks again - retry with older transaction id. -- Sphinx of black quartz judge my vow. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
> At one point lz4 wasn't supported on boot, I seem to remember that may > have been addressed but not 100% sure. yes, its been addressed and works fine. Note that these machines booted fine befroe and I havent chnaged the OS, simply ran a lot of Apache CGI scripts to force it out of swap, so its not a config issue. > Have you tried booting from a live cd and checking the status of the pool? Yes, instant panic on import unfortunately :-( ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
> When you say corrupt what do you mean, specifically what's the output > from zpool status? It doesnt get that far - these are the boot pools and it wont boot, one due to not finding the pool, the other due to panicing when trying to impport the pool. Attching the discs to another machine and trying to import the pools causes an instant panic. -pete. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
At one point lz4 wasn't supported on boot, I seem to remember that may have been addressed but not 100% sure. If it hasn't and your kernel is now compressed that may explain it? Have you tried booting from a live cd and checking the status of the pool? On 22/11/2016 08:43, Pete French wrote: zpool import -N -O readonly=on -f -R /mnt/somezpoool If that doesn't help try: zpool import -N -O readonly=on -f -R /mnt/somezpoool -Fn I got someone to do this (am still having toruble finding time as am supposed to be off sick) and it causes instant kernel panic on trying to import the pool. Same as it does on boot. Drop us a line of your configuration and used ZFS features. Like dedup, snapshots, external l2 logs and caches. 10.3-STABLE r303832 from start of August. One simple pool, two drives mirrored, GPT formatted drives. No dedup, no snapshots, no external logs or caches. We have lz4 comression enabled on the filesystem, but apart from that its an utterly bog-standard setup. Am an leaning towards faulty hardware now actually... seems most likely... -pete. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
When you say corrupt what do you mean, specifically what's the output from zpool status? One thing that springs to mind if zpool status doesn't show any issues, and: 1. You have large disks 2. You have performed an update and not rebooted since. You may be at the scenario where there's enough data on the pool such that the kernel / loader are out range of the BIOS. All depends on exactly what you're seeing? On 21/11/2016 17:47, Pete French wrote: So, I am off sick and my colleagues decided to load test our set of five servers excesively. All ran out of swap. So far so irritating, but whats has happened is that twoof them now will not boot, as it appears the ZFS pool they are booting from has become corrupted. One starts to boot, then crases importing the root pool. The other doenst even get that far with gptzfsboot saying it can't find the pool to boot from! Now I can recover these, but I am a bit worried, that it got like this at all, as I havent ever seen ZFS corrupt a pool like this. Anyone got any insights, or suggstions as to how to stop it happening again ? We are swapping to a separate partition, not to the pool by theway. -pete. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
> When I gte the drives back online on a system I will check this though, thanks Just took a look and the partitions do not overlap. Thanks for the idea though... ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
> zpool import -N -O readonly=on -f -R /mnt/somezpoool > > If that doesn't help try: > > zpool import -N -O readonly=on -f -R /mnt/somezpoool -Fn I got someone to do this (am still having toruble finding time as am supposed to be off sick) and it causes instant kernel panic on trying to import the pool. Same as it does on boot. > Drop us a line of your configuration and used ZFS features. Like dedup, > snapshots, external l2 logs and caches. 10.3-STABLE r303832 from start of August. One simple pool, two drives mirrored, GPT formatted drives. No dedup, no snapshots, no external logs or caches. We have lz4 comression enabled on the filesystem, but apart from that its an utterly bog-standard setup. Am an leaning towards faulty hardware now actually... seems most likely... -pete. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
> How much trust do you put in your hardware? Have you ever put the > hardware under full load for extended periods before e.g. run poudriere > to build pkg repos? Ah, now that is a good point. There are two drves in each machine, mirrored, but one of each of the pair is 6 years old. We havent seen any errors, but that doesnt mean it wouldnt errpr under test. On the other hand I would epect only one drive to be corrupt in that case though, and booting from the otherdrive still shows the same problems. But that may be due to ZFS finding both drievs and trying to use them as a pair ? Am not sure how far into the boot sequence it starts using more tha just the drive it was booted from -pete. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Help! two machines ran out of swap and corrupted their zpools!
> Silly question - have you checked that the swap partition does not > overlap your boot pool partition? It could well be that the end of > the swap partition intrudes into the affected ZFS pool Interesting idea - all partitons were created with gpart add -a 8 but I have explictly checked, and I cant currently access the drives. Was there eer a bug in gpart which could cause this that you can remember ? The drives are GPT partitoned. When I gte the drives back online on a system I will check this though, thanks -pete. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"