[developer] Re: [openzfs/openzfs] Native data and metadata encryption for zfs (#124)

2017-08-11 Thread GernotS
still crashing when trying to zfs send/receive from an unencrypted to an encrypted dataset: panic[cpu1]/thread=ff000feecc40: assertion failed: spa_do_crypt_abd(B_TRUE, spa, zio->io_bookmark.zb_objset, bp, zio->io_txg, psize, zio->io_abd, eabd, iv, mac, salt, _crypt) == 0 (0x5 == 0x0), file:

[developer] Re: [openzfs/openzfs] Native data and metadata encryption for zfs (#124)

2017-08-19 Thread GernotS
exactly. But you might need more than 10GB data to transfer. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/124#issuecomment-323513670 --

[developer] Re: [openzfs/openzfs] Native data and metadata encryption for zfs (#124)

2017-08-18 Thread GernotS
I am sending a unencrypted zvol to an encrypted pool. Now the crash is different: panic[cpu0]/thread=ff000fb50c40: BAD TRAP: type=e (#pf Page fault) rp=ff000fb50540 addr=68 occurred in module "zfs" due to a NULL pointer dereference zpool-zones: #pf Page fault Bad kernel fault at

[developer] Re: [openzfs/openzfs] Native data and metadata encryption for zfs (#124)

2017-06-20 Thread GernotS
doing a zfs destroy on an encrypted dataset without loading the key causes a panic: panic[cpu0]/thread=ff00116d0c40: BAD TRAP: type=e (#pf Page fault) rp=ff00116cff50 addr=18 occurred in module "zfs" due to a NULL pointer dereference sched: #pf Page fault Bad kernel fault at addr=0x18

[developer] Re: [openzfs/openzfs] Native data and metadata encryption for zfs (#124)

2017-06-21 Thread GernotS
found another assert when doing a simple send/recv from an unencrypted to an encrypted dataset: zfs create -o canmount=noauto -o encryption=on -o keyformat=passphrase -o keylocation=prompt backup/test zfs send zones/testdata@1 |zfs recv backup/test/testdata

[developer] Re: [openzfs/openzfs] Native data and metadata encryption for zfs (#124)

2017-10-05 Thread GernotS
sorry to report that send/receive (see Aug. 18th/19th) still crashes when testing with more than 10GB of data: panic[cpu1]/thread=ff000faf4c40: BAD TRAP: type=e (#pf Page fault) rp=ff000faf4870 addr=68 occurred in module "zfs" due to a NULL pointer dereference sched: #pf Page fault Bad

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-08 Thread GernotS
Probably unrelated, but there seems to be some additional error handling in https://github.com/illumos/illumos-gate/commit/fa98e487a9619b7902f218663be219e787a57dad Can you please rebase against its once more? -- You are receiving this because you are subscribed to this thread. Reply to this

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-09 Thread GernotS
and I think https://github.com/openzfs/openzfs/pull/489/commits/31a00d3962866cf5c0f0c9755b4620a978fc312a is not what you intended to do? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-07 Thread GernotS
OK, this one get me another dump after a few hours: panic[cpu2]/thread=fe44aab2c020: BAD TRAP: type=e (#pf Page fault) rp=fe003d7615e0 addr=0 occurred in module "zfs" due to a NULL pointer dereference #pf Page fault Bad kernel fault at addr=0x0 pid=114332, pc=0xf7d90366,

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-08 Thread GernotS
Issue remains: panic[cpu1]/thread=fe2ce8fbd180: BAD TRAP: type=e (#pf Page fault) rp=fe003e0dd580 addr=2e occurred in module "zfs" due to a NULL pointer dereference qemu-system-x86_: #pf Page fault Bad kernel fault at addr=0x2e pid=51237, pc=0xf7e3ef45, sp=0xfe003e0dd670,

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-22 Thread GernotS
I still get the PF from above after some time. The issue is gone as soon as I remove the l2arc device. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-390955163

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-01 Thread GernotS
latest commit does not compile (at least for me): Makefile:294: recipe for target '0-illumos-stamp' failed gmake: *** [0-illumos-stamp] Error 1 + /root/smartos-live/proto.strap/usr/bin/gcc -fident -finline -fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option -nodefaultlibs

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-02 Thread GernotS
sorry, another one: Makefile:294: recipe for target '0-illumos-stamp' failed gmake: *** [0-illumos-stamp] Error 1 [root@526d5932-f213-cc59-9414-c8d239762b3a ~/smartos-live]# vi /root/smartos-live/projects/illumos/log/latest/nightly.log + /root/smartos-live/proto.strap/usr/bin/gcc -fident

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-05-02 Thread GernotS
this one gets me an assert on booting, screenshot is at https://imgur.com/aXvuamR -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-385907991

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-12 Thread GernotS
I have now litterally written hundreds of GB using dd and zfs send with enabled kmem_flags, but i was unable to reproduce the crash, I will keep on trying. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-14 Thread GernotS
Got a dump with kmem_flags set, but I am not sure if this is the same issue, as I applied the patch to a more recent build. panic[cpu0]/thread=ff000f5e7c40: BAD TRAP: type=e (#pf Page fault) rp=ff000f5e7790 addr=0 occurred in module "zfs" due to a NULL pointer dereference zpool-zones:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-09 Thread GernotS
Dump is here: https://www.magentacloud.de/lnk/FWLB1qmb -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-364382103 --

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-06 Thread GernotS
This change did not cure my issue yet. I managed to write 15GB using dd until it crashed again. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-363355614

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-07 Thread GernotS
I´m sorry to report the latest commit didnt fix my problem yet, but it indeed cured a long open issue with compressed send/receive, see also https://www.illumos.org/issues/7252 This is now the first time I was able to use zfs send -c sucessfully for more than just a few GB of data. Thanks for

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-07 Thread GernotS
Albeit my current dump looks a bit different: panic[cpu1]/thread=ff000fa9cc40: BAD TRAP: type=e (#pf Page fault) rp=ff000fa9c870 addr=68 occurred in module "zfs" due to a NULL pointer dereference sched: #pf Page fault Bad kernel fault at addr=0x68 pid=0, pc=0xf7d4b7e6,

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-22 Thread GernotS
Bad news, issue remains: BAD TRAP: type=e (#pf Page fault) rp=ff000fb16970 addr=2e occurred in module "zfs" due to a NULL pointer dereference sched: #pf Page fault Bad kernel fault at addr=0x2e pid=0, pc=0xf7d4a1a5, sp=0xff000fb16a60, eflags=0x10246 cr0:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-02-22 Thread GernotS
removing the l2arc device indeed gets me a step further, I can no longer produce crashes using send/receive. But now I can reproduce a different error when trying to access a zvol (booting a kvm machine): panic[cpu3]/thread=ff003d40ec40: Invalid dnode block MAC ff003d40e370

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-08-16 Thread GernotS
yeah its me again, with a new bug. Trying to do an ls -l /dev/zvol/rdsk/pool/disk0 on an encrypted zvol (key loaded) will produce a core: panic[cpu3]/thread=fe003e350c20: assertion failed: spa_do_crypt_objset_mac_abd(B_TRUE, spa, dsobj, zio->io_abd, psize, (bp)->blk_prop) >> (63)) &

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-07-24 Thread GernotS
so far no further crashes on my side :) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-407324688 -- openzfs:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-07-24 Thread GernotS
@Ahrens glad to help a bit getting this forward, but the credit belongs to you, Tom, Jorgen and all the others who keep on pushing on with it. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-07-06 Thread GernotS
another pull request: https://github.com/zfsonlinux/zfs/pull/7650 thanks -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-403027939

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-07-04 Thread GernotS
@lundman would you mind integrating https://github.com/zfsonlinux/zfs/pull/7667 and https://github.com/zfsonlinux/zfs/pull/7637 ? I belive https://github.com/zfsonlinux/zfs/pull/7632 had some changes after integrating here as well. thanks! -- You are receiving this because you are subscribed

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-06 Thread GernotS
Latest build still gives me the page fault from above. Without using a cache device I get "Invalid dnode block MAC" when doing zfs send -c -I from an unencrypted to an encrypted dataset. Core for this case is at https://www.magentacloud.de/lnk/3MrBVdQy A full send does work. -- You are

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-04-10 Thread GernotS
Problem with 097d997: zfs_main.c: In function 'share_mount_one': zfs_main.c:6072: error: 'ZIO_CRYPT_OFF' undeclared (first use in this function) zfs_main.c:6072: error: (Each undeclared identifier is reported only once zfs_main.c:6072: error: for each function it appears in.) -- You are

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-04-18 Thread GernotS
I have a build failure, probably related to latest Illumos commits: ../../../uts/common/fs/zfs/vdev_indirect.c: In function 'vdev_indirect_checksum_error': ../../../uts/common/fs/zfs/vdev_indirect.c:1356: error: passing argument 3 of 'zfs_ereport_post_checksum' from incompatible pointer type

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-04-18 Thread GernotS
this is the commit: https://github.com/illumos/illumos-gate/commit/3a4b1be953ee5601bab748afa07c26ed4996cde6#diff-c6fefc71095424603bca3dd5d115010b -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-04-18 Thread GernotS
yea, sorry, that is probably the root cause :) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-382299920 -- openzfs:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-26 Thread GernotS
@ahrens will provide anything needed to nail this down. I sure have a huge interest in getting this merged. I will start with reproducing the L2ARC issue with kmem_flags set, as the second issue puzzles me as well. I can reproduce this currently only on my 700GB dataset, there migt be something

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-27 Thread GernotS
Dump for the assert in sa.c https://www.magentacloud.de/lnk/4xLBV469 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-376474714

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-25 Thread GernotS
I found another issue, which happens when trying to access a encrypted dataset (no zvol). panic[cpu0]/thread=ff03cf882b20: assertion failed: BSWAP_32(sa_hdr_phys->sa_magic) == SA_MAGIC, file: ../../common/fs/zfs/sa.c, line: 1286 ff000fb4a1e0 fba993b8 () ff000fb4a230

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-29 Thread GernotS
things up a bit? Von: Matthew Ahrens [mailto:notificati...@github.com] Gesendet: Donnerstag, 29. März 2018 17:54 An: openzfs/openzfs <open...@noreply.github.com> Cc: GernotS <gernot.stras...@freenet.de>; Mention <ment...@noreply.github.com> Betreff: Re: [openzfs/openzfs

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-31 Thread GernotS
While I cannot reproduce l2arc panic anymore, I had another (new) one reagding zvols: panic[cpu3]/thread=ff003fe9ec40: blkptr at ff0dbeb82200 has invalid TYPE 218 ff003fe9e330 genunix:vcmn_err+42 () ff003fe9e3a0 zfs:zfs_panic_recover+51 () ff003fe9e400

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-27 Thread GernotS
I have now moved around a few TBs using raw send, compressed send, etc with L2ARC devices, but I cannot reproduce the issue any more. Is it possible kmem_flags hides this? Will try to get a debug build done. -- You are receiving this because you are subscribed to this thread. Reply to this

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-28 Thread GernotS
another dump for the assert in sa.c, hopefully with debug bits now https://www.magentacloud.de/lnk/I0rB1RA1 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-376915757

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-29 Thread GernotS
@Ahrens ls -l /zones/data/fritz gives me /zones/data/fritz: Bad exchange descriptor trying other directories gives me the known crash -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-21 Thread GernotS
Hello Matt, the dump is at https://www.magentacloud.de/lnk/snrB1xiI -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/489#issuecomment-374850679

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-22 Thread GernotS
@ahrens yes. You will need an l2arc device to reproduce the null pointer reference, without that you will get the HMAC issue. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-03-24 Thread GernotS
@ahrens thx for fixing the MAC issue! I can still reproduce the PF when using an L2ARC device: panic[cpu0]/thread=ff000f793c40: BAD TRAP: type=e (#pf Page fault) rp=ff000f793680 addr=ff00f1c98000 zpool-zones: #pf Page fault Bad kernel fault at addr=0xff00f1c98000 pid=89,

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

2018-10-07 Thread GernotS
Was recounted DSL meant to fix my above issue with unencrypted send/receive? If so it was not successful, I still get: BAD TRAP: type=e (#pf Page fault) rp=fe003dd7dee0 addr=60 occurred in module "zfs" due to a NULL pointer dereference zfs: #pf Page fault Bad kernel fault at addr=0x60