[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Brian Behlendorf]

One encryption related fix to add to the this PR, 
zfsonlinux/zfs@d7e4b30a672c1d7eb77dbbe2cdcf14cac102839a.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-428354045
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M539bd7293d67b712a0fb2fcc
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS Unfortunately no, that wasn't intended to fix your issue.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-427883495
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mc1deec26a3faf2e93f9d8624
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

Was recounted DSL meant to fix my above issue with unencrypted send/receive?
If so it was not successful, I still get:

BAD TRAP: type=e (#pf Page fault) rp=fe003dd7dee0 addr=60 occurred in 
module "zfs" due to a NULL pointer dereference


zfs:
#pf Page fault
Bad kernel fault at addr=0x60
pid=76825, pc=0xf7e86b91, sp=0xfe003dd7dfd0, eflags=0x10283
cr0: 80050033  cr4: 
1626f8
cr2: 60
cr3: 7b8107000
cr8: 0

rdi:0 rsi:0 rdx:d
rcx:2  r8: 9ae16a3b2f90408f  r9:0
rax:0 rbx:0 rbp: fe003dd7e050
r10: fb871d1c r11: fe2cef7c5860 r12:d
r13:0 r14:0 r15:0
fsb:0 gsb: fe2ce8928000  ds:   4b
 es:   4b  fs:0  gs:  1c3
trp:e err:0 rip: f7e86b91
 cs:   30 rfl:10283 rsp: fe003dd7dfd0
 ss:0

fe003dd7ddd0 unix:real_mode_stop_cpu_stage2_end+bcbc ()
fe003dd7ded0 unix:trap+e60 ()
fe003dd7dee0 unix:cmntrap+e6 ()
fe003dd7e050 zfs:dbuf_read+31 ()
fe003dd7e0f0 zfs:dbuf_prefetch_indirect_done+18e ()
fe003dd7e210 zfs:arc_read+b89 ()
fe003dd7e2b0 zfs:dbuf_prefetch_indirect_done+104 ()
fe003dd7e3d0 zfs:arc_read+b89 ()
fe003dd7e560 zfs:dbuf_prefetch+386 ()
fe003dd7e5f0 zfs:dmu_prefetch+198 ()
fe003dd7e640 zfs:receive_read_prefetch+5d ()
fe003dd7e710 zfs:receive_read_record+1eb ()
fe003dd7e980 zfs:dmu_recv_stream+497 ()
fe003dd7ebd0 zfs:zfs_ioc_recv+20f ()
fe003dd7ec70 zfs:zfsdev_ioctl+4ef ()
fe003dd7ecb0 genunix:cdev_ioctl+39 ()
fe003dd7ed00 specfs:spec_ioctl+60 ()
fe003dd7ed90 genunix:fop_ioctl+55 ()
fe003dd7eeb0 genunix:ioctl+9b ()
fe003dd7ef10 unix:brand_sys_sysenter+1d3 ()


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-427726957
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Ma8d89a4ea994d6dc26161fe8
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Refcounted DSL added - one missing `zvol_create_minors()` call in illumos, but 
I believe that is as expected. Added a couple of `(void)` before 
`refcount_add()` due to lint.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-426853100
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mb0c8da330d638829d5df84a8
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

bc64070  Refcounted DSL Crypto Key Mappings


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/e2df9b70741ea3c091a30c03c08847c7bacec46e..bc640707c92028b91bfe86437929b63fe484ea9f

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M4df13c9ac220200a8378a767
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@lundman FYI, I added 3 commits to fix a few more issues.  There were some 
changes on linux that were not directly related to encryption but which the 
encryption code depends on.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-417078930
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M5b41f703067a8a25e36e94bc
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 3 commits.

3de57cc  fix error handling in arc_read
5548355  spill blocks are metadata
e2df9b7  zfs_receive_one needs to restore keylocation prop


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/43b4a66f848e55526fc90560c6ad4e4ceccc4a01..e2df9b70741ea3c091a30c03c08847c7bacec46e

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M1dd14e934377216b21b123b4
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS Thanks for your report.  I was not able to reproduce this with a 
simple test.  Could you upload/send me the crash dump?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-417078260
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Maa09f3b0944d282e0cf582bd
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

yeah its me again, with a new bug. 
Trying to do an ls -l /dev/zvol/rdsk/pool/disk0 on an encrypted zvol (key 
loaded) will produce a core:

panic[cpu3]/thread=fe003e350c20:
assertion failed: spa_do_crypt_objset_mac_abd(B_TRUE, spa, dsobj, zio->io_abd, 
psize, (bp)->blk_prop) >> (63)) & ((1ULL << (1)) - 1)) != (1ULL))) == 0 
(0x2 == 0x0), file: ../../common/fs/zfs/zio.c, line: 3670


fe003e350920 genunix:strlog+0 ()
fe003e350a20 zfs:zio_encrypt+4aa ()
fe003e350a50 zfs:zio_execute+7f ()
fe003e350b10 genunix:taskq_thread+2d0 ()
fe003e350b20 unix:thread_start+8 ()


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-413453704
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mced17069cdb54dc67097aceb
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

3972768  Do not call  dmu_objset_disown twice


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/6f85552ff7a285fef0af14f31d322cc82bebd07c..3972768dc18f2f6b4b2c3c90ac9ce516f8e1cf99

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M0a87816ff166582825149505
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

@Ahrens glad to help a bit getting this forward, but the credit belongs to you, 
Tom, Jorgen and all the others who keep on pushing on with it.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-407524549
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M2421794460c2234c9e0e28fb
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Sean Eric Fagan]

mav pointed out something to me:  dmu_objset_disown() is called in both 
zfsvfs_create() and zfsvfs_create_impl() in case of error.  At the end of 
zfsvfs_create(), remove the if after

`error = zfsvfs_create_impl(zfvp, zfsvfs, os);`


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-407517148
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Me0f9230f1dd5f81484039ef9
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS That's great to hear!  It makes sense that the "Refactor 
arc_hdr_realloc_crypt()" commit would address the panics that you were seeing 
when using L2ARC + encryption.  I think that the other bug (causing an 
incorrect MAC on the target when doing encrypted send, causing some panics due 
to bad error handling) could still be out there.  Let us know if you hit that 
or any other panics / bugs.  Thanks again for your help with testing encryption 
on illumos!

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-407493701
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M429fae88311bf11cf469390b
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Woo, I contributed! \o/


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-407324973
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mdf13355aa69796a1a328160c
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

so far no further crashes on my side :)


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-407324688
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M5823fa312cd12959ac3037e0
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

In the interest in making `dsl_crypto.c` and `zio_crypto.c` to be the same as 
ZOL - I added a few commits that messed around in there. [1] The 3a5bf20 commit 
will replace the gist I pasted in 
https://github.com/openzfs/openzfs/pull/489#issuecomment-404715592 

[1] minus the QAT changes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-404990344
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M625ae3339547adaf794cdf34
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@GernotS Working on a theory to a couple of bugs over in OsX port, which might 
just help here as well. If you have the time/energy, could you see if this 
patch fixes things with "cache + crypto" and "raw receive" ?

https://gist.github.com/lundman/5c269ae5582bac1adf2a58e7b9513c1b



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-404715592
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-Mc2df76b7b1eb40522db69276
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@lundman Yes, we'd need to get `zfs recv -o/-x` in illumos in general before 
the encryption-specific `-o/-x` support.  I think we would be best to leave 
both for a separate PR.  Thanks for staying on top of the linux encryption 
work, though!

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-403528606
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M9f6bf274a87fed5beb5d640c
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Tom Caputi]

I wouldn't worry about the encryption related `zfs recv -o / -x` changes until 
the PR has been merged into ZoL. It's passing tests right now, but I still need 
to add some documentation / man pages to it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-403521773
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-Me1afd97ee3658a3a688d5cf7
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Hmm alas, commit 
https://github.com/zfsonlinux/zfs/pull/7650/commits/5248401f8bf3d8149ba6e559fa94a075bfdff9d4
 depends on 
https://github.com/zfsonlinux/zfs/commit/a3eeab2de68670a4481eab3d086982aff23b6906
 and we have a fair bit of divergence in the property parsing between the two. 
I'm just here to confirm that we want `-o / -x` included in this PR ?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-403349020
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M351ae6ca1a92ddfb347aee2b
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

another pull request: https://github.com/zfsonlinux/zfs/pull/7650

thanks

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-403027939
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M6076b0855958e879452ad1dd
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

58e8434  Odd git merge decisions


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/d123482538a05e9ed8dcea8f2872ee72a52b0a3f..58e8434463cc1ffd99b4fedd8995e11ea3c4af7c

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M537d909e225f59afcdd6711b
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

d123482  Correct typo in zfs.1m


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/ab2d8ac94cf227207f80ca9b221ef7bd3bcd6e99..d123482538a05e9ed8dcea8f2872ee72a52b0a3f

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M641da47a73de997cc132671a
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Dropped and re-added 7632, added 7667 and 7637.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-402581297
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M54d5b8133ce9fedbcbf9509b
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 3 commits.

23eb923  Raw receive fix and encrypted objset security fix
58dd6e4  Add ASSERT to debug encryption key mapping issues
ab2d8ac  Fix coverity defects: CID 176037


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/d9fa7a3b584f33433e414fff54b426e879afd091..ab2d8ac94cf227207f80ca9b221ef7bd3bcd6e99

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M9d6712a0b379e9e6c55dc5bd
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[gdamore]

gdamore commented on this pull request.



>  .It Sy filesystem_count
 The total number of filesystems and volumes that exist under this location in
 the dataset tree.
 This value is only available when a
 .Sy filesystem_limit
 has been set somewhere in the tree under which the dataset resides.
+.It Sy keystatus
+Indicates if an encryption key is currently loaded into ZFS.
+The possible values are
+.Sy none , available ,
+and
+.Sy unavaliable .

Typo here.  Just noticed it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#pullrequestreview-134454116
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-Mcb0258b99d9ab24e93bae6c7
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

@lundman would you mind integrating https://github.com/zfsonlinux/zfs/pull/7667 
and https://github.com/zfsonlinux/zfs/pull/7637 ?
I belive https://github.com/zfsonlinux/zfs/pull/7632 had some changes after 
integrating here as well.
thanks!

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-402521948
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/T02d6a9917e14b170-M93257070757dba62c151c2d1
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[marcheschi]

Hi, is there a way to try the work done on Smartos ?
Thank you
Paolo

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-399044477
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mf8700b92c5065aa4c3247be2
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

d9fa7a3  [PATCH] Raw receive fix and encrypted objset security fix


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/3fbbbf14b40989ba17b65966d77cb27abff100c3..d9fa7a3b584f33433e414fff54b426e879afd091

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mb64935d1b6f0e819ac8574ec
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

02dceca  [PATCH] Make zvol update volsize operation synchronous.


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/ed11924afad60b692dee0cf34689e186533900a5..02dceca32e57399589f59d4f9c0fe13515ba3019

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M58a490ca71eb153e8c164244
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@lundman @GernotS I added 3 commits which should gather more debugging info.  
@GernotS if you can run your workload with the latest bits, hopefully we will 
get more info to diagnose the issues you've been seeing.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-394165036
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mcb51f26af55b616a0fe2c514
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

d95dfde  Remove test not yet ported to illumos


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/473f80e35fb25e0778ab1678fa0f4fa7ed3a48c0..d95dfdee9d978010290cfde087b25f4e0da4f56d

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-M170b301ee4d0e2c2d47f2397
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 2 commits.

5da0a65  Raw receive functions must not decrypt data
473f80e  Update the correct abd in l2arc_read_done()


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/9c0a780d61739bd28e15c5d564df0dbaaff284d6..473f80e35fb25e0778ab1678fa0f4fa7ed3a48c0

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Med99b280071328e36c880448
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 1 commit.

9c0a780  mismerge in ztest_freeze()


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/7d80e85eb7d645666127053da17be18e17f4..9c0a780d61739bd28e15c5d564df0dbaaff284d6

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/Te9a3d86d2367f775-Mf09875af4e788af2c69516b6
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Ah, so that is the last outstanding issue? Do you want anything squashed? 
Squashed-to-some, or Squashed-to-one ?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-391905514
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-Mdbab0c9b973079f39c06d18f
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

The above problem (*.37) was determined to be a mismerge between @GernotS's 
version of illumos and the code in this repo.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-391883187
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M53dc3dea005ef285c88486b8
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS I got the dump from your latest panic (*.37) and I'm still not sure 
what's going on there.  I am running some stress tests with encryption and 
l2arc and haven't hit a panic yet.  Is there any chance you can reproduce with 
debug bits?  Even just debug zfs kernel module would be helpful since then 
assertions would be enabled.

We are evicting a hdr, and it points to a buf that has been freed.  The hdr is 
for a block of dnodes from the MOS (it's not encrypted/protected).  The hdr is 
in the l2arc (`HAS_L2HDR`).  The only place that the buf is freed is 
`arc_buf_destroy_impl()`.   The `hash_lock` protects the hdr from changing 
while we remove the buf from the linked list and free it.

The PREDICTIVE_PREFETCH flag is set.  Therefore the hdr was initially read due 
to a predictive prefetch (dmu_zfetch), and has not had a demand (non-prefetch) 
read since.  But the state is MFU, which implies that it's been accessed 
multiple times.

HAS_L2HDR is set, but L2CACHE is not.  Therefore we went through arc_read_done()
 with the PREFETCH flag set, after it was written to L2ARC.

if (l2arc_noprefetch && HDR_PREFETCH(hdr))
arc_hdr_clear_flags(hdr, ARC_FLAG_L2CACHE);

This doesn't make sense to me - if it was only read via a single prefetch read, 
when that read completed, we would clear L2CACHE, so it would never have an 
opportunity to be written to L2ARC.

```
> ::stack
arc_buf_remove+0x16(0, fe2d3fdfd0b0)
arc_buf_destroy_impl+0x94(fe2d3fdfd0b0)
arc_evict_hdr+0xb3(fe30c7fd8bb0, c00347d0)
arc_evict_state_impl+0x154(fe2ce930b468, 2, fe320c44a2e0, 0, 53b800)
arc_evict_state+0x12e(c0033120, 0, 581800, 2)
arc_adjust_impl+0x44(c0033120, 0, 2a05ecaa, 2)
arc_adjust+0x10b()
arc_adjust_cb+0x19(0, fe2ce9403e40)
zthr_procedure+0xba(fe2ce9403e40)
thread_start+8()

> fe2d3fdfd0b0::whatis
fe2d3fdfd0b0 is freed from arc_buf_t

> fe30c7fd8bb0::print arc_buf_hdr_t
{
b_dva = {
dva_word = [ 0x10, 0x3814e9ab0 ]
}
b_birth = 0x170ab9d
b_type = 2 (ARC_BUFC_METADATA)
b_hash_next = 0
b_flags = 0xf0f0464 
(ARC_FLAG_{PREFETCH|PREDICTIVE_PREFETCH|IN_HASH_TABLE|PRIO_ASYNC_READ|BUFC_METADATA|HAS_L1HDR|HAS_L2HDR|COMPRESSED_ARC|COMPRESS_0|COMPRESS_1|COMPRESS_2|
COMPRESS_3})
b_psize = 0x8
b_lsize = 0x20
b_spa = 0x1b440bfb53f69336
b_l2hdr = {
b_dev = 0xfe2ce91f0cd0
b_daddr = 0x91237c000
b_l2node = {
list_next = 0xfe30c8177160
list_prev = 0xfe30d506ad38
}
}
b_l1hdr = {
b_freeze_lock = {
_opaque = [ 0 ]
}
b_freeze_cksum = 0
b_buf = 0xfe2d3fdfd0b0
b_bufcnt = 0x1
b_cv = {
_opaque = 0
}
b_byteswap = 0xa
b_state = ARC_mfu
b_arc_node = {
list_next = 0xfe2ce9226698
list_prev = 0xfe320c44a360
}
b_arc_access = 0x1b586a4f
b_refcnt = {
rc_count = 0
}
b_acb = 0
b_pabd = 0xfe30d8ef6c98
}

> fe2cf19f1000+3380::blkptr
DVA[0]=<0:7029d356000:2000>  [ 0x10, 0x3814e9ab0 ]
DVA[1]=<0:36b82d72000:2000>
DVA[2]=<0:2fae7d96000:2000>
[L0 DNODE] FLETCHER_4 LZ4 LE contiguous unique triple
size=4000L/1000P birth=24161181L/24161181P [0x170ab9d] fill=10
cksum=47c40d0691:f6e8203aa596:1ad111bee454314:f60708759a96e348

> fe2cf1be35c0::print dmu_buf_impl_t db.db_data
db.db_data = 0xfe2cf19f1000

> fe2cf1be35c0::print dmu_buf_impl_t db_objset->os_dsl_dataset
db_objset->os_dsl_dataset = 0
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-391178074
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-Mda4854a01748f41aeb130659
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

I added a commit to fix a mismerge in ztest_freeze().

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-391141740
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M641895910edf1249531f78e4
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 1 commit.

56bbddd  mismerge in ztest_freeze()


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/d1dab40ab051d5f52296e064fd3c4b990f65b37c..56bbddd350db21002218dab709bdfea65a53a388

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M6869be0622d401717c8ec68e
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS Can you share a dump from the page fault you're hitting?  I looked 
through the dumps I have from you and I don't think I have one with that stack.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-391064732
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M444c6142570e13b4217b8d0c
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

I still get the PF from above after some time. The issue is gone as soon as I 
remove  the l2arc device.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-390955163
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M0c861be0a1ce6151f8744dcd
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Igor K]

@lundman i hope it can be integrated before next year anniversary :)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-389523909
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M0e0f148bbba006e0b4caf23f
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

latest rebase was non-trivial due to the parallel mount commit. I had to guess 
a bit. But good news, we are coming up to 2 year anniversary! \o/


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-389409212
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M7032b2a51e6e3378e083f5bb
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

1fa5148  Add support for decryption faults in zinject


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/31a00d3962866cf5c0f0c9755b4620a978fc312a..1fa5148fe59c8d1a7347fe5d1d5f5f4362a70f4c

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/Te9a3d86d2367f775-M7850018c280efc6765e82605
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

and I think 
https://github.com/openzfs/openzfs/pull/489/commits/31a00d3962866cf5c0f0c9755b4620a978fc312a
 is not what you intended to do?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-387708627
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-M989705c364c685fa6855d792
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

Probably unrelated, but there seems to be some additional error handling in  
https://github.com/illumos/illumos-gate/commit/fa98e487a9619b7902f218663be219e787a57dad
Can you please rebase against its once more?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-387623218
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-Mb878b198dc009736c34bedf5
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Not sure about that one, could be the new commits in master triggering 
something new, is probably beyond my abilities.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-387619163
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-M2461c53a45cbfceb265bdabc
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

Issue remains:

panic[cpu1]/thread=fe2ce8fbd180:
BAD TRAP: type=e (#pf Page fault) rp=fe003e0dd580 addr=2e occurred in 
module "zfs" due to a NULL pointer dereference


qemu-system-x86_:
#pf Page fault
Bad kernel fault at addr=0x2e
pid=51237, pc=0xf7e3ef45, sp=0xfe003e0dd670, eflags=0x10246
cr0: 8005003b  cr4: 
1626f8
cr2: 2e
cr3: 7b81ff000
cr8: 0

rdi: fe2cff2a3de8 rsi:1 rdx: fe2ce8fbd180
rcx:1  r8: 9ae16a3b2f90408f  r9: 9ae16a3b2f90404f
rax:0 rbx: fe2cff2a3de8 rbp: fe003e0dd680
r10: fe2da9713630 r11:0 r12: c0034140
r13:0 r14: c00341a0 r15:1
fsb: fc7fef064240 gsb: fe2ce860c000  ds:   4b
 es:   4b  fs:0  gs:0
trp:e err:0 rip: f7e3ef45
 cs:   30 rfl:10246 rsp: fe003e0dd670
 ss:   38

fe003e0dd470 unix:real_mode_stop_cpu_stage2_end+bc1c ()
fe003e0dd570 unix:trap+108e ()
fe003e0dd580 unix:_cmntrap+e6 ()
fe003e0dd680 zfs:arc_buf_size+15 ()
fe003e0dd6c0 zfs:arc_evictable_space_increment+a8 ()
fe003e0dd730 zfs:arc_change_state+1fd ()
fe003e0dd770 zfs:arc_access+16e ()
fe003e0dd890 zfs:arc_read+1f4 ()
fe003e0dd950 zfs:dbuf_read_impl+4d0 ()
fe003e0dd9e0 zfs:dbuf_read+ea ()
fe003e0dda90 zfs:dmu_buf_hold_array_by_dnode+128 ()
fe003e0ddb30 zfs:dmu_read_uio_dnode+5a ()
fe003e0ddb90 zfs:dmu_read_uio+5b ()
fe003e0ddc20 zfs:zvol_read+172 ()
fe003e0ddc50 genunix:cdev_read+2d ()
fe003e0ddcf0 specfs:spec_read+2b9 ()
fe003e0ddd90 genunix:fop_read+f3 ()
fe003e0ddf00 genunix:pread+1a3 ()
fe003e0ddf10 unix:brand_sys_syscall+21a ()



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-387488269
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-M0d8e467e2379b52a53c79fe0
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

dad661b  minor dbuf_read_done work


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/7743d9912c456bce8cc6085031ef7080c8c848a3..dad661b040d6e0d69089efc0513a450d004d61ca

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-M0df2fcae6b84d7bad4be56a9
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Not sure about that, but in that place, ZOL has the following
```
ASSERT(db->db.db_data == NULL);
if (db->db_level == 0 && db->db_freed_in_flight) {
/* we were freed in flight; disregard any error */
+if (buf == NULL) {
+buf = arc_alloc_buf(db->db_objset->os_spa,
+db, DBUF_GET_BUFC_TYPE(db), db->db.db_size);
+}
arc_release(buf, db);
```

Not sure I get why they do that, but I suppose we could try it and see if it 
helps.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-387247635
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-M70f1942bd65a966af2a05336
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

OK, this one get me another dump after a few hours:

panic[cpu2]/thread=fe44aab2c020:
BAD TRAP: type=e (#pf Page fault) rp=fe003d7615e0 addr=0 occurred in module 
"zfs" due to a NULL pointer dereference

#pf Page fault
Bad kernel fault at addr=0x0
pid=114332, pc=0xf7d90366, sp=0xfe003d7616d0, eflags=0x10282
cr0: 8005003b  cr4: 
1626f8
cr2: 0
cr3: 7615fb000
cr8: 0

rdi:0 rsi: fe2e33f7b808 rdx: fe44aab2c020
rcx: fe2e33f7b808  r8: 9ae16a3b2f90408f  r9: 9ae16a3b2f90404f
rax:2 rbx: fe2e33f7b808 rbp: fe003d761700
r10: fb87bd58 r11:1 r12:0
r13:5 r14: fe2e33f7b808 r15:   80
fsb: fc7fef053a40 gsb: fe2d373d7000  ds:   4b
 es:   4b  fs:0  gs:0
trp:e err:0 rip: f7d90366
 cs:   30 rfl:10282 rsp: fe003d7616d0
 ss:   38

CPU  ADDRESSTIMESTAMP TYPE  VC HANDLER  PC
  0 fbc2f488 44460127bfda sysc  36 ioctlfc7fef29876a
  0 fbc2f310 44460127610b sysc  36 ioctlfc7fef29876a
  0 fbc2f198 44460126c839 sysc  36 ioctlfc7fef29876a
  0 fbc2f020 4446012673f6 sysc  36 ioctlfc7fef29876a
  0 fbc2eea8 444601263a02 sysc  4d syslwp_park  fc7fef291c84
  0 fbc2ed30 44460124ce05 sysc  36 ioctlfc7fef29876a
  0 fbc2ebb8 4446012457f0 sysc  36 ioctlfc7fef29876a
  0 fbc2ea40 4446012048da sysc  36 ioctlfc7fef29876a
  0 fbc3a4c8 444601164582 sysc  36 ioctlfc7fef29876a
  0 fbc3a350 4446011607f6 sysc  b7 pollsys  fc7fef298a0a
  1 fe2d373bde28 444601294139 intr  f1 xc_serv  44e737
  1 fe2d373bdcb0 4446010ad8cb intr  f2 unknown  44e702
  1 fe2d373bdb38 444600ca0ab6 sysc  18 getuid   7fffed8ea827
  1 fe2d373bd9c0 444600adf294 intr  21 e1000g_intr_pcie 44e763
  1 fe2d373bd848 444600a9b097 intr  21 e1000g_intr_pcie 44e72e
  1 fe2d373bd6d0 4446007efaa0 intr  f1 xc_serv  44e6fd
  1 fe2d373bd558 4446007e2739 intr  f1 xc_serv  44e779
  1 fe2d373bd3e0 4446007dd460 intr  f1 xc_serv  44e702
  1 fe2d373bd268 4446007d6fce intr  f1 xc_serv  44e702
  1 fe2d373bd0f0 44460076754e intr  f1 xc_serv  44e702
  2 fe2d3740e0c8 44460129c7c8 trap   e #pf  arc_buf_destroy+26
  2 fe2d3740df50 44460126e77c sysc  ad preadfc7fef298a2a
  2 fe2d3740ddd8 444601266c33 intr  f2 unknown  4422dc
  2 fe2d3740dc60 4446010adc73 intr  f2 unknown  44e798
  2 fe2d3740dae8 444600814f85 sysc  4d syslwp_park  fc7fef291c57
  2 fe2d3740d970 44460081149a sysc  25 kill fc7fef2987aa
  2 fe2d3740d7f8 44460032a00b sysc  7b preadv   fc7fef298a4a
  2 fe2d3740d680 444600324b5d intr  f2 unknown  44e702
  2 fe2d3740d508 4446001b8a33 sysc  4d syslwp_park  fc7fef291c57
  2 fe2d3740d390 4446001b4cfc sysc  25 kill fc7fef2987aa
  3 fe2d37489ff0 44460129410e intr  f1 xc_serv  44e702
  3 fe2d37489e78 4446010a7842 intr  f6 cbe_fire 44e710
  3 fe2d37489d00 4446007efccb intr  f1 xc_serv  44e763
  3 fe2d37489b88 4446007e2936 intr  f1 xc_serv  44e748
  3 fe2d37489a10 4446007dd679 intr  f1 xc_serv  44e702
  3 fe2d37489898 4446007d71c8 intr  f1 xc_serv  44e763
  3 fe2d37489720 44460076775a intr  f1 xc_serv  44e763
  3 fe2d374895a8 44460075d6ff intr  f1 xc_serv  44e763
  3 fe2d37489430 44460075a31a intr  f1 xc_serv  44e798
  3 fe2d374892b8 44460075641d intr  f1 xc_serv  44e737

fe003d7614c0 unix:real_mode_stop_cpu_stage2_end+c854 ()
fe003d7615d0 unix:trap+148b ()
fe003d7615e0 unix:cmntrap+1ca ()
fe003d761700 zfs:arc_buf_destroy+26 ()
fe003d761750 zfs:dbuf_read_done+f8 ()
fe003d761870 zfs:arc_read+11ba ()
fe003d761940 zfs:dbuf_read_impl+6fe ()
fe003d7619d0 zfs:dbuf_read+fa ()
fe003d761a80 zfs:dmu_buf_hold_array_by_dnode+138 ()
fe003d761b20 zfs:dmu_read_uio_dnode+5a ()
fe003d761b80 zfs:dmu_read_uio+5b ()
fe003d761c10 zfs:zvol_read+172 ()
fe003d761c40 genunix:cdev_read+2d ()
fe003d761ce0 specfs:spec_read+263 ()
fe003d761d80 genunix:fop_read+f3 ()
fe003d761ef0 genunix:pread+1a2 ()
fe003d761f00 unix:brand_sys_syscall+320 ()


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-387130454

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

7743d99  DMU objset should not be encrypted


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/099741e7ff87c29c3876debac50acbc99ce0ab26..7743d9912c456bce8cc6085031ef7080c8c848a3

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T3a42c8a97e84a347-Mabb1114875eda855014043d3
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Hmm odd it wasnt too big a conflict in `arc.c` but I'll go over it again


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385909026
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M12c0cf94b69070154156f3c7
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

this one gets me an assert on booting, screenshot is at 
https://imgur.com/aXvuamR


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385907991
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M4332d51ba5cdabcf5f31da7b
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

099741e  Correct ztest build


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/77c699a32931847d5557f0ebadcec07a680504b1..099741e7ff87c29c3876debac50acbc99ce0ab26

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M2323f3e40b12e233938ee459
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Nah it's my fault, my VM is super slow, so its faster to rely on this build 
setup :)



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385876826
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Meae52625dbe9d1fb8789df02
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

sorry, another one:

Makefile:294: recipe for target '0-illumos-stamp' failed
gmake: *** [0-illumos-stamp] Error 1
[root@526d5932-f213-cc59-9414-c8d239762b3a ~/smartos-live]# vi 
/root/smartos-live/projects/illumos/log/latest/nightly.log
+ /root/smartos-live/proto.strap/usr/bin/gcc -fident -finline 
-fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option 
-nodefaultlibs -D__sun -O -fno-inline-small-functions 
-fno-inline-functions-called-once -m32 -Wall -Wextra -Werror 
-Wno-missing-braces -Wno-sign-compare -Wno-unknown-pragmas 
-Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds 
-Wno-switch -std=gnu99 -fno-inline-small-functions 
-fno-inline-functions-called-once -fno-ipa-cp -gdwarf-2 -std=gnu99 -gdwarf-2 
-I/root/smartos-live/projects/illumos/usr/src/lib/libfakekernel/common 
-D_FAKE_KERNEL -DTEXT_DOMAIN="SUNW_OST_OSCMD" -D_TS_ERRNO 
-I/root/smartos-live/proto/usr/include 
-I/root/smartos-live/proto.strap/usr/include -D_LARGEFILE64_SOURCE=1 
-D_REENTRANT -I../../../lib/libzpool/common -I../../../uts/common/fs/zfs 
-I../../../uts/common/fs/zfs/lua -I../../../common/zfs -DDEBUG -c ../ztest.c
../ztest.c: In function 'ztest_freeze':
../ztest.c:6344: error: 'spa_t' has no member named 'spa_debug'
*** Error code 1
make: Warning: Target `install' not remade because of errors
Current working directory 
/root/smartos-live/projects/illumos/usr/src/cmd/ztest/i386
*** Error code 1
The following command caused the error:
cd i386; pwd; /root/smartos-live/proto.strap/usr/bin/dmake install
/root/smartos-live/projects/illumos/usr/src/cmd/ztest/amd64
/root/smartos-live/projects/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw
 -_gcc -xO3 -_gcc=-fno-inline-small-functions  
-_gcc=-fno-inline-functions-called-once -m64 -Ui386 -U__i386  -Xa  -xildoff 
-errtags=yes -errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT 
-erroff=E_STATEMENT_NOT_REACHED -_gcc=-Wno-missing-braces 
-_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas -_gcc=-Wno-unused-parameter 
-_gcc=-Wno-missing-field-initializers -_gcc=-Wno-array-bounds -_gcc=-Wno-switch 
-xc99=%all-W0,-xglobalstatic -_gcc=-fno-inline-small-functions  
-_gcc=-fno-inline-functions-called-once  -_gcc=-fno-ipa-cp-g -xc99=%all 
-W0,-noglobal  -Wu,-save_args -xdebugformat=stabs -g -v 
-I/root/smartos-live/projects/illumos/usr/src/lib/libfakekernel/common 
-D_FAKE_KERNEL -DTEXT_DOMAIN=\"SUNW_OST_OSCMD\" -D_TS_ERRNO  
-I/root/smartos-live/proto/usr/include 
-I/root/smartos-live/proto.strap/usr/include -D_LARGEFILE64_SOURCE=1 
-D_REENTRANT -I../../../lib/libzpool/common -I../../../uts/common/fs/zfs 
-I../../../uts/common/fs/zfs/lua -I../../../common/zfs -DDEBUG -c ../ztest.c


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385876520
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M36fcd648c2be61d22e6c27fe
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

77c699a  Correct merge collision


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/cd8b6763fa2413d2f3950c719a6f4f7024b5a768..77c699a32931847d5557f0ebadcec07a680504b1

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mbd4bb1bc1845742215be2c08
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

latest commit does not compile (at least for me):

Makefile:294: recipe for target '0-illumos-stamp' failed
gmake: *** [0-illumos-stamp] Error 1

+ /root/smartos-live/proto.strap/usr/bin/gcc -fident -finline 
-fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option 
-nodefaultlibs -D__sun -O -fno-inline-small-functions 
-fno-inline-functions-called-once -m32 -Wall -Wextra -Werror 
-Wno-missing-braces -Wno-sign-compare -Wno-unknown-pragmas 
-Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds 
-Wno-unused -Wno-empty-body -Wno-uninitialized -Wno-switch -std=gnu99 
-fno-inline-small-functions -fno-inline-functions-called-once -fno-ipa-cp 
-DTEXT_DOMAIN="SUNW_OST_OSCMD" -D_TS_ERRNO 
-I/root/smartos-live/proto/usr/include 
-I/root/smartos-live/proto.strap/usr/include -D_REENTRANT 
-D_LARGEFILE64_SOURCE=1 
-I/root/smartos-live/projects/illumos/usr/src/uts/common/fs/zfs -c ../expound.c
/root/smartos-live/projects/illumos/usr/src/tools/proto/root_i386-nd/opt/onbld/bin/i386/cw
 -_gcc -O -_gcc=-fno-inline-small-functions  
-_gcc=-fno-inline-functions-called-once -m32 -xspace -Xa  -xildoff -errtags=yes 
-errwarn=%all -erroff=E_EMPTY_TRANSLATION_UNIT -erroff=E_STATEMENT_NOT_REACHED 
-_gcc=-Wno-missing-braces -_gcc=-Wno-sign-compare -_gcc=-Wno-unknown-pragmas 
-_gcc=-Wno-unused-parameter -_gcc=-Wno-missing-field-initializers 
-_gcc=-Wno-array-bounds -_gcc=-Wno-unused -_gcc=-Wno-empty-body 
-_gcc=-Wno-uninitialized -_gcc=-Wno-switch -xc99=%all-W0,-xglobalstatic 
-_gcc=-fno-inline-small-functions  -_gcc=-fno-inline-functions-called-once  
-_gcc=-fno-ipa-cp-v  -DTEXT_DOMAIN=\"SUNW_OST_OSCMD\" -D_TS_ERRNO  
-I/root/smartos-live/proto/usr/include 
-I/root/smartos-live/proto.strap/usr/include -D_REENTRANT 
-D_LARGEFILE64_SOURCE=1 
-I/root/smartos-live/projects/illumos/usr/src/uts/common/fs/zfs -c ../codes.c
+ /root/smartos-live/proto.strap/usr/bin/gcc -fident -finline 
-fno-inline-functions -fno-builtin -fno-asm -fdiagnostics-show-option 
-nodefaultlibs -D__sun -O -fno-inline-small-functions 
-fno-inline-functions-called-once -m32 -Wall -Wextra -Werror 
-Wno-missing-braces -Wno-sign-compare -Wno-unknown-pragmas 
-Wno-unused-parameter -Wno-missing-field-initializers -Wno-array-bounds 
-Wno-unused -Wno-empty-body -Wno-uninitialized -Wno-switch -std=gnu99 
-fno-inline-small-functions -fno-inline-functions-called-once -fno-ipa-cp 
-DTEXT_DOMAIN="SUNW_OST_OSCMD" -D_TS_ERRNO 
-I/root/smartos-live/proto/usr/include 
-I/root/smartos-live/proto.strap/usr/include -D_REENTRANT 
-D_LARGEFILE64_SOURCE=1 
-I/root/smartos-live/projects/illumos/usr/src/uts/common/fs/zfs -c ../codes.c
cc1: warnings being treated as errors
../codes.c:1292: error: braces around scalar initializer
../codes.c:1292: error: (near initialization for 'ioc[686].datastruct')
../codes.c:1292: error: initialization makes pointer from integer without a cast
../codes.c:1292: error: excess elements in scalar initializer
../codes.c:1292: error: (near initialization for 'ioc[686].datastruct')
../codes.c:1293: error: excess elements in scalar initializer
../codes.c:1293: error: (near initialization for 'ioc[686].datastruct')
../codes.c:1294: error: extra brace group at end of initializer
...
../codes.c:1660: error: extra brace group at end of initializer
../codes.c:1660: error: (near initialization for 'ioc[686]')
../codes.c:1661: error: excess elements in struct initializer
../codes.c:1661: error: (near initialization for 'ioc[686]')
../codes.c:1664: error: extra brace group at end of initializer
../codes.c:1664: error: (near initialization for 'ioc[686]')
../codes.c:1664: error: excess elements in struct initializer
../codes.c:1664: error: (near initialization for 'ioc[686]')
../codes.c:1665: error: expected '}' before ';' token
*** Error code 1


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385871769
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M309086030a6688eea078b448
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[kpande]

i've been using this in production (in the form of ZoL) for a few months and 
it's worked very well since the project quota fix was merged.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385846687
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M90cac9d3343c801d5ef046de
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

rebased and up to date


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-385841807
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mfb2272088bf627dd167140d3
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

a381944  Fix issues found with zfs diff


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/03c0787a23cdd0fc44a47690d69ffcfae64eb72b..a3819442d8617598d8e1463f8b52ea0589fe96cf

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M2f98ee40aa7224f7d964999f
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

yea, sorry, that is probably the root cause :)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-382299920
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M3db4bb0a9347c300c1cbec65
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Are you hinting that you want it rebased against master maybe?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-382294674
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mbebabd10b274462570d0805f
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

this is the commit:
https://github.com/illumos/illumos-gate/commit/3a4b1be953ee5601bab748afa07c26ed4996cde6#diff-c6fefc71095424603bca3dd5d115010b


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-382294029
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M9057c954cfb5d38478891e09
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Hmm I don't even have `vdev_indirect.c:1356` ?


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-382291271
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M550c100ce3fe00cb46abedb3
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Oppsie, my bad, one sec


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-382289822
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mda1b5e2d6061b7e17813e5a3
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

I have a build failure, probably related to latest Illumos commits:

../../../uts/common/fs/zfs/vdev_indirect.c: In function 
'vdev_indirect_checksum_error':
../../../uts/common/fs/zfs/vdev_indirect.c:1356: error: passing argument 3 of 
'zfs_ereport_post_checksum' from incompatible pointer type
../../../uts/common/fs/zfs/sys/zio.h:629: note: expected 'const struct 
zbookmark_phys_t *' but argument is of type 'struct zio_t *'
../../../uts/common/fs/zfs/vdev_indirect.c:1356: error: passing argument 4 of 
'zfs_ereport_post_checksum' makes pointer from integer without a cast
../../../uts/common/fs/zfs/sys/zio.h:629: note: expected 'struct zio *' but 
argument is of type 'uint64_t'
../../../uts/common/fs/zfs/vdev_indirect.c:1356: error: passing argument 6 of 
'zfs_ereport_post_checksum' makes integer from pointer without a cast
../../../uts/common/fs/zfs/sys/zio.h:629: note: expected 'uint64_t' but 
argument is of type 'void *'
../../../uts/common/fs/zfs/vdev_indirect.c:1356: error: passing argument 8 of 
'zfs_ereport_post_checksum' from incompatible pointer type
../../../uts/common/fs/zfs/sys/zio.h:629: note: expected 'const struct abd_t *' 
but argument is of type 'struct zio_bad_cksum_t *'
../../../uts/common/fs/zfs/vdev_indirect.c:1356: error: too few arguments to 
function 'zfs_ereport_post_checksum'
../../../uts/common/fs/zfs/vdev_indirect.c: In function 
'vdev_indirect_all_checksum_errors':
../../../uts/common/fs/zfs/vdev_indirect.c:1437: error: passing argument 3 of 
'zfs_ereport_post_checksum' from incompatible pointer type
../../../uts/common/fs/zfs/sys/zio.h:629: note: expected 'const struct 
zbookmark_phys_t *' but argument is of type 'struct zio_t *'
../../../uts/common/fs/zfs/vdev_indirect.c:1437: error: passing argument 4 of 
'zfs_ereport_post_checksum' makes pointer from integer without a cast
../../../uts/common/fs/zfs/sys/zio.h:629: note: expected 'struct zio *' but 
argument is of type 'uint64_t'
../../../uts/common/fs/zfs/vdev_indirect.c:1437: error: too few arguments to 
function 'zfs_ereport_post_checksum'
*** Error code 1
make: Warning: Command failed for target `pics/vdev_indirect.o'


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-382289553
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M2014eaae8090d0547c3ba043
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

dc23634  Fix issues with raw sends of spill blocks


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/5d6cd7d203a1eb182729d2390aedbe8d09ecf7a8..dc236348dd738254d462a5fdf8bc5e47ad65887b

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M058e20694b17768e2a3a3299
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 2 commits.

bc5a90b  Correct swapped keylocation error messages
5d6cd7d  Fix race in dnode_check_slots_free()


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/f46f0922baeffdc1b7b8750a4fe9ed4ef202ade6..5d6cd7d203a1eb182729d2390aedbe8d09ecf7a8

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Ma0374a66b3b83cc73d8c9511
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 1 commit.

f46f092  add codes to truss


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/893c519c9dcb7eae00acb466c9d19c2c37b26ed3..f46f0922baeffdc1b7b8750a4fe9ed4ef202ade6

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M08f38eba50343f77032f8681
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

893c519  receive_spill does not byte swap spill contents


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/2b622bf65fed5df4a70f8b63f9e9b42c4792bfea..893c519c9dcb7eae00acb466c9d19c2c37b26ed3

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M9d5fe51b60fd8fb843fb3eaf
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 3 commits.

363b893  assertion failure in arc_release() during encrypted receive
48f5098  tom feedback
2b622bf  more feedback


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/ef73ffd309419acb8389095d2887b2316cb1ba1d..2b622bf65fed5df4a70f8b63f9e9b42c4792bfea

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M3d8963760c77245a97894f28
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

ef73ffd  Move enum zio_encrypt into sys/fs/zfs.h


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/bf72974e371a6b1d097024f7cb567e9ea78fa65b..ef73ffd309419acb8389095d2887b2316cb1ba1d

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mc91e3730cb4f7d3fcc8bbea0
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Brian Behlendorf]

> Well, there's an unfortunate precedent of userland (at least libzfs) 
> including sys/zfs headers directly

For what it's worth, this is something we're slowly working to clean up so we 
should be able to pull back these kind of changes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-380621979
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M93e2d094b5ddcd9e2a8d7e43
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@lundman let's move `enum zio_encrypt` to `sys/fs/zfs.h`.  Thanks!

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-380618411
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M56987fb4c07d04d89b42935a
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

bf72974  Correct manpage for --raw


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/097d997308675d4d2387d5bb8a786fb09a189dfc..bf72974e371a6b1d097024f7cb567e9ea78fa65b

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mfe2dd9b5e8947750676a2670
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Not sure I'm seeing it. The difference to ZOL is that they have `zfs_main.c` add
```
#include "libzfs_impl.h"
```

But I can not add that to IllumOS, as none of the -I paths include 
`libzfs_impl.h` nor `sys/zio.h` - are you suggesting I change Makefiles to add 
the include path? As for existing precedent in `cmd/zfs` regards to property 
comparison is `canmount == ZFS_CANMOUNT_OFF` which works as it is defined in 
`sys/fs/zfs.h`  Please indicate your preference :)


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-380290582
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mb89861725f93f8d800357b9d
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

Well, there's an unfortunate precedent of userland (at least libzfs) including 
sys/zfs headers directly, so we could probably just try to include them here.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-380164599
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mae3aad66a3f231142ec0fb91
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

Hmm that's defined in `zio.h` but the `cmd`'s tend not to include the sys/zfs* 
headers directly? Should they be in `zfs_context`?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-380063331
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M6f046015fda8b8c4f414d8af
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

Problem with 097d997:

zfs_main.c: In function 'share_mount_one':
zfs_main.c:6072: error: 'ZIO_CRYPT_OFF' undeclared (first use in this function)
zfs_main.c:6072: error: (Each undeclared identifier is reported only once
zfs_main.c:6072: error: for each function it appears in.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-380061264
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M1d6fe5e59db011b7a667e4a0
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 1 commit.

097d997  Make encrypted "zfs mount -a" failures consistent


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/da391edb5d7057961eda8585b6d6af08cda5d64d..097d997308675d4d2387d5bb8a786fb09a189dfc

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M5b00b92b2717c4e4399b1d65
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 1 commit.

c3623f8  lint


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/b7734bfecda4baac05c3416bf20211086fea7275..c3623f89d3a48ffce3e6b9f0a825e1125af7f414

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M69e8b22535e9581beccfad3c
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

I pushed 2 commits.  I'm going to try to open ZoL PR's for each of them as well.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-378776850
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mbd8a1a3247e586d4ed2c6bbb
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 2 commits.

c8b50d0  fail fraction
b7734bf  fixes for SNPRINTF_BLKPTR with encrypted BP's


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/fa1f2c81b0425c4f72d3a332f667cc8dad71d91f..b7734bfecda4baac05c3416bf20211086fea7275

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M489c664c71e5f99ac782f64c
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

ahrens commented on this pull request.



> @@ -409,6 +489,26 @@ _NOTE(CONSTCOND) } while (0)
 #defineBP_GET_LEVEL(bp)BF64_GET((bp)->blk_prop, 56, 5)
 #defineBP_SET_LEVEL(bp, x) BF64_SET((bp)->blk_prop, 56, 5, 
x)
 
+/* encrypted, authenticated, and MAC cksum bps use the same bit */
+#defineBP_USES_CRYPT(bp)   BF64_GET((bp)->blk_prop, 61, 1)
+#defineBP_SET_CRYPT(bp, x) BF64_SET((bp)->blk_prop, 61, 1, 
x)
+
+#defineBP_IS_ENCRYPTED(bp) \
+   (BP_USES_CRYPT(bp) &&   \
+   BP_GET_LEVEL(bp) == 0 &&\

The ZoL version has `<= 0`.  That said, I think that BP_GET_LEVEL() will result 
in an unsigned value, so it can never be <0.  And the BP_GET_LEVEL() is never 
negative-ish (`ZB_ZIL_LEVEL` and `ZB_ROOT_LEVEL` are not used here, those block 
types have BP_GET_LEVEL()==0).  So I think we should change ZoL to use `==` 
here, @tcaputi.

> @@ -409,6 +489,26 @@ _NOTE(CONSTCOND) } while (0)
 #defineBP_GET_LEVEL(bp)BF64_GET((bp)->blk_prop, 56, 5)
 #defineBP_SET_LEVEL(bp, x) BF64_SET((bp)->blk_prop, 56, 5, 
x)
 
+/* encrypted, authenticated, and MAC cksum bps use the same bit */
+#defineBP_USES_CRYPT(bp)   BF64_GET((bp)->blk_prop, 61, 1)
+#defineBP_SET_CRYPT(bp, x) BF64_SET((bp)->blk_prop, 61, 1, 
x)
+
+#defineBP_IS_ENCRYPTED(bp) \
+   (BP_USES_CRYPT(bp) &&   \
+   BP_GET_LEVEL(bp) == 0 &&\
+   DMU_OT_IS_ENCRYPTED(BP_GET_TYPE(bp)))
+
+#defineBP_IS_AUTHENTICATED(bp) \
+   (BP_USES_CRYPT(bp) &&   \
+   BP_GET_LEVEL(bp) <= 0 &&\

I think we should also change this to `==` (here and in ZoL, @tcaputi)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#pullrequestreview-109527189
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mc4c72b4ee296249d833a672b
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS That last panic is quite bizarre, and doesn't appear to be related to 
encryption.  We have an indirect block (level 1) of a space map, and it's 
filled with a pattern like 0xdadada... but every 16th byte is incremented by 
0x10.  The block is in the L2ARC.  Of course, checksums should have ensured 
that even if the L2ARC is busted, we fall back on the regular pool data.

Could you check if the block is busted on disk, vs just in memory, by running 
`zdb -d zones 219` ?

Also, does the machine running this have ECC RAM?

```
> ff0dbe19e898::print dmu_buf_impl_t db_parent->db.db_data | /50J
0xff0dbeb82000: dadadadadadadada
dadadadadadadada dadadadadadadaea
dadadadadadadada dadadadadadadafa
dadadadadadadada dadadadadadada0a
dadadadadadadada dadadadadadada1a
dadadadadadadada dadadadadadada2a
dadadadadadadada dadadadadadada3a
dadadadadadadada dadadadadadada4a
dadadadadadadada dadadadadadada5a
dadadadadadadada dadadadadadada6a
dadadadadadadada dadadadadadada7a
dadadadadadadada dadadadadadada8a
dadadadadadadada dadadadadadada9a
dadadadadadadada dadadadadadadaaa
dadadadadadadada dadadadadadadaba
dadadadadadadada dadadadadadadaca
dadadadadadadada dadadadadadadada
dadadadadadadada dadadadadadadaea
dadadadadadadada dadadadadadadafa
dadadadadadadada dadadadadadada0a
dadadadadadadada dadadadadadada1a
dadadadadadadada dadadadadadada2a
dadadadadadadada dadadadadadada3a
dadadadadadadada dadadadadadada4a
dadadadadadadada dadadadadadada5a
dadadadadadadada
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-378327200
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mbcf17bdc0bfcd0f6c0049cdc
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Jorgen Lundman]

@lundman pushed 3 commits.

bdcf9d7  Encrypted dnode blocks should be prefetched raw
f016fc4  Decryption error handling improvements
9b1e59f  Remove ASSERT() in l2arc_apply_transforms()


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/1a503d4121bb673a3f99a9a16da28b6f9bf3349d..9b1e59f1e9bd6dd36ee256096d9228e8fd383001

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M3bd82c41e368ad2ca1e0eb61
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

While I cannot reproduce l2arc panic anymore, I had another (new) one reagding 
zvols:

panic[cpu3]/thread=ff003fe9ec40:
blkptr at ff0dbeb82200 has invalid TYPE 218


ff003fe9e330 genunix:vcmn_err+42 ()
ff003fe9e3a0 zfs:zfs_panic_recover+51 ()
ff003fe9e400 zfs:zfs_blkptr_verify+365 ()
ff003fe9e4e0 zfs:zio_read+55 ()
ff003fe9e600 zfs:arc_read+818 ()
ff003fe9e6c0 zfs:dbuf_read_impl+4e0 ()
ff003fe9e730 zfs:dbuf_read+ea ()
ff003fe9e780 zfs:dmu_buf_will_dirty_impl+cf ()
ff003fe9e7a0 zfs:dmu_buf_will_dirty+1d ()
ff003fe9e840 zfs:dmu_write_impl+73 ()
ff003fe9e8c0 zfs:dmu_write+98 ()
ff003fe9e990 zfs:space_map_write+2f6 ()
ff003fe9ea30 zfs:metaslab_sync+133 ()
ff003fe9ea90 zfs:vdev_sync+6f ()
ff003fe9eb70 zfs:spa_sync+46b ()
ff003fe9ec20 zfs:txg_sync_thread+227 ()
ff003fe9ec30 unix:thread_start+8 ()

I have a dump, but non-debug and not with kmem-flags



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377699909
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M93ebd6e00ba364c45b9de161
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS The problem causing the panic in sa.c is that zio_decrypt() was not 
properly propagating a decryption / MAC verification error.  I added a commit 
to this PR which fixes that.

You should now get an error message when doing the `ls -l`, rather than a 
panic.  The next issue is why it's getting the MAC verification error.  
Probably something has gone wrong with the encrypted send or receive.  If you 
can share the send stream, or more info on the contents of the filesystem that 
you're sending, that would be helpful.

Can you also see if you can still reproduce the l2arc panic with the latest 
fix?  It's possible that having kmem_flags set hides the error, so try without 
kmem_flags too.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377396867
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M05a8c6a163958658f60e721d
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@ahrens pushed 1 commit.

1a503d4  dont swallow error!


-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/openzfs/openzfs/pull/489/files/a048a01fa13fac430a9f9c838f70569e54122101..1a503d4121bb673a3f99a9a16da28b6f9bf3349d

--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M8f494767f757823d44e95ef4
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

I looks like I need to increase my dump device:

 

savecore -f vmdump.118

savecore: incomplete dump on dump device

savecore: System dump time: Thu Mar 29 09:50:51 2018

 

savecore: saving system crash dump in /var/crash/volatile/{unix,vmcore}.118

Constructing namelist /var/crash/volatile/unix.118

Constructing corefile /var/crash/volatile/vmcore.118

pfn 2180825 not found for as=fbc59080, va=ff000fc01000

pfn 2182138 not found for as=fbc59080, va=ff000fc02000

pfn 2199037 not found for as=fbc59080, va=ff000fb05000

pfn 2177482 not found for as=fbc59080, va=ff000f502000

pfn 2174251 not found for as=fbc59080, va=ff000f503000

pfn 2178956 not found for as=fbc59080, va=ff000f504000

pfn 2177325 not found for as=fbc59080, va=ff000f505000

pfn 2175198 not found for as=fbc59080, va=ff000f506000

pfn 2207933 not found for as=fbc59080, va=ff000f605000

pfn 2167922 not found for as=fbc59080, va=ff000fc0a000

3:08  99% donesavecore: dsize 0x9 != pagesize 0x1000

 

Will retry using 8GB dump device

Would you mind sharing your email address so we can perhaps speed things up a 
bit?

 

 

Von: Matthew Ahrens [mailto:notificati...@github.com] 
Gesendet: Donnerstag, 29. März 2018 17:54
An: openzfs/openzfs 
Cc: GernotS ; Mention 
Betreff: Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs 
(#489)

 

@GernotS   Thanks. I downloaded vmdump.118 but it 
doesn't seem to have come through correctly. Any chance you uploaded it while 
it was still being written? The file length I have is 3347091456 bytes, and I 
get:

$ savecore -f vmdump.118 .
savecore: bad magic number 78201ff0
savecore: failed to publish 'savecore_failure' event: Library API usage 
violation

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub 
 , or mute 
the thread 

 .



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377296782
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mb58118191b2c0ac715cf1b05
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS Thanks.  I downloaded vmdump.118 but it doesn't seem to have come 
through correctly.  Any chance you uploaded it while it was still being 
written?  The file length I have is `3347091456` bytes, and I get:
```
$ savecore -f vmdump.118 .
savecore: bad magic number 78201ff0
savecore: failed to publish 'savecore_failure' event: Library API usage 
violation
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377281600
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M52854b36ed9d74b4352511a9
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

@Ahrens
ls -l /zones/data/fritz gives me
/zones/data/fritz: Bad exchange descriptor

trying other directories gives me the known crash



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377275669
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mcf54ba34af577d2e5630bc08
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

Thanks! Can you try ls -l /zones/data/fritz too? Does that crash?

On Thu, Mar 29, 2018 at 5:28 AM GernotS  wrote:

> @ahrens 
>
> steps to reproduce
> zfs send -w zones|datatest@1 |zfs recv zones/data (in my case about 700GB)
> zfs set canmount=noauto zones/data
> zfs load-key -L prompt zones/data
> zfs mount zones/data
> ls /zones/data
> Crash
>
> full dump:
> https://www.magentacloud.de/lnk/MarBVNNc
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
> 
> .
>


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377245486
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M6b047afe57072a1fd890c291
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS Thanks.  I'm still not sure what's going on with the sa.c dump, and 
have a few more questions for you:

Can you describe how you reproduce the sa.c dump?  I could gather some of the 
sequence below but if you can confirm that and fill in any other details that 
would be helpful.

It looks like you created `/zones/data` from a zfs receive that is from a raw 
encrypted send stream?  At the time of the panic you're doing `ls --color -lhB 
--color /zones/data/` and it's doing a stat() syscall on `/zones/data/fritz`.  
Is the `ls` essentially all you did after booting the machine?

Could you try doing `ls -l /zones/data/fritz` (preferably just after doing the 
"zfs receive" to create zones/data), and see if that also panics?  This will 
help me tell if the problem is a race condition with the reading vs with the 
data on-disk.

Could you try getting a "full" dump by running `dumpadm -c all` before 
triggering the panic?  This will capture the encrypted data in the ARC as well 
(it will be bigger, but it looks like your machine only has 8GB RAM so 
hopefully it's still palatable).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-377078558
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-Mb5e6e80e2912cc97d9899543
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

another dump for the assert in sa.c, hopefully with debug bits now
https://www.magentacloud.de/lnk/I0rB1RA1


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-376915757
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M7ab14486ff7ea73a717bf05c
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

I have now moved around a few TBs using raw send, compressed send, etc with 
L2ARC devices, but I cannot reproduce the issue any more. Is it possible 
kmem_flags hides this?
Will try to get a debug build done.



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-376659581
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M65edea6d4cfb7223b5c342d9
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[Matthew Ahrens]

@GernotS Thanks.  I'm also very interested in getting this merged :-)

kmem_flags is great, but in addition it would be good to try a debug build, 
i.e. using the `nightly` rather than `nightly-nd` packages.  This might be able 
to hit an assertion sooner - closer to where the error occurred.

FYI, assuming the `+` is your code, that's the same as what's in this PR - we 
are using `DRR_OBJECT_PAYLOAD_SIZE()`.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-376625262
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M2debbb1e1e18430f460522d9
Delivery options: https://openzfs.topicbox.com/groups

[developer] Re: [openzfs/openzfs] 8727 Native data and metadata encryption for zfs (#489)

[GernotS]

Dump for the assert in sa.c
https://www.magentacloud.de/lnk/4xLBV469


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openzfs/openzfs/pull/489#issuecomment-376474714
--
openzfs: openzfs-developer
Permalink: 
https://openzfs.topicbox.com/groups/developer/discussions/T91797982fdd5b7d9-M50ad4f149291950a62e3d79e
Delivery options: https://openzfs.topicbox.com/groups