Darren,
It looks like you modeled your ENCRYPT/DECRYPT stages on the
COMPRESS/UNCOMPRESS stages. This is all well and good... however
your emulation is not quite accurate: The WRITE_COMPRESS stage
is *always* included in the write pipeline, so the zio_write()
func only adds it to the async_stages variable when compression
is turned on. The WRITE_ENCRYPT stage, on the other hand is
explicitly in the ASYNC_STAGES, but *not* always part of the
write pipeline. In zio_write() you are setting it in the
async_stages variable... which it is already in. I think you
need to add it to the pipeline rather than the async_stages.
Its still not clear to me how this can result in your problems, but
then I don't yet understand how the SPA io pipeline works in all
circumstances.
-Mark
Mark Maybee wrote:
> Darren,
>
> I looked a bit at your dumps... in both cases, the problem is that the
> os_phys block that we read from the disk is garbage:
>
> > 0xffffffff9377b000::print objset_phys_t
> {
> os_meta_dnode = {
> dn_type = 0
> dn_indblkshift = 0
> dn_nlevels = 0
> dn_nblkptr = 0
> dn_bonustype = 0
> dn_checksum = 0
> dn_compress = 0
> dn_flags = 0
> dn_datablkszsec = 0x3
> dn_bonuslen = 0
> dn_pad2 = [ 0, 0, 0 ]
> dn_crypt = 0
> dn_maxblkid = 0
> dn_used = 0
> dn_pad3 = [ 0, 0, 0, 0 ]
> dn_blkptr = [
> {
> blk_dva = [
> {
> dva_word = [ 0, 0 ]
> }
> ...
>
> I checked the actual arc buf this came from, and it looks the same. So
> the buf was successfully read, and it checksummed, but it doesn't have
> good data. This pretty much says that the problem is on the write side.
> When we wrote out the root block in dmu_objset_sync(), we must have
> written garbage. I'm not yet quite sure how this happened... perhaps
> something is messed up in your write path changes
> (arc_write->zio_write->...), but its not obvious. I'll investigate some
> more when I get a chance....
>
> -Mark
>
> Darren J Moffat wrote:
>
>> I really need some help on this. Without help the ZFS crypto project
>> is stalled.
>>
>> I've updated my bits to the ON gate as of last night. The way I
>> recreate this is slightly different but the assert is still the same.
>>
>> I can create a pool with my bits and export it; when I import it I
>> get the dn_levels assert.
>>
>> Please I really need help.
>>
>> Darren J Moffat wrote:
>>
>>> Using the ZFS crypto bits, see [1] for webrev, which are in sync with ON
>>> as of 2006-09-12 (ie they include the BrandZ stuff and the changes
>>> that Eric putback on the 12th).
>>>
>>> [1] http://cr.grommit.com/~darrenm/zfs-crypto/
