Re: [smartos-discuss] vioblk driver

[Nahum Shalman]

On Mon, Feb 13, 2017 at 4:15 PM, Robert Mustacchi  wrote:

> On 2/13/17 12:56 , Youzhong Yang wrote:
> > It seems it's not that simple. I will leave it for later troubleshooting
> as
> > I don't have cycles to debug it now.
>
> Do you have a write up somewhere about what you encountered and what
> went wrong?
>

I think you should be able to reproduce the issues encountered by booting
SmartOS under SmartOS KVM with qemu_extra_opts tweaking the virtio physical
and logical block sizes e.g.:

On Mon, Jan 30, 2017 at 2:07 PM, Nahum Shalman  wrote:

> I think you should be able to set "qemu_extra_opts" to e.g. "-global
> virtio-blk-pci.physical_block_size=8192 -global 
> virtio-blk-pci.logical_block_size=4096"
> to test tweaking those values.
>

I was certainly able to trip a bug with those particular settings which
seemed like the kind of thing an SSD might do.
When using those settings, diskinfo got the size wrong and the system was
unable to create a ZFS pool.

-Nahum



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Robert Mustacchi]

On 2/13/17 12:56 , Youzhong Yang wrote:
> It seems it's not that simple. I will leave it for later troubleshooting as
> I don't have cycles to debug it now.

Do you have a write up somewhere about what you encountered and what
went wrong?

Robert

> On Wed, Feb 1, 2017 at 8:33 PM, Robert Mustacchi  wrote:
> 
>> On 1/30/17 16:45 , Youzhong Yang wrote:
>>> Thanks for helping out.
>>>
>>> What I need is to set logical block size to something larger than 512,
>> e.g.
>>> 4096. It doesn't work.
>>>
>>> I guess the bug is here:
>>>
>>> http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/
>> common/io/vioblk/vioblk.c#919
>>>
>>> sc_capacity is in 512B, needs to be converted to logical blocks. I will
>>> test the patch and report back.
>>
>> I agree, that definitely appears to be wrong.
>>
>> Robert
>>
> 
> 


---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Robert Mustacchi]

On 1/30/17 16:45 , Youzhong Yang wrote:
> Thanks for helping out.
> 
> What I need is to set logical block size to something larger than 512, e.g.
> 4096. It doesn't work.
> 
> I guess the bug is here:
> 
> http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/vioblk/vioblk.c#919
> 
> sc_capacity is in 512B, needs to be converted to logical blocks. I will
> test the patch and report back.

I agree, that definitely appears to be wrong.

Robert


---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Nahum Shalman]

Only setting:
"qemu_extra_opts": "-global virtio-blk-pci.physical_block_size=4096"
appeared to work. diskinfo reported the correct size, creating a pool on it
succeeded, and it had an ashift of 12.

Not sure where that will leave you.
-Nahum


On Mon, Jan 30, 2017 at 2:30 PM, Nahum Shalman  wrote:

> I wonder if I've just stumbled on a bug. I provisioned a VM with a zvol of
> 8GB, and with those qemu_extra_opts, but "diskinfo" thinks the disk is 64GB
> in size...
> I don't know the full mdb-fu you used to find that address and dump out
> those properties.
>
> Something seems a little fishy here.
> -Nahum
>
> On Mon, Jan 30, 2017 at 2:07 PM, Nahum Shalman 
> wrote:
>
>> That's only modifying the volblocksize of the underlying zvol. Getting
>> QEMU to advertise a different blocksize might be trickier which is probably
>> what's necessary.
>> I found reference to "-global ide-drive.physical_block_size=4096" in
>> http://wiki.qemu.org/download/qemu-doc.html#sec_005finvocation
>>
>> After a bunch of digging I ran the following two commands and got the
>> output as shown:
>>
>> # /smartdc/bin/qemu-system-x86_64 -device ? 2>&1 | grep -i virtio
>> name "virtio-balloon-pci", bus PCI
>> name "virtio-serial-pci", bus PCI, alias "virtio-serial"
>> name "virtio-net-pci", bus PCI
>> name "virtio-blk-pci", bus PCI, alias "virtio-blk"
>> name "virtconsole", bus virtio-serial-bus
>> name "virtserialport", bus virtio-serial-bus
>> # /smartdc/bin/qemu-system-x86_64 -device virtio-blk-pci,? 2>&1 | grep
>> block_size
>> virtio-blk-pci.logical_block_size=uint16
>> virtio-blk-pci.physical_block_size=uint16
>>
>> So I think you should be able to set "qemu_extra_opts" to e.g. "-global
>> virtio-blk-pci.physical_block_size=8192 -global 
>> virtio-blk-pci.logical_block_size=4096"
>> to test tweaking those values.
>>
>> Good luck!
>> -Nahum
>>
>>
>> On Mon, Jan 30, 2017 at 12:11 PM, Youzhong Yang 
>> wrote:
>>
>>> Thanks, it works.
>>>
>>> However, I am having difficulty setting the block size to anything other
>>> than 512:
>>>
>>> vmadm tells the block size is 8192,
>>>
>>> # vmadm get b175850d-92fd-cf81-d352-90d40e5076ce |json disks
>>> [
>>>   {
>>> "path": "/dev/zvol/rdsk/zones/b175850d-92fd-cf81-d352-90d40e5076ce-d
>>> isk1",
>>> "boot": false,
>>> "model": "virtio",
>>> "media": "disk",
>>> "zfs_filesystem": "zones/b175850d-92fd-cf81-d352
>>> -90d40e5076ce-disk1",
>>> "zpool": "zones",
>>> "size": 65536,
>>> "compression": "on",
>>> "refreservation": 65536,
>>> "block_size": 8192
>>>   }
>>> ]
>>>
>>> but inside the VM, it says the block size is 512:
>>>
>>> > ff03d8df9700::print -t struct vioblk_softc sc_virtio.sc_features
>>> sc_blk_size sc_pblk_size
>>> uint32_t sc_virtio.sc_features = 0x1454
>>> int sc_blk_size = 0x200
>>> int sc_pblk_size = 0x200
>>>
>>> Quite confused ..
>>>
>>> Any thoughts?
>>>
>>> Thanks,
>>>
>>> --Youzhong
>>>
>>>
>>>
>>> On Mon, Jan 30, 2017 at 10:10 AM, Nahum Shalman 
>>> wrote:
>>>
 You can also create a SmartOS KVM VM on a SmartOS machine with a virtio
 virtual disk.

 -Nahum

 On Mon, Jan 30, 2017 at 9:38 AM, Youzhong Yang 
 wrote:

> Hi All,
>
> A quick question - how can I make use of vioblk device driver?
> Creating a SmartOS VM using Linux KVM?
>
> The reason I am asking this is that I want to test a change in blkdev,
> to exercise some code path in it which can't be done using NVMe devices,
> such as the 'dump' functionality.
>
> Thanks,
>
> Youzhong
>
>

>>> *smartos-discuss* | Archives
>>> 
>>>  |
>>> Modify
>>> 
>>> Your Subscription 
>>>
>>
>>
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Nahum Shalman]

I wonder if I've just stumbled on a bug. I provisioned a VM with a zvol of
8GB, and with those qemu_extra_opts, but "diskinfo" thinks the disk is 64GB
in size...
I don't know the full mdb-fu you used to find that address and dump out
those properties.

Something seems a little fishy here.
-Nahum

On Mon, Jan 30, 2017 at 2:07 PM, Nahum Shalman  wrote:

> That's only modifying the volblocksize of the underlying zvol. Getting
> QEMU to advertise a different blocksize might be trickier which is probably
> what's necessary.
> I found reference to "-global ide-drive.physical_block_size=4096" in
> http://wiki.qemu.org/download/qemu-doc.html#sec_005finvocation
>
> After a bunch of digging I ran the following two commands and got the
> output as shown:
>
> # /smartdc/bin/qemu-system-x86_64 -device ? 2>&1 | grep -i virtio
> name "virtio-balloon-pci", bus PCI
> name "virtio-serial-pci", bus PCI, alias "virtio-serial"
> name "virtio-net-pci", bus PCI
> name "virtio-blk-pci", bus PCI, alias "virtio-blk"
> name "virtconsole", bus virtio-serial-bus
> name "virtserialport", bus virtio-serial-bus
> # /smartdc/bin/qemu-system-x86_64 -device virtio-blk-pci,? 2>&1 | grep
> block_size
> virtio-blk-pci.logical_block_size=uint16
> virtio-blk-pci.physical_block_size=uint16
>
> So I think you should be able to set "qemu_extra_opts" to e.g. "-global
> virtio-blk-pci.physical_block_size=8192 -global 
> virtio-blk-pci.logical_block_size=4096"
> to test tweaking those values.
>
> Good luck!
> -Nahum
>
>
> On Mon, Jan 30, 2017 at 12:11 PM, Youzhong Yang 
> wrote:
>
>> Thanks, it works.
>>
>> However, I am having difficulty setting the block size to anything other
>> than 512:
>>
>> vmadm tells the block size is 8192,
>>
>> # vmadm get b175850d-92fd-cf81-d352-90d40e5076ce |json disks
>> [
>>   {
>> "path": "/dev/zvol/rdsk/zones/b175850d-92fd-cf81-d352-90d40e5076ce-
>> disk1",
>> "boot": false,
>> "model": "virtio",
>> "media": "disk",
>> "zfs_filesystem": "zones/b175850d-92fd-cf81-d352-90d40e5076ce-disk1",
>> "zpool": "zones",
>> "size": 65536,
>> "compression": "on",
>> "refreservation": 65536,
>> "block_size": 8192
>>   }
>> ]
>>
>> but inside the VM, it says the block size is 512:
>>
>> > ff03d8df9700::print -t struct vioblk_softc sc_virtio.sc_features
>> sc_blk_size sc_pblk_size
>> uint32_t sc_virtio.sc_features = 0x1454
>> int sc_blk_size = 0x200
>> int sc_pblk_size = 0x200
>>
>> Quite confused ..
>>
>> Any thoughts?
>>
>> Thanks,
>>
>> --Youzhong
>>
>>
>>
>> On Mon, Jan 30, 2017 at 10:10 AM, Nahum Shalman 
>> wrote:
>>
>>> You can also create a SmartOS KVM VM on a SmartOS machine with a virtio
>>> virtual disk.
>>>
>>> -Nahum
>>>
>>> On Mon, Jan 30, 2017 at 9:38 AM, Youzhong Yang 
>>> wrote:
>>>
 Hi All,

 A quick question - how can I make use of vioblk device driver? Creating
 a SmartOS VM using Linux KVM?

 The reason I am asking this is that I want to test a change in blkdev,
 to exercise some code path in it which can't be done using NVMe devices,
 such as the 'dump' functionality.

 Thanks,

 Youzhong


>>>
>> *smartos-discuss* | Archives
>> 
>>  |
>> Modify
>> 
>> Your Subscription 
>>
>
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Nahum Shalman]

That's only modifying the volblocksize of the underlying zvol. Getting QEMU
to advertise a different blocksize might be trickier which is probably
what's necessary.
I found reference to "-global ide-drive.physical_block_size=4096" in
http://wiki.qemu.org/download/qemu-doc.html#sec_005finvocation

After a bunch of digging I ran the following two commands and got the
output as shown:

# /smartdc/bin/qemu-system-x86_64 -device ? 2>&1 | grep -i virtio
name "virtio-balloon-pci", bus PCI
name "virtio-serial-pci", bus PCI, alias "virtio-serial"
name "virtio-net-pci", bus PCI
name "virtio-blk-pci", bus PCI, alias "virtio-blk"
name "virtconsole", bus virtio-serial-bus
name "virtserialport", bus virtio-serial-bus
# /smartdc/bin/qemu-system-x86_64 -device virtio-blk-pci,? 2>&1 | grep
block_size
virtio-blk-pci.logical_block_size=uint16
virtio-blk-pci.physical_block_size=uint16

So I think you should be able to set "qemu_extra_opts" to e.g.
"-global virtio-blk-pci.physical_block_size=8192
-global virtio-blk-pci.logical_block_size=4096" to test tweaking those
values.

Good luck!
-Nahum


On Mon, Jan 30, 2017 at 12:11 PM, Youzhong Yang  wrote:

> Thanks, it works.
>
> However, I am having difficulty setting the block size to anything other
> than 512:
>
> vmadm tells the block size is 8192,
>
> # vmadm get b175850d-92fd-cf81-d352-90d40e5076ce |json disks
> [
>   {
> "path": "/dev/zvol/rdsk/zones/b175850d-92fd-cf81-d352-
> 90d40e5076ce-disk1",
> "boot": false,
> "model": "virtio",
> "media": "disk",
> "zfs_filesystem": "zones/b175850d-92fd-cf81-d352-90d40e5076ce-disk1",
> "zpool": "zones",
> "size": 65536,
> "compression": "on",
> "refreservation": 65536,
> "block_size": 8192
>   }
> ]
>
> but inside the VM, it says the block size is 512:
>
> > ff03d8df9700::print -t struct vioblk_softc sc_virtio.sc_features
> sc_blk_size sc_pblk_size
> uint32_t sc_virtio.sc_features = 0x1454
> int sc_blk_size = 0x200
> int sc_pblk_size = 0x200
>
> Quite confused ..
>
> Any thoughts?
>
> Thanks,
>
> --Youzhong
>
>
>
> On Mon, Jan 30, 2017 at 10:10 AM, Nahum Shalman 
> wrote:
>
>> You can also create a SmartOS KVM VM on a SmartOS machine with a virtio
>> virtual disk.
>>
>> -Nahum
>>
>> On Mon, Jan 30, 2017 at 9:38 AM, Youzhong Yang 
>> wrote:
>>
>>> Hi All,
>>>
>>> A quick question - how can I make use of vioblk device driver? Creating
>>> a SmartOS VM using Linux KVM?
>>>
>>> The reason I am asking this is that I want to test a change in blkdev,
>>> to exercise some code path in it which can't be done using NVMe devices,
>>> such as the 'dump' functionality.
>>>
>>> Thanks,
>>>
>>> Youzhong
>>>
>>>
>>
> *smartos-discuss* | Archives
> 
>  |
> Modify
> 
> Your Subscription 
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Youzhong Yang]

Thanks, it works.

However, I am having difficulty setting the block size to anything other
than 512:

vmadm tells the block size is 8192,

# vmadm get b175850d-92fd-cf81-d352-90d40e5076ce |json disks
[
  {
"path":
"/dev/zvol/rdsk/zones/b175850d-92fd-cf81-d352-90d40e5076ce-disk1",
"boot": false,
"model": "virtio",
"media": "disk",
"zfs_filesystem": "zones/b175850d-92fd-cf81-d352-90d40e5076ce-disk1",
"zpool": "zones",
"size": 65536,
"compression": "on",
"refreservation": 65536,
"block_size": 8192
  }
]

but inside the VM, it says the block size is 512:

> ff03d8df9700::print -t struct vioblk_softc sc_virtio.sc_features
sc_blk_size sc_pblk_size
uint32_t sc_virtio.sc_features = 0x1454
int sc_blk_size = 0x200
int sc_pblk_size = 0x200

Quite confused ..

Any thoughts?

Thanks,

--Youzhong



On Mon, Jan 30, 2017 at 10:10 AM, Nahum Shalman  wrote:

> You can also create a SmartOS KVM VM on a SmartOS machine with a virtio
> virtual disk.
>
> -Nahum
>
> On Mon, Jan 30, 2017 at 9:38 AM, Youzhong Yang  wrote:
>
>> Hi All,
>>
>> A quick question - how can I make use of vioblk device driver? Creating a
>> SmartOS VM using Linux KVM?
>>
>> The reason I am asking this is that I want to test a change in blkdev, to
>> exercise some code path in it which can't be done using NVMe devices, such
>> as the 'dump' functionality.
>>
>> Thanks,
>>
>> Youzhong
>>
>>
> *smartos-discuss* | Archives
> 
>  |
> Modify
> 
> Your Subscription 
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vioblk driver

[Nahum Shalman]

You can also create a SmartOS KVM VM on a SmartOS machine with a virtio
virtual disk.

-Nahum

On Mon, Jan 30, 2017 at 9:38 AM, Youzhong Yang  wrote:

> Hi All,
>
> A quick question - how can I make use of vioblk device driver? Creating a
> SmartOS VM using Linux KVM?
>
> The reason I am asking this is that I want to test a change in blkdev, to
> exercise some code path in it which can't be done using NVMe devices, such
> as the 'dump' functionality.
>
> Thanks,
>
> Youzhong
>
> *smartos-discuss* | Archives
> 
>  |
> Modify
> 
> Your Subscription 
>



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

[smartos-discuss] vioblk driver

[Youzhong Yang]

Hi All,

A quick question - how can I make use of vioblk device driver? Creating a
SmartOS VM using Linux KVM?

The reason I am asking this is that I want to test a change in blkdev, to
exercise some code path in it which can't be done using NVMe devices, such
as the 'dump' functionality.

Thanks,

Youzhong



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com