Re: increasing bhyve VM_MAXCPU

[John Doherty via freebsd-virtualization]

On 18 May 2021, at 19:21, Rodney W. Grimes wrote:

>> I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD
>> I began working on a machine with only eight CPU cores/threads, so
>> rather than trying to increase VM_MAXCPU, I tried to decrease it, which
>> seemed to work fine. My procedure was:
>>
>>edit /usr/src/sys/amd64/include/vmm.h
>
> cp /usr/src/sys/amd64/include/vmm.h /usr/include/machine/vmm.h

Thanks so much for this. Things are now working fine, e.g.:

  [root@fbsd1] # grep 'Multiprocessor System Detected' /var/run/dmesg.boot
  FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: resizing aarch64 image on arm64

[John-Mark Gurney]

Rodney W. Grimes wrote this message on Tue, Apr 06, 2021 at 06:01 -0700:
> > tech-lists wrote this message on Mon, Apr 05, 2021 at 16:59 +:
> > > Can anyone tell me how to resize
> > > https://download.freebsd.org/ftp/releases/VM-IMAGES/13.0-RC5/aarch64/Latest/FreeBSD-13.0-RC5-arm64-aarch64.raw
> > > 
> > > on an amd64 host?
> > 
> > note that this has been unxz'd already.
> > 
> > truncate -s 6g FreeBSD-13.0-RC5-arm64-aarch64.raw   # enlarge
> > mdconfig -f FreeBSD-13.0-RC5-arm64-aarch64.raw  # make device
> > gpart recover md1   # fix up gpt
> > gpart resize -i 3 md1   # expand ufs 
> > partition
> > growfs md1p3# grow ufs
> > 
> > Note that mdconfig may output a different md device than md1, use
> > that instead.
> 
> NICE!  Maybe this could be a second example in growfs(8) as
> an example for "How to prepare an image for use with emulation"?

In most/all cases for images, just a truncate is needed as Glen
mentioned.  The growfs(7) rc script is run at first boot to size
up the root fs as needed.  It can always be run manually later via:
service growfs onestart

if/when the size of the disk changes...

Hmm, just noticed that growfs(7) isn't linked from growfs(8), guess
I need to fix that.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: resizing aarch64 image on arm64

[John-Mark Gurney]

tech-lists wrote this message on Mon, Apr 05, 2021 at 16:59 +:
> Can anyone tell me how to resize
> https://download.freebsd.org/ftp/releases/VM-IMAGES/13.0-RC5/aarch64/Latest/FreeBSD-13.0-RC5-arm64-aarch64.raw
> 
> on an amd64 host?

note that this has been unxz'd already.

truncate -s 6g FreeBSD-13.0-RC5-arm64-aarch64.raw   # enlarge
mdconfig -f FreeBSD-13.0-RC5-arm64-aarch64.raw  # make device
gpart recover md1   # fix up gpt
gpart resize -i 3 md1   # expand ufs partition
growfs md1p3# grow ufs

Note that mdconfig may output a different md device than md1, use
that instead.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."


signature.asc
Description: PGP signature

Re: vm-bhyve No disk(s) present to configure

[John Doherty via freebsd-virtualization]

Thanks to all who responded. For the record, the solution was to add 
this:


disk0_type="virtio-blk"

to my /vmm/fbsd1/fbsd1.conf file.

I also added that to my /vmm/.templates/freebsd-zvol.conf.

On 16 Mar 2021, at 8:01, John Doherty via freebsd-virtualization wrote:

Hi, I am trying to get started with vm-bhyve and getting a little 
stuck. I am sure there are things I don't understand here so I hope 
you will bear with me.


I am using a machine with an AMD FX-8370E eight-core CPU, running 
12.2-RELEASE-p4. The machine boots from a ~60GB SATA SSD which is 
configured as the zroot zpool.


I have run a few bhyve vms on this machine before, including a couple 
of linux variants, by just using the bhyve command directly and they 
worked fine. In those cases, I just used files created with 
truncate(1) for vm disks but I would like to use zvols. vm-bhyve looks 
pretty nice to me and I think I'd like using it also.


I have this in /boot/loader.conf:

vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"

and this in /etc/rc.conf:

vm_enable="YES"
vm_dir="zfs:zroot/vmm"
vm_list=""
vm_delay="5"

And so I thus have this:

[root@sand] # zfs list -r zroot/vmm
NAMEUSED  AVAIL  REFER  MOUNTPOINT
zroot/vmm  1.43G  36.8G  1.43G  /vmm

and this:

[root@sand] # vm datastore list
NAMETYPEPATH  ZFS DATASET
default zfs /vmm  zroot/vmm

I have populated /vmm/.templates with the files from 
/usr/local/share/examples/vm-bhyve but made some changes to them. For 
example, the freebsd-zvol.conf template now looks like this:


loader="bhyveload"
cpu=1
memory=4096M
network0_type="virtio-net"
network0_switch="public"
disk0_dev="sparse-zvol"
disk0_name="disk0"

I've also populated /vmm/.iso with some ISO images including 
FreeBSD-12.2-RELEASE-amd64-disc1.iso.


So with that, I can for example do this:

[root@sand] # vm create -t freebsd-zvol fbsd1
[root@sand] # vm list
NAME   DATASTORE  LOADER CPU  MEMORY  VNC  AUTOSTART  STATE
fbsd1  defaultbhyveload  14096M   -No Stopped

and zfs things then look like this:

[root@sand] # zfs list -r zroot/vmm
NAMEUSED  AVAIL  REFER  MOUNTPOINT
zroot/vmm  1.43G  36.8G  1.43G  /vmm
zroot/vmm/fbsd1 164K  36.8G   108K  /vmm/fbsd1
zroot/vmm/fbsd1/disk056K  36.8G56K  -

So that all seems pretty good so far. Then I do this:

vm install -f fbsd1 FreeBSD-12.2-RELEASE-amd64-disc1.iso

The vm boots and I'm asked to choose a console type, I take the 
default vt100, and boom -- I get the familiar FreeBSD Installer 
screen. I proceed through that and try to do a Guided Root-on-ZFS 
installation, but when I choose Pool Type/Disks, I'm told "No disk(s) 
present to configure."


So. Pretty clearly doing something wrong here but I'm not sure what. 
Happy to provide any further info that I might have neglected above. 
Any advice that points me in the right direction much appreciated. 
Thanks.


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

vm-bhyve No disk(s) present to configure

[John Doherty via freebsd-virtualization]

Hi, I am trying to get started with vm-bhyve and getting a little stuck. 
I am sure there are things I don't understand here so I hope you will 
bear with me.


I am using a machine with an AMD FX-8370E eight-core CPU, running 
12.2-RELEASE-p4. The machine boots from a ~60GB SATA SSD which is 
configured as the zroot zpool.


I have run a few bhyve vms on this machine before, including a couple of 
linux variants, by just using the bhyve command directly and they worked 
fine. In those cases, I just used files created with truncate(1) for vm 
disks but I would like to use zvols. vm-bhyve looks pretty nice to me 
and I think I'd like using it also.


I have this in /boot/loader.conf:

vmm_load="YES"
nmdm_load="YES"
if_tap_load="YES"
if_bridge_load="YES"

and this in /etc/rc.conf:

vm_enable="YES"
vm_dir="zfs:zroot/vmm"
vm_list=""
vm_delay="5"

And so I thus have this:

[root@sand] # zfs list -r zroot/vmm
NAMEUSED  AVAIL  REFER  MOUNTPOINT
zroot/vmm  1.43G  36.8G  1.43G  /vmm

and this:

[root@sand] # vm datastore list
NAMETYPEPATH  ZFS DATASET
default zfs /vmm  zroot/vmm

I have populated /vmm/.templates with the files from 
/usr/local/share/examples/vm-bhyve but made some changes to them. For 
example, the freebsd-zvol.conf template now looks like this:


loader="bhyveload"
cpu=1
memory=4096M
network0_type="virtio-net"
network0_switch="public"
disk0_dev="sparse-zvol"
disk0_name="disk0"

I've also populated /vmm/.iso with some ISO images including 
FreeBSD-12.2-RELEASE-amd64-disc1.iso.


So with that, I can for example do this:

[root@sand] # vm create -t freebsd-zvol fbsd1
[root@sand] # vm list
NAME   DATASTORE  LOADER CPU  MEMORY  VNC  AUTOSTART  STATE
fbsd1  defaultbhyveload  14096M   -No Stopped

and zfs things then look like this:

[root@sand] # zfs list -r zroot/vmm
NAMEUSED  AVAIL  REFER  MOUNTPOINT
zroot/vmm  1.43G  36.8G  1.43G  /vmm
zroot/vmm/fbsd1 164K  36.8G   108K  /vmm/fbsd1
zroot/vmm/fbsd1/disk056K  36.8G56K  -

So that all seems pretty good so far. Then I do this:

vm install -f fbsd1 FreeBSD-12.2-RELEASE-amd64-disc1.iso

The vm boots and I'm asked to choose a console type, I take the default 
vt100, and boom -- I get the familiar FreeBSD Installer screen. I 
proceed through that and try to do a Guided Root-on-ZFS installation, 
but when I choose Pool Type/Disks, I'm told "No disk(s) present to 
configure."


So. Pretty clearly doing something wrong here but I'm not sure what. 
Happy to provide any further info that I might have neglected above. Any 
advice that points me in the right direction much appreciated. Thanks.


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Warm Migration feature for bhyve - review on Phabricator

[John-Mark Gurney]

Willem Jan Withagen wrote this message on Thu, Jan 28, 2021 at 01:42 +0100:
> On 25-1-2021 19:42, John-Mark Gurney wrote:
> > Matt Churchyard wrote this message on Mon, Jan 25, 2021 at 10:46 +:
> >> -Original Message-----
> >> From: John-Mark Gurney 
> >> Sent: 25 January 2021 06:21
> >> To: Matt Churchyard 
> >> Cc: Elena Mihailescu ; 
> >> freebsd-virtualization@freebsd.org
> >> Subject: Re: Warm Migration feature for bhyve - review on Phabricator
> >>
> >>> There's also hastd which can aid with this...
> >> Thanks for the reply. I've always been wary of the additional complexity 
> >> of HAST and ZFS, as it doesn't seem to have widespread usage or support, 
> >> and things get ugly fast when storage is involved.
> > Totally agree...
> >
> >> However, the idea of using HAST on top of zvols to provide network 
> >> mirrored storage for a guest is interesting. It adds a lot of extra 
> >> complexity, and probably performance impact though if it's just for the 
> >> ability to move a guest between systems that may only happen every now and 
> >> then. I'm also not sure it would help (or would at least be even more 
> >> complex) if I have 4 hosts and want to be able to move guests anywhere.
> > gmirror + ggate is another option as well...
> 
> I did try all kinds of these solutions that are based on a 2 system 
> backend storage, and it was relatively easy for me to get the state of 
> the storage in a split-brain situation which is not really repairable 
> without a lot of manual user intervention.
> So I gave up on HAST and gmirror over ggate.

Can you explain more about the split brain w/ gmirror+ggate?  If writes
didn't happen to one of the members of gmirror, then that should be
investigated.. I guess I could see a situation where ggate got behind
on writes (connection drop), and gmirror didn't kick out the ggate
member, or failed to mark it as stale, but those are bugs that need
to be identified and fixed...

I'm partly currious about the ggate, because I've recently done some
work w/ ggate (performance improvements and working of sftp), and it's
simple enough that issues are a bit surprising..

I'm not at all familar w/ HAST to advice on any issues w/ that...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Warm Migration feature for bhyve - review on Phabricator

[John-Mark Gurney]

Matt Churchyard wrote this message on Mon, Jan 25, 2021 at 10:46 +:
> -Original Message-
> From: John-Mark Gurney  
> Sent: 25 January 2021 06:21
> To: Matt Churchyard 
> Cc: Elena Mihailescu ; 
> freebsd-virtualization@freebsd.org
> Subject: Re: Warm Migration feature for bhyve - review on Phabricator
> 
> Matt Churchyard wrote this message on Fri, Jan 22, 2021 at 10:09 +:
> > > Hello, all,
> > 
> > > We have recently opened a review on Phabricator for the warm migration 
> > > code for > bhyve [1]. Please take a look and let us know if it is 
> > > anything we can improve.
> > 
> > > [1] https://reviews.freebsd.org/D28270
> > 
> > > Thank you,
> > > Elena
> > 
> > I appreciate that this isn't really related to the current review, and 
> > commend the work being put into bhyve - it's an invaluable addition to 
> > FreeBSD. I'm just wondering if any thought has been put into the future 
> > possibility for transfer of disk data during migration (i.e. the equivalent 
> > of "storage vmotion")
> > 
> > The current process (from a mile high overview) effectively seems to be the 
> > following -
> > 
> > * Start guest on host2 pointing at a shared disk, and halt any execution
> > * use bhyvectl to pause and begin migration on host1
> > * Start the guest on host2
> > 
> > What would be the feasibility of being able to run a process such as the 
> > following? Obviously it would likely need to be orchestrated by an external 
> > tool, but to me it seems the main requirement is really just to be able to 
> > provide separate control over the pause and migrate steps on host1 -
> > 
> > * send a ZFS snapshot of the running machine to host2
> > * start the guest in migrate recv mode on host2
> > * pause the guest on host1
> > * send a new snapshot
> > * initiate the migration of memory/device data
> > * start guest on host2
> > 
> > Are there any major complications here I'm not aware of other than the 
> > requirement to pause the guest and kick off the state migration as two 
> > separate calls?
> 
> > There's also hastd which can aid with this...
> 
> Thanks for the reply. I've always been wary of the additional complexity of 
> HAST and ZFS, as it doesn't seem to have widespread usage or support, and 
> things get ugly fast when storage is involved.

Totally agree...

> However, the idea of using HAST on top of zvols to provide network mirrored 
> storage for a guest is interesting. It adds a lot of extra complexity, and 
> probably performance impact though if it's just for the ability to move a 
> guest between systems that may only happen every now and then. I'm also not 
> sure it would help (or would at least be even more complex) if I have 4 hosts 
> and want to be able to move guests anywhere.

gmirror + ggate is another option as well...

> The main reason for the email was to explore my theory that, by leveraging 
> ZFS, any bhyve user could roll their own storage migration ability with a few 
> commands as long as the following two (seemingly minor) abilities were present
> 
> 1) the ability to suspend a guest without it being done as part of the 
> migrate call. (I assume suspend/resume support via bhyvectl is planned 
> anyway, if not already in place at this point)

Yeah, I'd hope that there would be ability to insert custom commands
in between the suspend on one host and the resumption on the new host..
But it is good explicitly state this..

> 2) modification of the migrate call to skip suspend if the guest is already 
> suspended.
> 
> The main thing I'm not sure of is whether the migrate call has any specific 
> reliance on doing the suspend itself (e.g. if it needs to do anything before 
> the suspend, which will obviously be problematic if suspend & migrate are 
> called separately). Or if there's something else I missed that means this is 
> not feasible. I'm not really after massive changes to the current review to 
> implement disk migration in bhyve itself.

+1

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: RHEL virtualization

[John Kennedy]

On Sat, Jan 23, 2021 at 03:14:53PM -0800, John Kennedy wrote:
> At work, we have RHEL (-ish; some RHEL, some CentOS, some OEL).  Mostly v7,
> some v8.  Since I'm doing the Covid work-from-home telecommute, I'm trying to
> recreate some of my work infrastructure while trying to plan a bit towards
> the future (migrating a lot of VMs to Azure).
> 
> What I'd like to recreate is my existing kickstart infrastructure, where I
> PXE boot the system, feed it anaconda goodness which dovetails into puppet
> and I can generate a clean system from a template.  Works great for VMWare
> and HyperV, not so much for Azure but if I can generate a snapshot disk
> image Azure can ingest, I'll be happy on that score.
> 
> I've been very happy with bhyve for FreeBSD.  I messed with VirtualBox for
> a while (a long time ago), but with my tendency to track stable (think:
> kernel modules) and keep very current on ports-from-source (frequent
> package updates, upon which VirtualBox has MANY dependencies) made that a
> poorer experience than I had with it on Windows.  I've been very happy with
> bhyve since it's basically baked right in.

  Let me restate some of this in a different way to maybe get some more
thinking.

  Using the BHYVE_UEFI.fd from uefi-edk2-bhyve, I can boot my OEL8 (RHEL8
clone).  That currently worries me because it has the big python-2.7 warning
on it (as does uefi-edk2-bhyve-csm).  On physical boxes, I've been able to
grab a PXEBOOT ISO when the firmware lacks PXE booting, but I haven't got
that to work yet for these.  Those python worries are basically what is
driving me to look elsewhere (like fighting with grub-bhyve and away from
the only UEFI booting that I know about).


  I personally like PXE-booting a new system (and possibly making a gold image
from that, depending on what I'm doing) because it basically answers that
little auditor-voice in the back of my head that, in the event of some possible
security problem, how do I know that my backups haven't been compromised.  In
all of those gigabytes, after all of the toxic recursive mindless non-logic,
how do you *know*?  My happy answer to myself is: "here is a configuration
file that I can review, all the binaries are on the vendor's site or
re-downloaded, here are the puppet customization rules, blam!  done!
10 minutes later I have a clean system."

  In any case, that is why I'm chasing PXE booting, although I'd be interested
in the way other people solve that problem.  That really doesn't work that
way in Azure, thus the gold images approach I'll probably have to take with
them in the future.

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Warm Migration feature for bhyve - review on Phabricator

[John-Mark Gurney]

Matt Churchyard wrote this message on Fri, Jan 22, 2021 at 10:09 +:
> > Hello, all,
> 
> > We have recently opened a review on Phabricator for the warm migration code 
> > for > bhyve [1]. Please take a look and let us know if it is anything we 
> > can improve.
> 
> > [1] https://reviews.freebsd.org/D28270
> 
> > Thank you,
> > Elena
> 
> I appreciate that this isn't really related to the current review, and 
> commend the work being put into bhyve - it's an invaluable addition to 
> FreeBSD. I'm just wondering if any thought has been put into the future 
> possibility for transfer of disk data during migration (i.e. the equivalent 
> of "storage vmotion")
> 
> The current process (from a mile high overview) effectively seems to be the 
> following -
> 
> * Start guest on host2 pointing at a shared disk, and halt any execution
> * use bhyvectl to pause and begin migration on host1
> * Start the guest on host2
> 
> What would be the feasibility of being able to run a process such as the 
> following? Obviously it would likely need to be orchestrated by an external 
> tool, but to me it seems the main requirement is really just to be able to 
> provide separate control over the pause and migrate steps on host1 -
> 
> * send a ZFS snapshot of the running machine to host2
> * start the guest in migrate recv mode on host2
> * pause the guest on host1
> * send a new snapshot
> * initiate the migration of memory/device data
> * start guest on host2
> 
> Are there any major complications here I'm not aware of other than the 
> requirement to pause the guest and kick off the state migration as two 
> separate calls?

There's also hastd which can aid with this...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

RHEL virtualization

[John Kennedy]

At work, we have RHEL (-ish; some RHEL, some CentOS, some OEL).  Mostly v7,
some v8.  Since I'm doing the Covid work-from-home telecommute, I'm trying to
recreate some of my work infrastructure while trying to plan a bit towards
the future (migrating a lot of VMs to Azure).

What I'd like to recreate is my existing kickstart infrastructure, where I
PXE boot the system, feed it anaconda goodness which dovetails into puppet
and I can generate a clean system from a template.  Works great for VMWare
and HyperV, not so much for Azure but if I can generate a snapshot disk
image Azure can ingest, I'll be happy on that score.

I've been very happy with bhyve for FreeBSD.  I messed with VirtualBox for
a while (a long time ago), but with my tendency to track stable (think:
kernel modules) and keep very current on ports-from-source (frequent
package updates, upon which VirtualBox has MANY dependencies) made that a
poorer experience than I had with it on Windows.  I've been very happy with
bhyve since it's basically baked right in.


That being said, RHEL on bhyve has been a pain to figure out.  The best I've
done so far is using sysutils/grub2-bhyve to set up the boot CD, using
BHYVE_UEFI.fd as UEFI firmware (sysutils/bhyve-firmware I think) and then
getting at the console via net/tigervnc-viewer.

Currently I'm fighting grub-bhyve's issue finding the kernel to load (if I'm
finding the right problem reports, it doesn't seem to like modern XFS or
ext4 partitions).  I couldn't get net/ipxe to PXE boot anything, and I din't
manage to get very far with sysutils/uefi-edk2-bhyve.  And of course some
of these are flagged with python2.7 isses.

I'm not a fan of grub-bhyve, but that's mostly because you have to specify
the full kernel-with-version path (changes every kernel update), but I
figure I could make an expect-script that would figure it out if I could
find a /boot filesystem type that grub-bhyve could "ls" properly.


Ignoring my own setup details right now, what would someone currently
bhyving RHEL recommend that I be doing right now?

There is so much old information/documentation out there that I'm really
second-guessing myself and probably chasing a bunch of dying ports.  But
someone on here must be happy with what they've got going for them.

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: resizing a FreeBSD-12.1 vm on Azure

[John Kennedy]

On Wed, Dec 23, 2020 at 01:48:09PM +, Wei Hu via freebsd-virtualization 
wrote:
> Hello,
> 
> You can use resize option on Azure portal to increase the CPU and RAM of your 
> VM. I believe you don't need to shut down the VM. Once you start the 
> resizing, Azure will take care of rebooting the VM for you. The public IP 
> will not change after the resizing.
> 
> If you want to grow the disk size, you will have to shut  down the VM. Find 
> the disk for your VM on the portal. It should have option to grow the size. 
> You may need to increase the FreeBSD filesystem size manually once the guest 
> boots up again. I have not done this. I just heard from someone who has done 
> this for Linux VM on Azure.

  I don't have a FreeBSD azure host, but I do have linux azure hosts.  Yes, you
can try to resize on the fly but there is a big banner up top that says:

"If the virtual machine is currently running, changing its size will 
cause it to be restarted. Stopping the virtual machine may reveal additional 
sizes."

  My general experience with other hypervisors (HyperV, VMWare) and linux is
that you can't resize CPU and memory without rebooting the host.  For disks
though, I've done that a lot.  All my resizes on azure have, so far, required
reboots.  To be fair, I'm usually starting out with something small and
then resizing it something appropriate (like a D2s_v3 -> DS3_v2), so there
might be some resizing that wouldn't require a reboot.

  If you've got the VM tools installed on linux, it looks like it hooks the
systemd process for orderly shutdown.  Not sure what the equivalent for
FreeBSD would be, but if "shutdown -r now" does an orderly shutdown for you
(be careful...  some things like databases interpret that as a request to
do a quick-crash-stop), you're probably Ok.

> > -Original Message-
> > From: owner-freebsd-virtualizat...@freebsd.org  > virtualizat...@freebsd.org> On Behalf Of tech-lists
> > Sent: Tuesday, December 15, 2020 11:57 AM
> > To: freebsd-virtualization@freebsd.org
> > Subject: resizing a FreeBSD-12.1 vm on Azure
> > 
> > Hello,
> > 
> > Has anyone ever resized a freebsd vm on Azure? I don't mean just the disks.
> > Really the whole point is to have more CPU and RAM. I have the option to do
> > this via their portal, just select whats required and hit 'resize' but I'm 
> > a bit
> > hesitant. I guess it'll work on a windows VM but anything else?
> > 
> > If you did manage to do this, did you need to shut the VM down before 
> > resizing
> > it? Was your external IP preserved or did it force you to have another one? 
> > Did
> > you have to run anything else, like growfs?
> > 
> > The author of https://forums.freebsd.org/threads/azure-vm.62440/ tried to do
> > this in 2017 but wasn't able to at the time.

  All the IPs seem to survive reboots and even extended shutdown as long as
you don't delete the network interface.  If you delete the network interface,
azure acts like it is throwing it back into the free pool and your chances
of getting the same IP back are probably pretty slim.  I've staticly assigned
private IPs via the portal (which it'll let you do as long as nobody else has
it assigned), but never tried that with a public IP.

  If you're just trying to play with RAM+CPU for some temporary workload and
you don't want to play with disk at all because you're going to reduce it later
then you won't need to worry about disk at all.  If not...


  On linux, when I resize disk without rebooting, I try to set it up so that
my disks are just XFS with no partitions on anything I know will grow.  A non-
issue with HyperV and VMWare where I can have lots of individual disks.  In
azure, you're sort of motivated to have everything on one disk, both for IOPS
(you get more as the disk size goes up) and cost (you're charged in ^2 size
ranges).  That means partitions.

  For linux, I just lump everything dynamic into the last partition on the
disk.  When I resize live, I delete the entire partition (aiugh!), then
recreate it making sure the type and start match up to what it used to be.
If you haven't rebooted, you may need to partprobe or done your SCSI rescan
tricks to get the OS to notice that you have more space, extra disks, etc.
Then you grow the filesystem (xfs_growfs doesn't require reboot) and you're
good!  If your heart still isn't in your teeth for deleting a live partition
while the OS is using it.

  For FreeBSD, I'm almost always using ZFS.  In the cloud, I'm not using an
encrypted GELI disk.  That has been a show-stopper for me in home testing
because you somehow need to resize that before resizing.  For my Digital
Ocean VM, I think I just tracked down the auto-resize on boot option and
things generally worked.  In Azure, it seems like you would want to make
sure that is disabled.  With new native-ZFS-encryption, maybe non-issue?

  I don't think azure in any way bundles CPU+RAM sizing with disk, so I don't
thing you have to worry about any irreversible upsizing 

Re: When is a switch not a switch?

[John-Mark Gurney]

D'Arcy Cain wrote this message on Tue, Oct 20, 2020 at 05:28 -0400:
> On 10/20/20 4:36 AM, Patrick M. Hausen wrote:
> > It's officially documented here:
> > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-bridging.html
> 
> I did see that.  Does that mean that I don't even need to create switches at 
> all?
> 
> > "If the bridge host needs an IP address, set it on the bridge interface, 
> > not on the member interfaces."
> 
> But I don't necessarily need an IP on the bridge itself, right?

I can't say w/o more info in exactly how your system is setup...  If
you could provide more concrete information about what interfaces you
are using, and what IPs are configured on what interfaces (or VMs),
that'd be helpful.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."


signature.asc
Description: PGP signature

Re: When is a switch not a switch?

[John-Mark Gurney]

D'Arcy Cain wrote this message on Mon, Oct 19, 2020 at 22:02 -0400:
> I am using bhyve with vm-bhyve,  I am trying to set up a virtual network 
> with multiple hosts.  The idea is that a VM would be on the same virtual 
> network no matter which actual host it is on.
> 
> Say I have a public network a.b.c.0/24.  I thought I could create a switch 
> on a host.  The host would be a.b.c.1 and the VMs would be a.b.c.100 and 
> a.b.c.101.  The idea would be that the VMs would appear on the real network. 
>   Then the 101 VM could migrate to a.b.c.2 and still be accessible.  I 
> envisioned some sort of proxy arp would happen so that every VM would simply 
> announce itself wherever it was.
> 
> This did seem to work in that I could ping from the VM:
> 
> # ping 8.8.8.8
> PING 8.8.8.8 (8.8.8.8): 56 data bytes
> 64 bytes from 8.8.8.8: icmp_seq=0 ttl=114 time=1.734 ms
> 
> Even IPV6:
> 
> # ping6 2605:2600:1001::4b
> PING6(56=40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b
> 16 bytes from 2605:2600:1001::4b, icmp_seq=0 hlim=64 time=0.960 ms
> 16 bytes from 2605:2600:1001::4b, icmp_seq=1 hlim=64 time=0.415 ms
> 
> However TCP doesn't work.  In fact, I could only ping by IP because the 
> system couldn't connect to the DNS server, to get an address even though it 
> could ping it.
> 
> I guess my first question is does this seem doable?  If so, what am I 
> missing?  Is it possible that a bhyve switch is more like a router?

By switch, do you mean use bridge?  How specifically is the network
configured?

What you are describing sounds like what I do w/ bridge, but my use was
slightly more complicated.

Say your host has em0 as the main network, you would create a bridge0
interface, either via cloned_interfaces or via "ifconfig bridge0 create".
Then you would put the em0 interface as a member of the bridge
interface.  You would also add the tap interfaces of the various bhyve
vms as well (don't forget to make sure the tap interface is up on the
host, net.link.tap.up_on_open helps w/ this)...

I have heard (and that is the way I do that), that you have to put the
host IPs on the bridge0 interface, and not the em0 interface.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."


signature.asc
Description: PGP signature

bhyve configuration changes up for review

[John Baldwin]

I have been toying with a new approach to managing bhyve's
internal configuration for a while.  Internally it stores a
little configuration "database" of key-value pairs that is
stored in a hierarchical tree (using nvlist).  One of the
ideas is that the configuration for a given device model
would consist of keys hung off of a node and the device init
routines would be passed a reference to that node and lookup
the desired keys as relative values.

There is more detail in the review, but I wanted to give a
heads up on the list as well in case folks would like to test
it.  I think comments on the design and other review notes
probably belong on the review rather than this thread.

Note that right now it only supports a simple flat config
file, but it should be quite possible to support a nicer
UCL config file syntax where the parser of that configuration
file sets a bunch of configuration variables just as the
parser of the flat config file does.  There might need to be
some additional logic for a nice UCL syntax for things like
choosing PCI slots if an explicit one isn't given, etc. but
that should be doable.

https://reviews.freebsd.org/D26035

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Restricting IP ranges for guests over tap devices

[John-Mark Gurney]

Mark Raynsford via freebsd-virtualization wrote this message on Sat, Aug 01, 
2020 at 14:51 +:
> Let's say I have a machine running a few dozen bhyve guests. Each bhyve
> guest gets its own tap device, and all of the tap devices are connected
> to a bridge.
> 
> Everything works fine. I can write pf rules that control access between
> each guest, and between each guest and the world. I can't directly
> observe the IP addresses that the guests have assigned to the tap
> devices I gave them, but if I know the addresses beforehand, I can for
> example write pf rules that say things like:
> 
>   block log all
>   pass in on tap23 proto tcp \
> from any to $guest_23_ip port ssh modulate state
> 
> That then means that even if the guest is compromised and tries to bind
> a server to another address, the pf rules won't allow anyone else to
> actually connect to it.
> 
> The good thing about this is also the bad thing about this; I have to
> write specific rules that say "only allow access to this specific IP
> via this specific tap device". Over dozens of guests, that can multiply
> to hundreds of laboriously maintained rules.
> 
> Is there some more general way I can supply a mapping between tap
> devices and allowed addresses? Remember that pf can't see the guest
> addresses on the host sides of the tap devices, so I can't use the
> (device) syntax to expand to "the address of a NIC called 'device'".
> 
> I can generate rule sets, but perhaps there's something "better"[0]? The
> documentation isn't suggesting much.
> 
> [0] Better in the sense that, for example, a table is usually better
> than a massive list of macros. :)

Don't think there is anything better...

bridge does have sticky that binds the mac address to an interface, but
that doesn't deal w/ IP ARP.

One issue w/ this is how do you know the difference between one machine
that's been down for a long time, and an attacking machine that takes
over the down'd machine's IP address?

I assume that these addresses are assigned via DHCP server, otherwise
if you are launching the VM's w/ known static IP's, you could use
pf's anchor directive, and each start/stop of a VM, update the rule for
that tap's anchor.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."


signature.asc
Description: PGP signature

Re: patch to make vboxvfs work again

[John-Mark Gurney]

Mateusz Guzik wrote this message on Sat, May 09, 2020 at 23:17 +0200:
> That will do the trick indeed.
> 
> I don't know about that bcmp thingy, ideally it would be needed. I
> presume there is a sufficient header mess which makes this to be the
> easiest hack.

Yeah.  It appears that they compile libraries w/o the kernel headers
and then link those files into the kernel module wrapper...

I did notice that we mistakenly still have a prototype for bcmp in
sys/systm.h even though that function get's #defined to a builtin
instead...

> On 5/9/20, John-Mark Gurney  wrote:
> > Hello virtualbox maintainer(s),
> >
> > Attached is a patch to make VirtualBox vboxvfs work again.  I've only
> > tested under -current.
> >
> > mjg, what is the correct way to make this patch work on past versions
> > of FreeBSD?  would a simple #ifdef VFS_VOP_VECTOR_REGISTER wrapper
> > around the call work?

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

patch to make vboxvfs work again

[John-Mark Gurney]

Hello virtualbox maintainer(s),

Attached is a patch to make VirtualBox vboxvfs work again.  I've only
tested under -current.

mjg, what is the correct way to make this patch work on past versions
of FreeBSD?  would a simple #ifdef VFS_VOP_VECTOR_REGISTER wrapper
around the call work?

Thanks.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
Index: virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_Makefile.kmk
===
--- virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_Makefile.kmk	(revision 534710)
+++ virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_Makefile.kmk	(working copy)
@@ -1,8 +1,10 @@
 src/VBox/Additions/freebsd/vboxvfs/Makefile.kmk.orig	2017-04-28 16:59:22.0 +0200
-+++ src/VBox/Additions/freebsd/vboxvfs/Makefile.kmk	2017-07-13 00:52:49.020669620 +0200
-@@ -33,32 +33,36 @@
+--- src/VBox/Additions/freebsd/vboxvfs/Makefile.kmk.orig	2019-10-10 18:06:51.0 +
 src/VBox/Additions/freebsd/vboxvfs/Makefile.kmk	2020-05-09 06:51:51.411749000 +
+@@ -32,33 +32,38 @@
+ . \
 $(vboxvfs_0_OUTDIR)
  vboxvfs_SOURCES   = \
++	bcmp.c \
  	vboxvfs_vfsops.c \
 -	vboxvfs_vnops.c
 +	vboxvfs_vnops.c \
Index: virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_bcmp.c
===
--- virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_bcmp.c	(nonexistent)
+++ virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_bcmp.c	(working copy)
@@ -0,0 +1,12 @@
+--- src/VBox/Additions/freebsd/vboxvfs/bcmp.c.orig	2020-05-09 06:57:06.22914 +
 src/VBox/Additions/freebsd/vboxvfs/bcmp.c	2020-05-09 06:54:55.886751000 +
+@@ -0,0 +1,9 @@
++#include 
++
++int bcmp(const void *b1, const void *b2, size_t len);
++
++int
++bcmp(const void *b1, const void *b2, size_t len)
++{
++	return __builtin_memcmp((b1), (b2), (len));
++}

Property changes on: virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_bcmp.c
___
Added: fbsd:nokeywords
## -0,0 +1 ##
+yes
\ No newline at end of property
Added: svn:eol-style
## -0,0 +1 ##
+native
\ No newline at end of property
Added: svn:mime-type
## -0,0 +1 ##
+text/plain
\ No newline at end of property
Index: virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_vboxvfs__vnops.c
===
--- virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_vboxvfs__vnops.c	(revision 534793)
+++ virtualbox-ose/files/patch-src_VBox_Additions_freebsd_vboxvfs_vboxvfs__vnops.c	(working copy)
@@ -1,5 +1,5 @@
 src/VBox/Additions/freebsd/vboxvfs/vboxvfs_vnops.c.orig	2019-10-10 18:06:51 UTC
-+++ src/VBox/Additions/freebsd/vboxvfs/vboxvfs_vnops.c
+--- src/VBox/Additions/freebsd/vboxvfs/vboxvfs_vnops.c.orig	2019-10-10 18:06:51.0 +
 src/VBox/Additions/freebsd/vboxvfs/vboxvfs_vnops.c	2020-05-09 07:07:49.485808000 +
 @@ -1,10 +1,6 @@
 -/* $Id: vboxvfs_vnops.c $ */
 -/** @file
@@ -12,7 +12,7 @@
   *
   * This file is part of VirtualBox Open Source Edition (OSE), as
   * available from http://www.virtualbox.org. This file is free software;
-@@ -14,228 +10,1338 @@
+@@ -14,228 +10,1339 @@
   * VirtualBox OSE distribution. VirtualBox OSE is distributed in the
   * hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
   */
@@ -172,6 +172,7 @@
 +	.vop_write	= vboxfs_write,
 +	.vop_bmap	= VOP_EOPNOTSUPP
  };
++VFS_VOP_VECTOR_REGISTER(vboxfs_vnodeops);
  
 -static int vboxvfs_access(struct vop_access_args *ap)
 +static uint64_t
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve: TRIM support in AHCI controller

[John-Mark Gurney]

Allan Jude wrote this message on Sat, Apr 11, 2020 at 11:15 -0400:
> On 2020-03-19 08:54, Wanpeng Qian wrote:
> > Hi all
> > 
> > I found the output of diskinfo inside VM of AHCI disk as follow, no
> > TRIM support.
> > root@smart:~ #  diskinfo -v ada0
> > ada0
> > 512  # sectorsize
> > 21474836480 # mediasize in bytes (20G)
> > 41943040# mediasize in sectors
> > 131072  # stripesize
> > 0# stripeoffset
> > 41610# Cylinders according to firmware.
> > 16  # Heads according to firmware.
> > 63  # Sectors according to firmware.
> > BHYVE SATA DISK # Disk descr.
> > BHYVE-0B98-104F-E9A7 # Disk ident.
> > No  # TRIM/UNMAP support
> > 0# Rotation rate in RPM
> > Not_Zoned# Zone Mode
> > 
> > Also camcontrol identify ada0 -v show not support of TRIM
> > 
> > Data Set Management (DSM/TRIM) no
> > 
> > I check to pci_ahci.c, TRIM support is decided by block_if.c's candelete 
> > flag.
> > while candelete is decided here
> > 
> > strlcpy(arg.name, "GEOM::candelete", sizeof(arg.name));
> > arg.len = sizeof(arg.value.i);
> > if (ioctl(fd, DIOCGATTR, ) == 0)
> > candelete = arg.value.i;
> > 
> > Obvious file backend will not have the candelete flag.
> > 
> > If the backend storage support TRIM, I think we should present TRIM
> > ability to VM.
> > Any better idea to check whether backend storage support TRIM or not,
> > so we can enable it.
> > 
> > Thanks.
> > ___
> > freebsd-virtualization@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> > To unsubscribe, send any mail to 
> > "freebsd-virtualization-unsubscr...@freebsd.org"
> > 
> 
> I have updated my patch to add TRIM support to the bhyve block interface
> here:
> 
> https://reviews.freebsd.org/D21707
> 
> I am working on an update to the virtio driver, to make FreeBSD guests
> able to TRIM if the hypervisor supports it as well:
> 
> https://reviews.freebsd.org/D21708

One comment on that patch is that it looks like there are a number of
white space and definitions (unrelated to the patch) that are included
in the patch.  It'd be good to break out those fixes and just commit
them..  That would reduce the size of the patch a bit.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."


signature.asc
Description: PGP signature

Re: bhyve: passthrough SMART info from host nvme controller

[John-Mark Gurney]

Wanpeng Qian wrote this message on Thu, Mar 19, 2020 at 12:09 +0900:
> > Can't you do what something like pci_passthru.c does in passthru_init,
> > and open /dev/nvme0 in pci_nvme_init?
> 
> Yes, you are correct. but that will make /dev/nvme0 keep open all the time.
> I just thinking when guest fire a logpage command, open the /dev/nvme0
> and get the SMART info. then close /dev/nvme0.

But as you point out, the only way to have that happen is to remove
capsicum, but that would make byhve overall LESS secure.

Even if you were able to open /dev/nvme0 at any time after the process
was locked down, it doesn't provide additional security, as any
attacker could just open it up, and do the operation...

So, I'm really confused as to what the benefit of not opening it at
the start is..

> On Thu, Mar 19, 2020 at 2:59 AM John-Mark Gurney  wrote:
> >
> > Wanpeng Qian wrote this message on Wed, Mar 18, 2020 at 13:05 +0900:
> > > But currently bhyve has Capsicum capability, I cannot
> > > open /dev/nvme0 within pci_nvme.c without extra code.
> > > (currently I just disable the Capsicum capability)
> > >
> > > any feedback?
> >
> > Can't you do what something like pci_passthru.c does in passthru_init,
> > and open /dev/nvme0 in pci_nvme_init?
> >
> > Or am I missing something?

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve: passthrough SMART info from host nvme controller

[John-Mark Gurney]

Wanpeng Qian wrote this message on Wed, Mar 18, 2020 at 13:05 +0900:
> But currently bhyve has Capsicum capability, I cannot
> open /dev/nvme0 within pci_nvme.c without extra code.
> (currently I just disable the Capsicum capability)
> 
> any feedback?

Can't you do what something like pci_passthru.c does in passthru_init,
and open /dev/nvme0 in pci_nvme_init?

Or am I missing something?

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: RFC: in-guest CPU tick counters

[John-Mark Gurney]

Robert Crowston via freebsd-virtualization wrote this message on Wed, Jan 01, 
2020 at 16:28 +:
> I have written a patch that exposes to a sysctl the ticks spent by each CPU 
> executing guest code under bhyve.
> 
> This new feature differs from the existing vmm_stats code in that the 
> existing code expresses per-vm information on a per-virtual-cpu basis. This 
> new patch provides a per-physical-cpu counter of the aggregate number of 
> ticks dedicated to executing guest-mode code since the vmm.ko module was 
> loaded. Following the example of kern.cp_times, it is expressed as a new 
> sysctl ("hw.vmm.stat.guest_ticks") rather than a system call, but I am not 
> particularly attached to that.
> 
> The diffs may be seen here:
> https://github.com/freebsd/freebsd/compare/master...RobCrowston:vmm-host-stats
> 
> I am looking for comments. If the general idea meets with this list's 
> approval, I will submit it to phabricator.
> 
> The new data are intended to be used by utilities like htop. To that end, I 
> have written a separate patch for htop that colours the CPU bars to express 
> the time spent for guest execution. After re-compiling htop, enable it with 
> F2 ("Setup") -> "Display Options" -> "Detailed CPU time 
> (System/IO-Wait/Hard-IRQ/Soft-IRQ/Steal/Guest)". Presently, htop only 
> supports this feature for Linux hosts. If the above patch is accepted, I will 
> submit the following patch to htop separately:
> https://github.com/hishamhm/htop/compare/master...RobCrowston:freebsd-vmm-counter-1

I'm worried about the security implications of this.

It is likely to expose a side channel attack on the guest machines
from a normal user on the host if these cpu cycle counters are exposed
to all users, which is the default.

The counters should be restricted on who has access to them, and only
allowing root, or the owner of the guest vmm to access them.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: sysutils/grub2-bhyve: CURRENT build breaks on "gets" security hole

[John D Groenveld]

In message , =?UTF-8?Q?S=c3=a9r
gio_Siegrist?= writes:
>grub2-bhyve fails to compile with this error:
>gcc8 -DHAVE_CONFIG_H -I. -I../.. -I../../intl -MT argp-eexst.o -MD -MP 
>-MF .deps/argp-eexst.Tpo -c -o argp-eexst.o argp-eexst.c
>In file included from argp.h:22,
>from argp-eexst.c:25:
>./stdio.h:456:1: error: 'gets' undeclared here (not in a function); did 
>you mean 'getw'?
>_GL_WARN_ON_USE (gets, "gets is a security hole - use fgets instead");
>^~~
>gmake[5]: *** [Makefile:1030: argp-eexst.o] Error 1
>
>
>I suppose replacing 'gets' with 'fgets' should fix it.

I think the compiler is flagging the attempt in stdio.h to flag
the use of gets(3) as gets(3) has been removed from the C standard
and the C library. 

The package builds without the check.
$ cat files/patch-grub-core_gnulib_stdio.in.h
@@ -137,12 +137,6 @@ _GL_WARN_ON_USE (fflush, "fflush is not always POSIX 
compliant - "
  "use gnulib module fflush for portable POSIX compliance");
 #endif

-/* It is very rare that the developer ever has full control of stdin,
-   so any use of gets warrants an unconditional warning.  Assume it is
-   always declared, since it is required by C89.  */
-#undef gets
-_GL_WARN_ON_USE (gets, "gets is a security hole - use fgets instead");
-
 #if @GNULIB_FOPEN@
 # if @REPLACE_FOPEN@
 #  if !(defined __cplusplus && defined GNULIB_NAMESPACE)

John
groenv...@acm.org
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: New video BIOS for bhyve

[John Baldwin]

On 7/19/19 3:02 AM, Henrik Gulbrandsen wrote:
> In the last few months, I have been extending the video support in bhyve
> to allow booting from live ISO images, which sometimes lack UEFI 
> loaders.
> There was already a video BIOS with dormant support for VGA, but I could
> not use that, because the X11 int10 module runs a real-mode x86 emulator
> that aborts when it tries to enter the C code running in protected mode.
> 
> Instead, I added support for PCI Expansion ROMs to bhyve and wrote a new
> SVGA video BIOS from scratch in 8086 assembly code. The current features
> include VGA modes 3 and 12h, 24-bit and 32-bit VESA modes from 640x480 
> to
> 1920x1200, and enough BIOS functions to boot a typical Linux desktop.
> 
> At this point, I think it's good enough for a preliminary release, so if
> you are interested, you will find all patch files at the following URL:
> 
>  https://www.gulbra.net/freebsd-bhyve/
> 
> My apologies for the 15-year-old web layout, which will probably break
> horribly on modern smartphones. Also, I hope I have not duplicated the
> work of someone else. I didn't want to announce this before it worked.

This is pretty impressive work!  Would you be able to post these patches to
a public repository, e.g. on github or gitlab or some such?  That would
really help with review and allow others to assist with the work as well.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

New "bhyve" bugzilla component

[John Baldwin]

bugmeister@ has kindly added a new "bhyve" component for base system
bugs.  It would be great going forward for new bhyve-related bugs to
use this component.  Fixing the component of open bugs might also be
useful if you are updating them to add a comment, etc.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Windows 10 guests fail to boot when attempting to passthrough network card

[John Baldwin]

On 5/21/19 9:48 AM, Rodney W. Grimes wrote:
>> On 5/19/19 6:24 AM, Rodney W. Grimes wrote:
>>>> Does the card have multiple BARs? Can you send us the output of pciconf 
>>>> -lvb for the card?
>>>>
>>>> There is a bug in bhyve where if the OS assigns the 0x sentinel to 
>>>> more than one BAR simultaneously it is not compatible with bhyve?s memory 
>>>> model for the guest.
>>>
>>> The "we talked" below is myself and Nick talking about exactly that
>>> which came from jhb@ pointing out to me in a hall wall
>>> conversation here at BSDCan that this is the cause of this
>>> problem and is a known issue.
>>>
>>> Does anyone have fixes for this issue?
>>
>> I have not tested this at all, but I have a candidate patch to fix the root
>> issue at 
>> https://github.com/freebsd/freebsd/compare/master...bsdjhb:bhyve_passthrough_barsize
> 
> Could you please factor out the white space changes and do that
> seperately and sooner, it clouds the S/N in this review a bit.

I can pull those out, but this is just a testing patch still, not yet a commit
candidate.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Windows 10 guests fail to boot when attempting to passthrough network card

[John Baldwin]

On 5/19/19 6:24 AM, Rodney W. Grimes wrote:
>> Does the card have multiple BARs? Can you send us the output of pciconf -lvb 
>> for the card?
>>
>> There is a bug in bhyve where if the OS assigns the 0x sentinel to 
>> more than one BAR simultaneously it is not compatible with bhyve?s memory 
>> model for the guest.
> 
> The "we talked" below is myself and Nick talking about exactly that
> which came from jhb@ pointing out to me in a hall wall
> conversation here at BSDCan that this is the cause of this
> problem and is a known issue.
> 
> Does anyone have fixes for this issue?

I have not tested this at all, but I have a candidate patch to fix the root
issue at 
https://github.com/freebsd/freebsd/compare/master...bsdjhb:bhyve_passthrough_barsize

My first guess at the problem wasn't quite right (though Windows does indeed 
size
BARs in parallel).  For pass-through devices we aren't currently correctly 
disabling
access to a BAR when the guest disables it in the PCI command register which in
turn trips the assertion when using a Windows guest, but in general we need to 
be
honoring the enable bits when set or cleared by the guest.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: 12.0-RELEASE und kvm/qemu using on AMD EPYC

[John Nielsen]

> On Feb 11, 2019, at 6:32 AM, Christian Kratzer  wrote:
> 
> I am running freebsd vm on debian 10 buster with libvirt/kvm/qemu.
> 
> I have several kvm hosts in the cluster.  Some with various intel xeon and 
> others with AMD EPYC 7301 cpu.
> 
> FreeBSD vms upto 11.2-RELEASE-p9 boo fine on all systems when passing through 
> the host cpu using following libvirt xml
> 
>  
>
>  

Probably not the same issue, but this sounds similar to this bug I reported a 
few years ago:
https://bugs.launchpad.net/qemu/+bug/1329956

It's just as likely to be a bug in Qemu or KVM as it is in FreeBSD IMO. Maybe 
you can start by determining which CPU feature or features trigger(s) the 
issue. You'll have to hand-roll either some libvirt XML or qemu command lines 
to do it. Assuming you want to stick with XML, first grab the CPU model and 
features list from `virsh capabilities`. Then start with just the model without 
any extra features (using AMD hardware I have access to as an example, replace 
"Opteron_G3" as appropriate):

  
Opteron_G3

  

If that works, then add the other features a few at a time until you break it. 
Here's an example feature list from my same hardware.

  
Opteron_G3


















  

Once you identify the feature or features that cause things to break, you can 
report back here, look for open bugs in Qemu or KVM regarding those features, 
and/or open new bugs.

> FreeBSD 12.0-RELEASE and later hang after boot when swithcing to usermode in 
> start_init: trying /sbin/init
> 
> Following is dmesg from a succesfull boot of 12.0-RELEASE using host-model on 
> Intel CPU
> 
>   Copyright (c) 1992-2018 The FreeBSD Project.
>   Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>   The Regents of the University of California. All rights 
> reserved.
>   FreeBSD is a registered trademark of The FreeBSD Foundation.
>   FreeBSD 12.0-RELEASE-p3 GENERIC amd64
>   FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on 
> LLVM 6.0.1)
>   VT(vga): text 80x25
>   CPU: QEMU Virtual CPU version 2.1.0 (2400.13-MHz K8-class CPU)
> Origin="GenuineIntel"  Id=0x663  Family=0x6  Model=0x6  Stepping=3
> 
> Features=0x783fbfd
> Features2=0x80a02001
> AMD Features=0x20100800
> AMD Features2=0x1
>   Hypervisor: Origin = "KVMKVMKVM"
>   real memory  = 1073741824 (1024 MB)
>   avail memory = 158880 (953 MB)
>   Event timer "LAPIC" quality 100
>   ACPI APIC Table: 
>   FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>   FreeBSD/SMP: 4 package(s) x 1 core(s)
>   ...
> 
> Following is dmesg from a succesfull boot of 12.0-RELEASE using host-model on 
> the qemu virtual cpu
> 
> 
>   Copyright (c) 1992-2018 The FreeBSD Project.
>   Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>   The Regents of the University of California. All rights 
> reserved.
>   FreeBSD is a registered trademark of The FreeBSD Foundation.
>   FreeBSD 12.0-RELEASE-p3 GENERIC amd64
>   FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on 
> LLVM 6.0.1)
>   VT(vga): text 80x25
>   CPU: QEMU Virtual CPU version 2.1.0 (2200.06-MHz K8-class CPU)
> Origin="AuthenticAMD"  Id=0x663  Family=0x6  Model=0x6  Stepping=3
> 
> Features=0x783fbfd
> Features2=0x80a02001
> AMD Features=0x20100800
> AMD Features2=0x65
> SVM: NAsids=16
>   Hypervisor: Origin = "KVMKVMKVM"
>   real memory  = 4294967296 (4096 MB)
>   avail memory = 4099080192 (3909 MB)
>   Event timer "LAPIC" quality 100
>   ACPI APIC Table: 
>   FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>   FreeBSD/SMP: 4 package(s) x 1 core(s)
> 
> Following is dmesg from a succesfull boot of 11.2-RELEASE using host-model on 
> AMD EPYC
> 
>   Copyright (c) 1992-2018 The FreeBSD Project.
>   Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>   The Regents of the University of California. All rights 
> reserved.
>   FreeBSD is a registered trademark of The FreeBSD Foundation.
>   FreeBSD 11.2-RELEASE-p9 #0: Tue Feb  5 15:30:36 UTC 2019
>   r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC 
> amd64
>   FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on 
> LLVM 6.0.0)
>   VT(vga): text 80x25
>   CPU: AMD EPYC Processor (with IBPB) (2200.05-MHz K8-class CPU)
> Origin="AuthenticAMD"  Id=0x800f12  Family=0x17  Model=0x1  Stepping=2
> 
> Features=0x783fbff
> 
> Features2=0xfff83203
> AMD Features=0x2e500800
> AMD 
> Features2=0x8003f7
> Structured Extended 
> Features=0x201c01ab
> XSAVE Features=0x7
> AMD Extended Feature Extensions ID EBX=0x2001000
>

Re: The status of docker

[John Nielsen]

> On Jan 22, 2019, at 11:54 PM, Sergey Zakharchenko  
> wrote:
> 
> Hello there guys,
> 
>> Not quite. I took over the docker freebsd port. Currently I am trying to
>> change him to moby project on GH.
> 
> Jochen, I wish you the best of luck. As a couple of cents, and on
> behalf of Digital Loggers, Inc., I've uploaded some old patches that
> we use to run an ancient version of Docker on FreeBSD:
> https://github.com/digitalloggers/docker-zfs-patches . They speed up
> building of large containers by not iterating over all container files
> at every single stage, using ZFS diffs instead. No warranty, express
> or implied, is provided on those patches; I'm sure you'll find some
> edge cases where they'll break your container builds; you have been
> warned. Also, forgive my Go: that was the first and hopefully the last
> time I wrote something in it.
> 
> That's not much; the real problems are with volume (e.g. single-file
> "volumes" which are hard links) and networking support; they were
> solved (kind of) by us by dynamically generating Dockerfiles and
> adding container startup wrappers, to the point that most would say
> it's too mutilated to be named Docker, so I'm afraid we aren't sharing
> those for the time being.
> 
> My answers to why on earth one would run Docker under FreeBSD instead
> of using plain (or wrapped in yet another wrapper unknown to
> non-FreeBSD) jails would be uniformity, simplicity, skill reuse, etc.
> of quite a broad range of operations. However, Docker/Moby is really
> too tied to Linux; there seem to be random attempts at overcoming that
> but they don't receive enough mind share. Jetpack
> (https://github.com/3ofcoins/jetpack/) could probably also benefit
> from the patches (with appropriate adjustments). Interested people
> willing to invest time in this should gather and decide how to move
> on.

Responding to a random message to share a random-ish thought: has anyone looked 
at Firecracker?

https://firecracker-microvm.github.io/
https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/

It's the now-open-source basis of AWS's Fargate service. The idea is to be more 
secure and flexible than Docker for Kubernetes-like workloads. Linux-only at 
the moment I'm sure but I don't see any reason that FreeBSD couldn't run inside 
a Firecracker microVM (using a stripped-down kernel with virtio_blk, if_vtnet, 
uart and either atkbdc or a custom driver for the 1-button keyboard. It's also 
feasible that FreeBSD could be a Firecracker host (and able to unmodified 
pre-packaged Linux or other microVMs) if someone with the right Go skills 
wanted to port the KVM bits to use VMM/bhyve.

JN

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: The status of docker

[John Nielsen]

> On Jan 23, 2019, at 11:26 AM, John Nielsen  wrote:
> 
>> On Jan 22, 2019, at 11:54 PM, Sergey Zakharchenko  
>> wrote:
>> 
>> Hello there guys,
>> 
>>> Not quite. I took over the docker freebsd port. Currently I am trying to
>>> change him to moby project on GH.
>> 
>> Jochen, I wish you the best of luck. As a couple of cents, and on
>> behalf of Digital Loggers, Inc., I've uploaded some old patches that
>> we use to run an ancient version of Docker on FreeBSD:
>> https://github.com/digitalloggers/docker-zfs-patches . They speed up
>> building of large containers by not iterating over all container files
>> at every single stage, using ZFS diffs instead. No warranty, express
>> or implied, is provided on those patches; I'm sure you'll find some
>> edge cases where they'll break your container builds; you have been
>> warned. Also, forgive my Go: that was the first and hopefully the last
>> time I wrote something in it.
>> 
>> That's not much; the real problems are with volume (e.g. single-file
>> "volumes" which are hard links) and networking support; they were
>> solved (kind of) by us by dynamically generating Dockerfiles and
>> adding container startup wrappers, to the point that most would say
>> it's too mutilated to be named Docker, so I'm afraid we aren't sharing
>> those for the time being.
>> 
>> My answers to why on earth one would run Docker under FreeBSD instead
>> of using plain (or wrapped in yet another wrapper unknown to
>> non-FreeBSD) jails would be uniformity, simplicity, skill reuse, etc.
>> of quite a broad range of operations. However, Docker/Moby is really
>> too tied to Linux; there seem to be random attempts at overcoming that
>> but they don't receive enough mind share. Jetpack
>> (https://github.com/3ofcoins/jetpack/) could probably also benefit
>> from the patches (with appropriate adjustments). Interested people
>> willing to invest time in this should gather and decide how to move
>> on.
> 
> Responding to a random message to share a random-ish thought: has anyone 
> looked at Firecracker?
> 
> https://firecracker-microvm.github.io/
> https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/
> 
> It's the now-open-source basis of AWS's Fargate service. The idea is to be 
> more secure and flexible than Docker for Kubernetes-like workloads. 
> Linux-only at the moment I'm sure but I don't see any reason that FreeBSD 
> couldn't run inside a Firecracker microVM (using a stripped-down kernel with 
> virtio_blk, if_vtnet, uart and either atkbdc or a custom driver for the 
> 1-button keyboard. It's also feasible that FreeBSD could be a Firecracker 
> host (and able to unmodified pre-packaged Linux or other microVMs) if someone 
> with the right Go skills wanted to port the KVM bits to use VMM/bhyve.

S/Go/Rust

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: CFT: Reload LDTR after #VMEXIT on AMD-v in bhyve

[John Baldwin]

On 10/15/18 10:11 AM, Mike Tancsa wrote:
> On 10/15/2018 12:23 PM, John Baldwin wrote:
>>
>> That panic doesn't really make sense. :(  The patch only changes behavior
>> when you are actually running a guest, it doesn't affect anything in the
>> vmm.ko initialization.
> 
> I dont understand either. I have a few klds to load. Is it possible
> something is getting initialized out of order at boot up time ?  r339355
> seems to have fixed it for me. But I suspect the real issue might come
> back some other way
> 
> https://lists.freebsd.org/pipermail/svn-src-head/2018-October/118820.html
> 
> Regardless, I tested with your patch an amd64 and i386 vm and all seems
> to work fine

Ok, thanks!

-- 
John Baldwin


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: CFT: Reload LDTR after #VMEXIT on AMD-v in bhyve

[John Baldwin]

On 10/11/18 1:04 PM, Mike Tancsa wrote:
> On 10/11/2018 3:02 PM, John Baldwin wrote:
>> Can someone using bhyve on an AMD host test this patch?  Just booting a
>> guest to multiuser is probably sufficient testing:
>>
>> https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch
> 
> well, if I let the box boot fully and then load the vmm.ko, all is good.
> But if I load it from /boot/loader.conf, I get a panic at boot up (img
> attached)
> 
> But other than that, it works fine.

That panic doesn't really make sense. :(  The patch only changes behavior
when you are actually running a guest, it doesn't affect anything in the
vmm.ko initialization.
> 
> 
> 0{ryzenbsd12}# fetch -o p
> https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch
> fetch:
> https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch:
> size of remote file is not known
> p 1618  B   15
> MBps    00s
> 0{ryzenbsd12}# patch < p
> Hmm...  Looks like a unified diff to me...
> The text leading up to this was:
> ------
> |From 97323364e196900548f5293ac97bfb22b8a2ba72 Mon Sep 17 00:00:00 2001
> |From: John Baldwin 
> |Date: Tue, 9 Oct 2018 14:49:37 -0700
> |Subject: [PATCH] Reload the LDT selector after an AMD-v #VMEXIT.
> |
> |cpu_switch() always reloads the LDT, so this can only affect the
> |hypervisor process itself.  Fix this by explicitly reloading the host
> |LDT selector after each #VMEXIT.  The stock bhyve process on FreeBSD
> |never uses a custom LDT, so this change is cosmetic.
> |---
> | sys/amd64/vmm/amd/svm.c | 13 +
> | 1 file changed, 13 insertions(+)
> |
> |diff --git a/sys/amd64/vmm/amd/svm.c b/sys/amd64/vmm/amd/svm.c
> |index 2597bf9775706..c420db550bc7e 100644
> |--- a/sys/amd64/vmm/amd/svm.c
> |+++ b/sys/amd64/vmm/amd/svm.c
> --
> Patching file sys/amd64/vmm/amd/svm.c using Plan A...
> Hunk #1 succeeded at 1940.
> Hunk #2 succeeded at 2025.
> Hunk #3 succeeded at 2064.
> done
> 0{ryzenbsd12}#
> 
> I confirmed prior to the patch I could run stock 11.2R  i386 and amd64
> guest images on 12.0-ALPHA9 FreeBSD 12.0-ALPHA9 r339287 as the hypervisor
> 
> CPU: AMD Ryzen 5 1600X Six-Core Processor    (3593.34-MHz
> K8-class CPU)
>   Origin="AuthenticAMD"  Id=0x800f11  Family=0x17  Model=0x1  Stepping=1
>  
> Features=0x178bfbff
>  
> Features2=0x7ed8320b
>   AMD Features=0x2e500800
>   AMD
> Features2=0x35c233ff
>   Structured Extended
> Features=0x209c01a9
>   XSAVE Features=0xf
>   AMD Extended Feature Extensions ID EBX=0x1007
>   SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
>   TSC: P-state invariant, performance statistics
> 
> 
> 
> 
> with the patched version
> 
> ivhd0:  on acpi0
> ivhd0: Flag:b0
> ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0
> ivhd0: Extended features[31:0]:22294ada HATS =
> 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1
> DualPortLogSup = 0x2 DualEventLogSup = 0x2
> ivhd0: Extended features[62:32]:f77ef Max PASID: 0x2f
> DevTblSegSup = 0x3 MarcSup = 0x1
> ivhd0: supported paging level:7, will use only: 4
> ivhd0: device range: 0x0 - 0x
> ivhd0: PCI cap 0x190b640f@0x40 feature:19

Are these messages (ivhd0) new with the patched version and not present with
the old one?  These shouldn't change due to the patch as these are about the
AMD I/O MMU only used by bhyve when doing PCI pass through.

-- 
John Baldwin


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

CFT: Reload LDTR after #VMEXIT on AMD-v in bhyve

[John Baldwin]

Can someone using bhyve on an AMD host test this patch?  Just booting a
guest to multiuser is probably sufficient testing:

https://github.com/bsdjhb/freebsd/commit/97323364e196900548f5293ac97bfb22b8a2ba72.patch

Thanks.

-- 
John Baldwin


___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Bhyve guest boot error, Buffer I/O error on dev vda1, logical block 0, lost async page write

[John-Mark Gurney]

Rajil Saraswat wrote this message on Mon, Aug 27, 2018 at 17:42 -0500:
> On 08/27/2018 11:50 AM, Pete Wright wrote:
> >
> >
> > On 8/26/18 10:07 PM, Sami Halabi wrote:
> >> Hi,
> >> IIRC pool shouldn't be above 90% used otherwise bad things happen..
> >> but at
> >> 69%
> >> Can someone explain thid please?
> >
> > it's possible the filesystem was heavily fragmented?  seems suspicious
> > - did scrubbing your pool find any errors?
> >
> > -pete
> >
> 
> This is how the pool looked like,
> 
> |# zpool list vmpool NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP
> HEALTH ALTROOT vmpool 372G 257G 115G - - 64% 69% 1.00x ONLINE - |
> 
> Scrub did not show the errors. After deleting unused zvols, the guests
> started to work and pool looks like this,
> |
> |
> 
> |# zpool list vmpool
> NAME SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP 
> HEALTH  ALTROOT
> vmpool   372G   197G   175G    - -    56%    53%  1.00x 
> ONLINE  -

There are a couple things that can happen.. you could have a quota
or refquota that prevents a dataset/zdev, or it's descendants from growing
larger, or you could have refreservation or reservation set which limits
the space that other datasets can consume..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

access usb device from bhyve vm

[John Hein]

I asked this on usb@, but might get a better bhyve perspective on this
list...

Does anyone have thought about what it would take to support access to
individual usb devices "natively" from a bhyve vm?
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: [HyperV-2016] urandom performance issue...

[John-Mark Gurney]

Support SimpleRezo wrote this message on Tue, Aug 21, 2018 at 19:06 +0200:
> I was figuring a performance issue of a FreeBSD on HyperV, after:
>   - moving it from an Hyper-V 2012 host to a Hyper-V 2016
>   - updating to FreeBSD 11.2 (custom kernel, but same issue with GENERIC)
> After loosing some times because i was thinking about hardware issue (disk
> controller was my first thought...) and testing with "dd
> if=/dev/urandom"... I discovered this...
> 
> I ran the following test... and quite surprised by the speed result!
> $ dd if=/dev/urandom of=/dev/null bs=10M count=3
> 3+0 records in
> 3+0 records out
> 31457280 bytes transferred in 27.960431 secs (1125064 bytes/sec)
> The test was also sometimes "freezing" the Hyper-V host!
> 
> Looking into "dmesg" i have found this:
> random: registering fast source Intel Secure Key RNG
> random: fast provider: "Intel Secure Key RNG"
> 
> So I rebuilt my KERNEL without "rdrand_rng" (didn't find another way to
> disable it...)... and performances went back to normal!
> 
> I think this issue should documented on https://wiki.freebsd.org/HyperV !

This is likely the same issue as:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230808

If you're having troubles w/ the host freezing, this should probably
still be investigated as even w/ the excessive locking, it should never
freeze.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Passthrough not working with OpenBSD nor NetBSD

[John Baldwin]

On 8/8/18 11:08 AM, Farid Joubbi wrote:
> That's what I also thought, but it's not anything I can force it to do, is 
> it? Isn't it supposed to detect the MSI interrupt compatibility automatically?

Apparently Open/Net always try to setup INTx before trying MSI even if they
won't use INTx per the commit log in revision 280725.  We could perhaps try
to provide a "fake" INTx interrupt that doesn't work, but I'll have to think
about how to implement that. 

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Passthrough not working with OpenBSD nor NetBSD

[John Baldwin]

On 7/6/18 8:07 AM, Farid Joubbi wrote:
> Hello,
> 
> I have configured passthrough of two different PCI Express network
> interface cards. I can't get neither OpenBSD nor NetBSD to work with these
> NICs.
> 
> I get this in dmesg when booting the NetBSD install:
> 
> wm0 at pci0 dev 5 function 0: 82576 quad-1000BaseT Ethernet (rev. 0x01)
> pci_intr_map: no mapping for pin B (line=ff)
> wm0: unable to map interrupt
> bge0 at pci0 dev 6 function 0: Broadcom BCM5720 Gigabit Ethernet
> pci_intr_map: no mapping for pin B (line=ff)
> bge0: couldn't map interrupt
> 
> With OpenBSD it's the same error.
> 
> I have no problems running FreeBSD or CentOS with the exact same setup.
> 
> This is how vm is configured for the NetBSD install:
> loader="grub"
> cpu=1
> memory=4G
> passthru0="12/0/1"
> passthru1="3/0/1"
> disk0_type="virtio-blk"
> disk0_name="disk0.img"
> uuid="1c9596b9-80a0-11e8-960c-5cb9013bbb64"
> grub_install0="knetbsd -h -r cd0a /netbsd"
> grub_run0="knetbsd -h -r ld0a /netbsd"
> 
> 
> I started out with OpenBSD and a Intel NIC. Then I added the Broadcom NIC
> since I figured that maybe it's a driver issue.
> I'm running bhyve on 11.1-RELEASE-p11.
> 
> Any ideas?

You have to use MSI interrupts for pass-through devices for bhyve.  Those
errors look as if the guests are trying to use legacy INTx interrupts.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Overcommitting CPUs with BHyve?

[John-Mark Gurney]

Alan Somers wrote this message on Tue, Jul 24, 2018 at 15:30 -0600:
> What are people's experiences with overcommitting CPUs in BHyve?  I have an
> 8-core machine that often runs VMs totalling up to 5 allocated CPUs without
> problems.  But today I got greedy.  I assigned 8 cores to one VM for a big
> build job.  Obviously, some of those were shared with the host.  I also
> assigned it 8GB of RAM (out of 16 total).  Build performance fell through
> the floor, even though the host was idle.  Eventually I killed the build
> and restarted it with a more modest 2 make jobs (but the VM still had 8
> cores).  Performance improved.  But eventually the system seemed to be
> mostly hung, while I had a build job running on the host as well as in the
> VM.  I killed both build jobs, which resolved the hung processes.  Then I
> restarted the host's build alone, and my system completely hung, with
> top(1) indicating that many processes were in the pfault state.
> 
> So my questions are:
> 1) Is it a known problem to overcommit CPUs with BHyve?

Likely as someone else mentioned the spin lock problem...  It's best if
you can schedule ALL vCPUs at the same time, but obviously the more vCPUs
the harder this becomes, and I don't believe that FreeBSD has a scheduler
that allows you to do this.

The late Benjamin Perrault (iirc) said that his limit was 7 vCPU's per
CPU, I don't remember if that was core or threads (likely core)..  But
I also don't know his work load, or vCPUs per VM...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve panic on netcup root server (vmx enabled)

[John Baldwin]

On 6/15/18 11:22 AM, John Baldwin wrote:
> On 6/15/18 4:44 AM, richard bader wrote:
>> Hi,
>>
>> i try to install a virtual host on a virtual root-server (netcup.de)
>>
>> I've enabled the VMX-Options on the root-server.
>>
>> I've followed the instruction from 
>> https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html
>>
>> on the root-server i've installed FreeBSD-11.1.
>>
>> but wenn starting the guest it panics:
>>
>> sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 5Gb -w -t tap0 -d 
>> guest.img -i -I FreeBSD-11.1-RELEASE-anly.iso test1
>>
>>
>> /boot/kernel/kernel text=0x14972f8 data=0x1384c0+0x4c15e8 
>> syms=[0x8+0x15e8b0+0x8+0x178422]
>> Booting...
>> Copyright (c) 1992-2017 The FreeBSD Project.
>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>>      The Regents of the University of California. All rights reserved.
>> FreeBSD is a registered trademark of The FreeBSD Foundation.
>> FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017
>>      r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
>> FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on 
>> LLVM 4.0.0)
>> VT(vga): text 80x25
>> CPU: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (2294.60-MHz K8-class CPU)
>>    Origin="GenuineIntel"  Id=0x50654  Family=0x6  Model=0x55 Stepping=4
>> Features=0x1f83fbff
>> Features2=0xfeda3203
>>    AMD Features=0x24100800
>>    AMD Features2=0x121
>>    Structured Extended 
>> Features=0x10010b38
>>    XSAVE Features=0x1
>> Hypervisor: Origin = "bhyve bhyve "
>> real memory  = 6442450944 (6144 MB)
>> avail memory = 5131386880 (4893 MB)
>> Event timer "LAPIC" quality 600
>> ACPI APIC Table: 
>> panic: APIC: CPU with APIC ID 224 is not enabled
>> cpuid = 0
>> KDB: stack backtrace:
>> #0 0x80aada97 at kdb_backtrace+0x67
>> #1 0x80a6bb76 at vpanic+0x186
>> #2 0x80a6b9e3 at panic+0x43
>> #3 0x8101dc11 at madt_set_ids+0x91
>> #4 0x80a07c9c at mi_startup+0x9c
>> #5 0x8030202c at btext+0x2c
>> Uptime: 1s
>>
>> Any ideas to get it working or is it just not possible.
> 
> This indicates a bug in the ACPI tables generated by bhyve to give to the
> guest.  Try setting the environment variable 'BHYVE_ACPI_KEEPTMPS=yes'
> when running vmrun.sh and then look in /tmp for files named 'bhyve.*'.
> If you can provide them via a URL or some such, I'm mostly interested in
> the MADT table.

I looked at the raw AML richard provided and it looked fine. :-/
In particular, the MADT only lists a single CPU with APIC ID of 0,
so I'm not sure why it thinks there is a CPU with an APIC ID of 224.

Can you do a 'boot -v' so I can see more messages from the MADT code
while it is running?

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve panic on netcup root server (vmx enabled)

[John Baldwin]

On 6/15/18 4:44 AM, richard bader wrote:
> Hi,
> 
> i try to install a virtual host on a virtual root-server (netcup.de)
> 
> I've enabled the VMX-Options on the root-server.
> 
> I've followed the instruction from 
> https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html
> 
> on the root-server i've installed FreeBSD-11.1.
> 
> but wenn starting the guest it panics:
> 
> sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 5Gb -w -t tap0 -d 
> guest.img -i -I FreeBSD-11.1-RELEASE-anly.iso test1
> 
> 
> /boot/kernel/kernel text=0x14972f8 data=0x1384c0+0x4c15e8 
> syms=[0x8+0x15e8b0+0x8+0x178422]
> Booting...
> Copyright (c) 1992-2017 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>      The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017
>      r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on 
> LLVM 4.0.0)
> VT(vga): text 80x25
> CPU: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (2294.60-MHz K8-class CPU)
>    Origin="GenuineIntel"  Id=0x50654  Family=0x6  Model=0x55 Stepping=4
> Features=0x1f83fbff
> Features2=0xfeda3203
>    AMD Features=0x24100800
>    AMD Features2=0x121
>    Structured Extended 
> Features=0x10010b38
>    XSAVE Features=0x1
> Hypervisor: Origin = "bhyve bhyve "
> real memory  = 6442450944 (6144 MB)
> avail memory = 5131386880 (4893 MB)
> Event timer "LAPIC" quality 600
> ACPI APIC Table: 
> panic: APIC: CPU with APIC ID 224 is not enabled
> cpuid = 0
> KDB: stack backtrace:
> #0 0x80aada97 at kdb_backtrace+0x67
> #1 0x80a6bb76 at vpanic+0x186
> #2 0x80a6b9e3 at panic+0x43
> #3 0x8101dc11 at madt_set_ids+0x91
> #4 0x80a07c9c at mi_startup+0x9c
> #5 0x8030202c at btext+0x2c
> Uptime: 1s
> 
> Any ideas to get it working or is it just not possible.

This indicates a bug in the ACPI tables generated by bhyve to give to the
guest.  Try setting the environment variable 'BHYVE_ACPI_KEEPTMPS=yes'
when running vmrun.sh and then look in /tmp for files named 'bhyve.*'.
If you can provide them via a URL or some such, I'm mostly interested in
the MADT table.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve -g gets killed by EPIPE

[John Baldwin]

On Thursday, November 24, 2016 06:13:45 PM Andriy Gapon wrote:
> 
> I am trying to follow this guide:
> https://wiki.freebsd.org/bhyve/DebuggingWithGdb
> 
> It seems to work, once or twice, but after that when I active kdb in the guest
> bhyve would terminate with status 141.  I interpret it as signal 13, EPIPE.

FWIW, for the debug server I made the sockets disable SIGPIPE to avoid
killing the guest if the debugger dies.  The -g sockets probably need to
disable SIGPIPE as well.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Cross-compiling socat

[John-Mark Gurney]

Darius Mihai wrote this message on Mon, Sep 25, 2017 at 13:58 +0300:
> I am currently implementing the VirtIO devices for bhyve on ARM, more
> specifically the VirtIO-console device.
> 
> However, I am stuck since I need to use 'socat' on the host, but
> cross-compiling always fails (the ports do not seem to have a
> straight-forward make system for cross-compiling as far as I could tell).
> I have tried using `./configure --host=amd64 --build=armv6
> CC=arm-none-eabi-gcc`,
> in the 'work/socat-1.7.3.2' directory, but this doesn't seem to work.
> 
> If anyone could provide any insight, it would be greatly appreciated.

You shoudl be able to do a toolchain TARGET_ARCH=armv6 to build a cross
build environment, and then you can run make buildenv to launch a shell
to build socat in this environment...  I've done it this way in the past,
though I forget if it needs a complete buildworld to work, or if toolchain
is enough..

There also appears to be the xdev-build and xdev-install which I think
install them into the current environment which you can use... Though I'm
not sure if the libraries and other things are built as I've only used
it once a long time ago..

Hope this helps.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: PCIE card with USB ports

[John-Mark Gurney]

Rajil Saraswat wrote this message on Sat, May 13, 2017 at 20:34 -0500:
> I am looking for a recommendation for an out-of-the-box working PCIE
> card with USB controller.

They all should just work.  The driver interface is standardized and
should just work no mater what card you pick.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Boot Physical Windows in bhyve

[John-Mark Gurney]

Will haben wrote this message on Sat, Apr 29, 2017 at 17:21 +0200:
> Is it possible to load a physical Windows partition within bhyve? The windows 
> installation is on a separate hard drive that???s using MBR. 
> I found the possibility of transforming the drive to a VMware virtual drive 
> and then to bhyve img. 

Have you just tried to point the disk image at your HD?  This should
"just work" as zvol (which is one way to run a VM) works, and a disk
isn't any different...

bhyve doesn't have a format, it's just raw disk images..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

VirtualBox won't build from ports

[John Klos]

Hi,

I find it a little odd that a clean install of FreeBSD 11.0 on amd64 
followed by:


portsnap fetch ; portsnap extract
echo "DISABLE_LICENSES=yes" > /etc/make.conf
echo "BATCH=yes" >> /etc/make.conf
cd /usr/ports/emulators/virtualbox-ose
make install clean

fails. What's the recommended (and non-embarrassing) way to set up a clean 
FreeBSD installation with VirtualBox?


Here's the error:

...
/bin/ln -sf python-2.7.pc 
/usr/ports/lang/python3/work/stage/usr/local/libdata/pkgconfig/python2.pc

> Compressing man pages (compress-man)
===>  Installing for python3-2_3
===>  Checking if python3 already installed
===>   Registering installation for python3-2_3 as automatic
Installing python3-2_3...
pkg-static: python3-2_3 conflicts with python2-2_3 (installs files into 
the same place).  Problematic file: /usr/local/bin/2to3-2

*** Error code 70

Stop.
make[6]: stopped in /usr/ports/lang/python3
*** Error code 1

Stop.
...

Thanks,
John Klos
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: pptdevs option crashes system

[John Baldwin]

On Sunday, April 24, 2016 11:57:55 AM David Ehrmann wrote:
> It sounds like the bug I reported a while ago, but on a different chip
> family:
> 
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=207602

As a workaround on 10.3 and later you can use 'devctl set driver' to
mark devices for pass through post-boot rather than using the loader.conf
option.  E.g.:

# devctl set driver pci0:11:0:0 ppt

Currently there's no way to undo this, you can can 'set driver' to a
"native" driver if one exists.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: ZFS subvolume support inside Bhyve vm

[John Nielsen]

> On Mar 10, 2016, at 5:31 PM, Paul Vixie  wrote:
> 
> Сергей Мамонов wrote:
>> Hello!
>> 
>> Yes - zvols looks awesome. But what driver you use for it?
> 
> virtio-blk.
> 
>> And what
>> about disk usage overhead in guest?
> 
> ufs on zvol is faster, either in the parent or a bhyve using virtio-blk, than 
> zfs. at least for writing, which is my dominant work load. i expect that this 
> is due to zfs's compression logic rather than anything having to do with 
> creating/extending files to accommodate writes.
> 
>> virtio-blk doesnt support fstrim (ahci-hd support it, but slower? "/At
>> this point virtio-blk is indeed faster then ahci-hd on high IOPS/").
>> In linux && kvm we try used virtio-scsi driver with support fstrim, but
>> how I see it not availble now in 10-2 stable for bhyve.
>> And I not lonely with this question -
>> https://lists.freebsd.org/pipermail/freebsd-virtualization/2015-March/003442.html
> 
> i'm just going to live without fstrim until it's supported in virtio-blk. i 
> know that this option isn't available to everybody, but at the moment storage 
> is cheap enough to waste.

At the risk of getting farther off-topic..

Virtio-blk can't and won't support fstrim. If/when bhyve were to have 
virtio-scsi support that could work with trim, but last I was aware there 
wasn't a whole lot of momentum in that direction. The virtual AHCI controller 
in bhyve does support trim and performs remarkably well. That's what I use for 
my vol-backed VMs and I haven't had any complaints. Worth testing, IMO.

JN

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: mount bhyve image

[John]

On Sun, Mar 06, 2016 at 01:15:38PM -0500, Allan Jude wrote:


Yes, you mount it the same way as a cdrom image:

mdconfig -a -t vnode -f /path/to/image

then: mount -t  /dev/md0p1 /mnt

note that depending on your partitioning scheme, the exact device name
will different. md0p1 is the first GPT partition, it might be MBR
instead, which would be md0s1, etc.


Thanks for the confirmation, thats useful info.

--
John 
___

freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

mount bhyve image

[John]

Hello list,

Can a bhyve image be mounted to the filesystem? Not an active image, I
mean, say one has a problem loading a bhyve instance. Is it possible
to mount the image as one would for example mount a cdrom image as per
md(8) ? If it is possible, what are the parameters please?

thanks,
--
John 
___

freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

patch to improve use of ZFS volumes with vmrun.sh

[John Dickinson]

Hi,

In case it is of use to anyone, here is a little patch to improve use of ZFS 
volumes with vmrun.sh


--- /usr/share/examples/bhyve/vmrun.sh  2015-08-12 15:27:20.0 +
+++ ./vmrun.sh  2016-01-14 13:17:27.972055736 +
@@ -166,11 +166,24 @@
 make_and_check_diskdev()
 {
 local virtio_diskdev="$1"
+local ZVOL=0
 # Create the virtio diskdev file if needed
-if [ ! -f ${virtio_diskdev} ]; then
+case "${virtio_diskdev}" in
+/dev/zvol/*) ZVOL=1 ;;
+esac
+
+if [ $ZVOL -eq 0 ]; then
+if [ ! -f ${virtio_diskdev} ]; then
  echo "virtio disk device file \"${virtio_diskdev}\" does not 
exist."
 echo "Creating it ..."
 truncate -s 8G ${virtio_diskdev} > /dev/null
+fi
+else
+if [ ! -c ${virtio_diskdev} ]; then
+echo "virtio disk device volume \"${virtio_diskdev}\" does not 
exist."
+echo "Please create it"
+    exit 1
+fi
 fi
 
 if [ ! -r ${virtio_diskdev} ]; then


regards
John

John Dickinson

http://sinodun.com

Sinodun Internet Technologies Ltd.
Magdalen Centre
Oxford Science Park
Robert Robinson Avenue
Oxford OX4 4GA
U.K.

signature.asc
Description: OpenPGP digital signature

Re: FreeBSD 11 - Bhyve - Spoof MAC address

[John Nielsen]

> On Jan 4, 2016, at 9:32 AM, James Lodge  wrote:
> 
> Hi All,
> 
> 
> I'm just getting started with Bhyve. So far everything is working as 
> expected. My original goal was to be running Ubuntu 12.04 i386 as I need it 
> for a particular project. One issue I'm having is MAC address spoofing. I'm 
> aware I can change the MAC address within Ubuntu but I'd like to configure 
> the tap interface from the host which should be possible according to man 
> pages.
> 
> 
> Bhyve Man Page: https://www.freebsd.org/cgi/man.cgi?query=bhyve=8
> 
> 
> 
> The issue I have is that by setting the below, the vm boots, I can console 
> via null modem, but there is no eth0 interface, only the loopback. Removing 
> the static MAC, reboot and everything is present and correct.
> 
> 
> -s 2:0,virtio-net,tap0,mac=xx:xx:xx:xx:xx:xx

It looks like you are setting the MAC correctly on your bhyve command line and 
bhyve is running; so far so good. Is it possible that Ubuntu has a different 
MAC saved for its idea of eth0 and is therefore not doing what you expect? 
(Perhaps udev is renaming the device?)

Can you run these two commands within the VM and post the output?
 ip link show
 lspci


JN

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: BAR and pci passthrough

[John Baldwin]

On Saturday, December 12, 2015 02:04:17 PM G. Amanakis via 
freebsd-virtualization wrote:
> Changing
> #define   PCIM_BAR_MEM_BASE   0xfff0ULL
> to
> #define   PCIM_BAR_MEM_BASE   0xf000ULL
> in
> sys/dev/PCI/pcireg.h makes the min size of a memory BAR equal to 0x1000 or 
> 4096 bytes and it lets me passthrough the device. Any objections to this 
> strategy?

This constant defines the spec as implemented in hardware.  You will break
real hardware on bare metal if you make this change.  As Neel notes, this is
hard to fix properly.  The proper fix is to modify the ppt(4) driver so that
it asks the PCI bus to allocate a full page for BARs that are smaller than a
page.  Unfortunately the PCI bus driver doesn't currently provide a way to do
that.  Even if it did it would not necessarily always work since the device
may be behind a PCI-PCI bridge whose windows can't grow due to existing
allocations of adjacent addresses.  Alternatively the hypervisor could trap
all accesses to this page and only permit accesses to the range that contains
the BAR, but that would be quite slow.

> On December 2, 2015 11:54:03 PM GMT+01:00, Neel Natu <neeln...@gmail.com> 
> wrote:
> >On Wed, Dec 2, 2015 at 3:13 AM, G. Amanakis via freebsd-virtualization
> ><freebsd-virtualization@freebsd.org> wrote:
> >> I am facing the following problem:
> >> on a X9SCM with an E3-1220Lv2 I am trying to passthrough the
> >onboard usb controllers. I disable the usb module from the kernel
> >config and using pptdev the controllers are assigned to ppt devices.
> >However running bhyve on FreeBSD 10.2 with:
> >>
> >> sudo bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s
> >2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img -s
> >4:0,passthru,0/29/0 -l com1,stdio -c 2 -m 2048M linuxguest
> >>
> >> returns :
> >>
> >> passthru device 0/29/0 BAR 0: base 0xdf823000 or size 0x400 not
> >page aligned
> >>
> >> The problem probably that the length of the bar is smaller and not
> >aligned with the pagesize. Could the length of the BAR be modified in
> >order to perform the pci passthrough?
> >
> >Yes, that's correct - the size of the BAR is not a multiple of the
> >page size which leads to the error. If this BAR is mapped into the
> >guest's address space then it will "leak" an additional 3K into the
> >guest (since the minimum nested mapping is 4KB in size).
> >
> >It is hard to fix this in the general case if you want to truly
> >passthrough the BAR to the guest.
> >
> >best
> >Neel
> >
> >> --
> >> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> >> ___
> >> freebsd-virtualization@freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> >> To unsubscribe, send any mail to
> >"freebsd-virtualization-unsubscr...@freebsd.org"
> 
> 


-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: adding diskspace to a bhyve instance

[John-Mark Gurney]

Matt Churchyard via freebsd-virtualization wrote this message on Wed, Nov 25, 
2015 at 10:18 +:
> Of course that's the easy bit. The more dangerous part is resizing the 
> partitions inside the guest (if it's not whole disk ZFS), and then resizing 
> the filesystems. If the disk is GPT partitioned in the guest you will 
> probably have to recover the partition table first, as the secondary copy 
> will no longer exist at the end of the disk. You'll then need to resize the 
> partitions (hopefully the 'main' partition you want to resize is the last on 
> the disk as that'll probably make it easier). Once done you then need to 
> resize the filesystem. For ZFS you can usually just 'zpool online -e'. For 
> UFS you'll need to grow the filesystem as shown in the handbook.

I have written an rc.d script growfs that is in HEAD that makes this
painless...  If you have a single UFS fs, w/ the root as the last
partition/fs on the disk, simply grow the disk, and then you can run
"service growfs start", and it just works...

This will work on any system, not just VMs...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: adding diskspace to a bhyve instance

[John]

On Thu, Nov 19, 2015 at 02:42:50PM -0500, Manas Bhatnagar wrote:

On 19/11/15 12:20 PM, John wrote:

Hello list,

What's the best way of increasing the space of a bhyve guest instance?
Would it be via growfs? Inside or outside of the vm? Or would it be
better to truncate another chunk of space and refer to it in /etc/fstab?

thanks,


Hello John,

-   Do you use virtio-blk (a file created with 'truncate') or ahci-hd (A
ZFS filesystem, for example) for your guest disk? You have to increase
the space of the virtual disk that bhyve uses.
- If you used 'truncate', my guess is that you can use truncate to
create a new file of larger size. Then, boot into a livecd in bhyve with
both disks and do a 'dd' from one disk to the other. I have tried to
'dd' between files on the host but that didn't seem to work.
- If it is a ZFS filesystem, create a new filesystem with 'zfs create
-V 50gb zroot/new-volume/' then use a 'zfs send ... | zfs receive ...'

-   Are these FreeBSD guests (and which filesystem - UFS or ZFS) or Linux
guests?
- If these are FreeBSD guests running UFS, look at
https://www.freebsd.org/doc/handbook/disks-growing.html
- If these are FreeBSD guests running ZFS, you can probably create the
filesystems on your new disk and then use zfs send & receive
- If these are linux guests, you will have to use a livecd and
something like 'gpart'.


Hello Manas

The host runs ZFS on FreeBSD-10.2-STABLE. The FreeBSD guests (10.2 and
11-) run UFS. The Linux guests run Ubuntu 14.04-server on ext4. All
the guest disks were created using truncate. Thanks for the links,
it's what I was looking for. 
--

John
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

adding diskspace to a bhyve instance

[John]

Hello list,

What's the best way of increasing the space of a bhyve guest instance?
Would it be via growfs? Inside or outside of the vm? Or would it be
better to truncate another chunk of space and refer to it in /etc/fstab?

thanks,
--
John 
___

freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Python on FreeBSD is slower than on Linux

[John-Mark Gurney]

Alfred Perlstein wrote this message on Thu, Nov 12, 2015 at 11:35 -0800:
> I'm adding Freebsd-virtualization to this thread as both problems point 
> to some possible issue with FreeBSD as a guest.  (although a bare metal 
> comparison should likely be done as well).

This could simply be a python compiled w/ different compiler and
compiler optimization flags issue...

One easy way to eliminate FreeBSD is to take the Linux version of
python, and run it on FreeBSD using the linux emulator to help ensure
that the performance is the same when running the same binary on the
two OS's...

> On 11/12/15 11:26 AM, Vladimir Bogrecov wrote:
> > Hello,
> >
> > I'm developing a little project on Python 3.5. The server's operating
> > system is FreeBSD 10.2. Today I decided to do a little test "just for fun"
> > and the result has confused me. I ran the following code
> >
> > import random
> > import time
> >
> >
> > def test_sort(size):
> >  sequence = [i for i in range(0, size)]
> >  random.shuffle(sequence)
> >  start = time.time()
> >  ordered_sequence = sorted(sequence)
> >  print(time.time() - start)
> >
> >
> > if __name__ == '__main__':
> >  test_sort(100)
> >
> > on FreeBSD 10.2 x64 and on Debian 8 x64. Both computers was the smallest
> > (5$ per month) virtual machines on the Digital Ocean (
> > https://www.digitalocean.com). The average result on the FreeBSD was 1.5
> > sec, on the Debian 1.0 sec. Both machines was created specially for test
> > and had not any customization. Could you help me to understand why python
> > is so slower on FreeBSD and may be there are some steps I can perform to
> > speed up the python to work not slower than on Debian.
> >
> > I have found in Google the similar question:
> > https://lists.freebsd.org/pipermail/freebsd-python/2012-June/004306.html so
> > it has an interest not only for me.
> >
> > P.S. I really like FreeBSD and I would be happy to solve this issue. If you
> > will have an interest to this issue I can provide SSH access for both
> > machines :)

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

error running grub-bhyve with -S

[John Nielsen]

I’m running bhyve on an Intel machine running FreeBSD 11-CURRENT, updated a few 
days ago. I have a Debian 8.2 VM that has been running fine but now I’d like to 
add a PCI pass-through device. Unfortunately, when I add the required “-S” flag 
to my grub-bhyve command line it doesn’t work:

# bhyvectl —vm=vm0 --destroy
# grub-bhyve -m /images/vm0-device.map -M 1024 -r hd1 -S vm0
Could not setup memory for VM
Error in initializing VM

Obviously, running “bhyve” after that fails as well.

What would cause that error and what can I do about it?

Thanks!

JN

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Are the sched_choose() or tdq_choose() functions called after returning from an interrupt?

[John Baldwin]

On Friday, June 12, 2015 10:51:20 AM Stefan Andritoiu wrote:
> Hello,
> 
> When returning from an interrupt, does it switch directly the thread
> that was interrupted? Or is the scheduler called to choose a thread to
> run (most probable the thread that was interrupted)?

The scheduler is always called to choose.  An ithread may have awakened
a more important thread than the originally interrupted thread.

> More specifically, are the sched_choose() or tdq_choose() functions
> called after returning from an IPI?

As I said in my previous reply.  This is not always true for IPIs.
Device interrupt handlers that schedule an ithread will schedule the
ithread and possibly preempt to it before returning from the interrupt.
Only when the interrupted thread is resumed does it then fully cleanup
the interrupted stack and execute iret.
 
> Does an interrupt have it's own thread, or does it run in the context
> of the interrupted thread as in Linux?

Most device interrupts use a dedicated thread.  Some device interrupt
handlers do not.  A driver can choose to register a non-threaded handler
that runs on the stack of the interrupted thread.  However, such handlers
are limited in what they can do and must often defer work to other threads
that they explicitly schedule (such as via a taskqueue).

-- 
John Baldwin

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: What is the sequence of context switches when an IPI is received?

[John Baldwin]

On Thursday, June 11, 2015 04:50:52 PM Stefan Andritoiu wrote:
> Hello,
> 
> From the FreeBSD Handbook: "FreeBSD deals with interrupt handlers by
> giving them their own thread context".
> From my understanding when a IPI is received the thread that will run
> it is placed on the real-time runq, and the scheduler will be invoked
> to schedule it.
> 
> So the sequence should be:
> currently running thread -> scheduler thread -> interrupt handler ->
> scheduler thread -> previously interrupted thread (if no thread
> priority change took place inside the interrupt handler)
> 
> Is this correct?

(Sorry these replies are dated.)

IPIs do not run in a dedicated thread.  Only device interrupt handlers
run in a dedicated thread.  IPIs (and some device handlers such as timer
interrupts) borrow the stack of the currently executing thread to run
their handler.  As a result, you should have no context switches when
an IPI is received unless the IPI handler specifically invokes one.

One IPI that explicitly swithes is IPI_PREEMPT.  However, other IPIs can
result in a switch if the interrupted thread is running in userland and
a context switch to another thread is performed via the checks in
userret() and/or ast().  This is how IPI_AST is used.  It has a NULL
handler and is only sent to trigger the side effects in userret() in
case the thread is executing in userland.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve: fix bhyve warning CTASSERT

[John-Mark Gurney]

Conrad Meyer wrote this message on Thu, Aug 13, 2015 at 08:12 -0700:
 Better to just replace CTASSERT() with _Static_assert() while you're here.

And make sure that sys/cdefs.h is included for compatibility w/ pre-C11
compilers...

 On Thu, Aug 13, 2015 at 5:05 AM, Stefano Garzarella
 stefanogarzare...@gmail.com wrote:
  Hi all,
  when I compile bhyve, I have the following errors from clang:
  pci_emul.c:750:2: error: unused typedef '__assert750'
  [-Werror,-Wunused-local-typedef]
  CTASSERT(sizeof(struct msicap) == 14);
  pci_emul.c:776:2: error: unused typedef '__assert776'
  [-Werror,-Wunused-local-typedef]
  CTASSERT(sizeof(struct msixcap) == 12);
  pci_emul.c:928:2: error: unused typedef '__assert928'
  [-Werror,-Wunused-local-typedef]
  CTASSERT(sizeof(struct pciecap) == 60);
 
  I fixed them in this simple way:
 
  diff --git a/bhyverun.h b/bhyverun.h
  index 87824ef..7ac3aa9 100644
  --- a/bhyverun.h
  +++ b/bhyverun.h
  @@ -32,7 +32,8 @@
   #ifndef CTASSERT   /* Allow lint to override */
   #defineCTASSERT(x) _CTASSERT(x, __LINE__)
   #define_CTASSERT(x, y) __CTASSERT(x, y)
  -#define__CTASSERT(x, y)typedef char __assert ## y[(x) ? 1
  : -1]
  +#define__CTASSERT(x, y)typedef char __assert ## y[(x) ? 1
  : -1] \
  +   __unused
   #endif

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: troubles booting OpenBSD 5.7 or NetBSD 6.1.5 w/ grub2-bhyve

[John-Mark Gurney]

John-Mark Gurney wrote this message on Tue, Jul 28, 2015 at 12:36 -0700:
 I'm trying to run either Net or Open for some testing, and I'm not
 having luck.
 
 After first figuring out that the README on github doesn't include that
 you need to include the boot device to load the files from, I finally
 got kopenbsd and knetbsd loading, but when I run boot, it just drops
 me to a prompt w/o doing anything.

With a little bit of help (thanks @nahannisys), I managed to fix my
issues... I put up a blog entry at:
http://blog.funkthat.com/2015/07/installing-and-running-netbsd-and.html

that describes how I did both an install and boot of both NetBSD and
OpenBSD...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

troubles booting OpenBSD 5.7 or NetBSD 6.1.5 w/ grub2-bhyve

[John-Mark Gurney]

I'm trying to run either Net or Open for some testing, and I'm not
having luck.

After first figuring out that the README on github doesn't include that
you need to include the boot device to load the files from, I finally
got kopenbsd and knetbsd loading, but when I run boot, it just drops
me to a prompt w/o doing anything.

For NetBSD...
[dev.map]
(cd0) NetBSD-6.1.5-amd64.iso
(hd1) netbsd.img

grub-bhyve command:
# grub-bhyve -m dev.map nbsd615


   GNU GRUB  version 2.00

   Minimal BASH-like line editing is supported. For the first word, TAB lists 
possible
   command completions. Anywhere else TAB lists possible device or file 
completions.


grub 
grub knetbsd -h -r cd0a (cd0)/netbsd
grub boot

# 

For OpenBSD...
# cat dev.map
(cd0) install57.iso
(hd1) openbsd.img
# grub-bhyve -m dev.map obsd57


   GNU GRUB  version 2.00

   Minimal BASH-like line editing is supported. For the first word, TAB lists 
possible
   command completions. Anywhere else TAB lists possible device or file 
completions.


grub kopenbsd -h com0 -r sd0a (cd0)/5.7/amd64/bsd
grub boot

#

So, kernels are loading fine, just things crash when trying to run
them...  FreeBSD VM's run fine...

FreeBSD carbon.funkthat.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r282774: Mon 
May 11 11:41:38 PDT 2015 
j...@carbon.funkthat.com:/a/obj/a/home/jmg/FreeBSD.svn/HEAD/sys/GENERIC  amd64

# pkg info | grep bhyve
grub2-bhyve-0.30_1 Grub-emu loader for bhyve

CPU: AMD A10-5700 APU with Radeon(tm) HD Graphics(3393.90-MHz K8-class CPU)
  Origin=AuthenticAMD  Id=0x610f01  Family=0x15  Model=0x10  Stepping=1
  
Features=0x178bfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT
  
Features2=0x3e98320bSSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C
  AMD Features=0x2e500800SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM
  AMD 
Features2=0x1ebbfffLAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC
  Structured Extended Features=0x8BMI1
  SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=65536

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve: bhyveload, bhyve, bhyvectl --destroy

[John-Mark Gurney]

Andriy Gapon wrote this message on Wed, Jun 03, 2015 at 10:10 +0300:
 On 03/06/2015 02:04, John-Mark Gurney wrote:
  Andriy Gapon wrote this message on Tue, Jun 02, 2015 at 21:15 +0300:
  I guess you are running bhyve through the shell script vmrun.sh?
  I am doing everything by hand.
  
  Correct:
  sh /usr/share/examples/bhyve/vmrun.sh -g 6444 -d mach10s.img -t tap0
  
  It's nice.. shutdown -r now and shutdown -p now both work exactly as
  you'd expect them to... :)
 
 Yes, it's quite convenient.  The only drawback is that currently it can't pass
 multiple -d options to bhyveload and I need that from time to time.

Hmm... Looking at the script, it looks like you can (at least on HEAD):
d)
eval disk_dev${disk_total}=\${OPTARG}\
disk_total=$(($disk_total + 1))

 Probably not hard to change that.

It'd also be nice to allow you to switch the default from virtio-blk
to ahci when you're image doesn't support it..

Hmm.. It wouldn't be hard to try to use environment vars for the defaults
either...

Maybe it's time to move vmrun.sh to bhyverun, install it and give
it a proper man page?

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve: bhyveload, bhyve, bhyvectl --destroy

[John-Mark Gurney]

Andriy Gapon wrote this message on Tue, Jun 02, 2015 at 21:15 +0300:
 On 02/06/2015 19:49, John-Mark Gurney wrote:
  Andriy Gapon wrote this message on Tue, Jun 02, 2015 at 14:20 +0300:
  I am very new to bhyve, so sorry if I am asking something silly or obvious.
  I am using bhyve to speed up my testing and it seems that each time I need 
  to
  restart a VM I need to go through the cycle of destroying it with bhyvectl
  --destroy, then re-loading a kernel with bhyveload and then actually 
  booting the
  VM with bhyve.  It seems that I have to do this even if I don't change th 
  kernel
  between reboots.  My first naive impression was that the point of 
  bhyveload was
  to load the kernel once.  Seems it ain't so?
  
  Hmm...  I'm not seeing that here...  I just scp a new kernel into the
  vm, install it, and run shutdown -r now which drops bhyve back to
  loader, and loads the new kernel... I've been doing this quite
  successfully over the last few months...
  
  I am running a month old HEAD though...
 
 I guess you are running bhyve through the shell script vmrun.sh?
 I am doing everything by hand.

Correct:
sh /usr/share/examples/bhyve/vmrun.sh -g 6444 -d mach10s.img -t tap0

It's nice.. shutdown -r now and shutdown -p now both work exactly as
you'd expect them to... :)

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve: bhyveload, bhyve, bhyvectl --destroy

[John-Mark Gurney]

Andriy Gapon wrote this message on Tue, Jun 02, 2015 at 14:20 +0300:
 I am very new to bhyve, so sorry if I am asking something silly or obvious.
 I am using bhyve to speed up my testing and it seems that each time I need to
 restart a VM I need to go through the cycle of destroying it with bhyvectl
 --destroy, then re-loading a kernel with bhyveload and then actually booting 
 the
 VM with bhyve.  It seems that I have to do this even if I don't change th 
 kernel
 between reboots.  My first naive impression was that the point of bhyveload 
 was
 to load the kernel once.  Seems it ain't so?

Hmm...  I'm not seeing that here...  I just scp a new kernel into the
vm, install it, and run shutdown -r now which drops bhyve back to
loader, and loads the new kernel... I've been doing this quite
successfully over the last few months...

I am running a month old HEAD though...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: can we get some interaction between halt/reboot and bhyve?

[John-Mark Gurney]

Paul Vixie wrote this message on Sun, May 24, 2015 at 18:05 -0700:
 bhyve interprets SIGTERM as an ACPI poweroff, and usually the VM will
 shutdown very quickly. but it may not happen in five seconds, which is
 all halt(8) (and reboot(8)) promises. the code for reboot and halt
 (/usr/src/sbin/reboot/reboot.c) is shown below. what's the right way to
 change this so that if there are 50 bhyve processes running and they are
 ACPI powerdowning as fast as they can, then the halt or reboot of the
 bare metal will wait for them?

Shouldn't shutdown be used instead of reboot/halt?  This allows
various rc.d scripts to run, and you can use that to make sure all
your bhyve instances are shutdown...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve code question, pci_virtio_net.c

[John-Mark Gurney]

Allan Jude wrote this message on Sat, May 16, 2015 at 13:02 -0400:
 While looking at the code, I noticed the following comment:
 
   /*
* The default MAC address is the standard NetApp OUI of 00-a0-98,
* followed by an MD5 of the PCI slot/func number and dev name
*/
   if (!mac_provided) {
   snprintf(nstr, sizeof(nstr), %d-%d-%s, pi-pi_slot,
   pi-pi_func, vmname);
 
 
 which raised 2 questions:
 1) According to the comment, the mac address should be based on the md5
 of the DEVICE name, not the VM name. Which is correct? Which is more
 desirable?
 
 I suppose most people's VMs will have similar slot/func numbers for the
 NIC (especially with certain config engines that always assign devices
 in a specific order), and so the VM name is more likely to be unique
 than the name of the tap device.

Correct, VM name makes the most sense here...

 Would it make sense to include more unique information in this hash?
 like the host's UUID or something, to prevent the chance of more than 1
 VM on the same LAN having the same MAC if they have the same VM Name?

Sure...

 2) Should bhyve instead use the FreeBSD assigned OUI for these MAC
 addresses?

Depends, if NetApp gave us permission to use them here, then we might
as well keep using them.  gnn would be the persion to talk to about
this..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Bhyve storage improvements (was: Several bhyve quirks)

[John Nielsen]

On Mar 27, 2015, at 10:47 AM, John Nielsen li...@jnielsen.net wrote:

 On Mar 27, 2015, at 3:46 AM, Alexander Motin m...@freebsd.org wrote:
 
 I've always assumed virtio driver  emulated driver so it didn't occur
 to me to try ahci-hd.
 
 I've just merged to FreeBSD stable/10 branch set of bhyve changes that
 should significantly improve situation in the storage area.
 
 virtio-blk driver was fixed to work asynchronously and not block virtual
 CPU, that should fix many problems with performance and interactivity.
 Both virtio-blk and ahci-hd drivers got ability to execute multiple (up
 to 8) requests same time, that should proportionally improve parallel
 random I/O performance on wide storages.  At this point virtio-blk is
 indeed faster then ahci-hd on high IOPS, and they both are faster then
 before.
 
 On the other side ahci-hd driver now got TRIM support to allow freeing
 unused space on backing ZVOL. Unfortunately there is no any TRIM/UNMAP
 support in virtio-blk API to allow the same.
 
 Also both virtio-blk and ahci-hd drivers now report to guest logical and
 physical block sizes of underlying storage, that allow guests properly
 align partitions and I/Os for best compatibility and performance.
 
 Mav, thank you very much for all this great work and for the concise summary. 
 TRIM on AHCI makes it compelling for a lot of use cases despite the probable 
 performance hit.
 
 Does anyone have plans (or know about any) to implement virtio-scsi support 
 in bhyve? That API does support TRIM and should retain most or all of the 
 low-overhead virtio goodness.

Okay, some belated googling reminded me that this has been listed as an open 
task in the last couple of FreeBSD quarterly status reports and discussed at 
one or more devsummits. I'd still be interested to know if anyone's actually 
contemplated or started doing the work though. :)

JN

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: How to connect ssh and /dev/nmdm devices Was:( Re: Native Linux guest in Bhyve (no grub2-bhyve) status? )

[John-Mark Gurney]

Willem Jan Withagen wrote this message on Tue, Jan 20, 2015 at 17:37 +0100:
 On 2015-01-05 9:41, Willem Jan Withagen wrote:
  On 2015-01-05 0:28, Peter Grehan wrote:
  Hi Willem,
 
  Would it it be possible in the meantime to enhance grub2-bhyve with the
  same possibility as bhyve itself has for the console?
 
  So I can redirect grub screen access to /dev/nmdm12041, and one can even
  use this channel to see the grubscreen during rebooting.
 
  Or if it is not that hard, give some pointers to write it myself.
 
Conrad contributed this to grub-bhyve with
https://github.com/grehan-freebsd/grub2-bhyve/pull/2
 
It's available in v0.30 which has been in ports for a while now:
 
https://github.com/grehan-freebsd/grub2-bhyve/releases/tag/v0.30
 
  Right,
 
  Did miss that, I guess.
  I'll upgrade my bhyve-grub.
 
 Yup, works really nice.
 
 Now in a continuance from this...
 
 What is the easiest way to propagate  the full-duplex tty stream from 
 a SSH-login to a /dev/nmdmA.
 
 This will give easy access to the console screens.
 
 one would type:
   ssh -p  user@management-ip
 and end up at the console on /dev/nmdmA
 
 Currently I'm using tip for this, but that also requires hacking new VMs 
 into /etc/remote.
 
 So a very basic TIP, without the serial/ACU/speed stuff, would really be 
 useful. Preferably all signals are sent thru as well.
 And if this might not be 100% secure, it would require an ssh-jailed 
 setup. A bit like sftp.
 
 hints and or suggestion welcome.

If you want this, look at RFC2217 compatible programs...  RFC2217 extends
telnet to provide changing things like baud rates and parity inline...

The Python module PySerial includes both a client:
http://pyserial.sourceforge.net/examples.html#miniterm

and a server:
http://pyserial.sourceforge.net/examples.html#single-port-tcp-ip-serial-bridge-rfc-2217

Looks like ports has sredird that does the same for a server...

So, you could use a server attached the nmdm plus ssh port tunneling
to provide access...

I've been tempted to write a program using cuse that would present a
modem device that could be used w/ tip or cu and connect to an RFC2217
server on the back end giving you complete remove access to all settings
of the serial port, but haven't gotten around to doing that yet...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Boot OpenBSD with bhyve and libvirt automatically.

[John-Mark Gurney]

Peter Grehan wrote this message on Tue, Jan 06, 2015 at 08:45 -0800:
  Perhaps this is a deficiency that should be addressed in the bootloader
  compontent of libvirt to make this process a bit easier?
 
   I can't speak too much to libvirt since I didn't do that work, but 
 grub-bhyve is really a workaround to boot non-FreeBSD o/s's until a 
 UEFI/BIOS solution is done. If the changes to libvirt to support this 
 are large, it's most likely not worth it.

Hmm...  Nathan just did some work to make loader into a userland
program for calling Linux's kexec, could this work be leveraged?

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: backups of bhyve images

[John-Mark Gurney]

Pete Wright wrote this message on Mon, Dec 08, 2014 at 15:44 -0800:
 
 
 On 12/08/14 15:30, Craig Rodrigues wrote:
(3)  When you want to backup the VM, do a zfs snapshot take take a
  snapshot of the ZFS zvol.
 
 will this ensure that your zvol is consistent, or rather will the
 filesystem overlaid on the zvol device be ensured it is consistent when
 the hypervisor issues a snapshot command?

That's the beauty of FreeBSD... UFS provides this w/ soft updates, and
ZFS does this through COW...  In both cases, as far as I understand it,
it is safe to snapshot the FS...

 it's been a while since i've done this - but IIRC on NetApp WAFL systems
 (which are similar to zfs in terms of being a COW filesystem) you need
 to ensure the guest filesystem is in a consistent state before issuing a
 snapshot from it's parent.
 
 my data may be out of date since it's been several years since i've done
 this though...

With either UFS and ZFS, create a snapshot, and then do the copy of the
image from the snapshot... This will get you a consistant copy of the
image as if you had powered off the vm at that moment..  You'll still
have to fsck a UFS file system, but that is to be expected...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: backups of bhyve images

[John-Mark Gurney]

Pete Wright wrote this message on Wed, Dec 10, 2014 at 11:40 -0800:
 On 12/10/14 11:29, John-Mark Gurney wrote:
  Pete Wright wrote this message on Mon, Dec 08, 2014 at 15:44 -0800:
 
 
  On 12/08/14 15:30, Craig Rodrigues wrote:
(3)  When you want to backup the VM, do a zfs snapshot take take a
  snapshot of the ZFS zvol.
 
  will this ensure that your zvol is consistent, or rather will the
  filesystem overlaid on the zvol device be ensured it is consistent when
  the hypervisor issues a snapshot command?
  
  That's the beauty of FreeBSD... UFS provides this w/ soft updates, and
  ZFS does this through COW...  In both cases, as far as I understand it,
  it is safe to snapshot the FS...
 
 hrm not sure I think that is a reliable methodology.  You'll be forced
 to recover you VM's filesystem at best, and potentially have corrupt
 data if blocks were still sitting in cache and had not returned a
 filehande.  My guess is that this method will work with out issues
 %80-%90 of the time - but you still expose yourself to possible data
 loss or corruption.

If the application(s) do not properly handle data ordering issues
itself (making sure previous writes are completed using fsync), then
that application(s) will have issues if you suffer power loss at the wrong
moment, and you'll still have issues...

 I think you and others have mentioned the proper way to do this -
 snapshot the guess filesystem from with-in the guest VM itself.

Doing it in the VM is only marginally better...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: HEADS UP: Merging projects/bhyve_svm to HEAD

[John Nielsen]

On Oct 26, 2014, at 4:40 PM, Zaphod Beeblebrox zbee...@gmail.com wrote:

 On Sat, Oct 25, 2014 at 8:20 PM, Neel Natu neeln...@gmail.com wrote:
 
 On Sat, Oct 25, 2014 at 3:50 PM, Zaphod Beeblebrox zbee...@gmail.com
 wrote:
 I tried to integrate this patch into 10.1_RC3 and I failed.  Is there a
 timeframe to MFC this to 10.1 or 10-STABLE?
 
 It will be MFCed to 10-STABLE but I don't have a specific time frame in
 mind.
 
 I'll guess that it'll be towards the end of November but can be
 accelerated if someone has a need for this in 10-STABLE sooner.
 
 I would be using such a patch as soon as it was available.  On a friend's
 advice, I upgraded a ZFS server here at home with an AMD 9590 and 32Gig of
 RAM.  I'd dearly like to use it to track down the netgraph bug (see my
 other recent posts), but it doesn't currently qualify.

Ping? I'm also waiting with bated breath for this to be MFCed. :) Thanks for 
the great work!

JN

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

backups of bhyve images

[John]

Hello list,

I have a few questions about creating backups to be stored offsite.

If a guest is running, can I compress the image without it becoming 
inconsistent? If not, can it be copied without it becoming inconsistent?  
By inconsistent, I mean will I see weird effects and broken files if the 
backup is restored? Previously I've shut the VM down to avoid this, 
before archiving.

I have each image on its own (external to the image) ZFS filesystem.  
Internally the image is using ufs if freebsd, ext3fs if linux.  Would 
using some ZFS method of duplication be better? In this case, would the 
image become inconsistent?

Basically, what I want to do is to run accurate backups without shutting 
down and restarting the VM. Is this possible? If it isn't, I think the 
only alternative is to make a script that shuts the vm down, copies it, 
restarts the vm then runs its compression and backup-over-ssh routine.

thanks,

-- 
John 
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: FreeBSD i386 guest doesn't see console...

[John-Mark Gurney]

Peter Grehan wrote this message on Tue, Dec 02, 2014 at 19:27 -0800:
 This is w/ a resonably recent current i386 kernel (r275062)
 
  I verified the r274463 i386 snapshot boots fine. That's not quite as 
 recent, but:
 
 ...  Any suggestions?
 
 uart0: 16550 or compatible port 0x3f8-0x3ff irq 4 on acpi0
 ...
 uart0: 16550 or compatible port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 uart0: console (9600,n,8,1)
 
  The flags 0x10 text isn't being displayed in the i386 dmesg. This is 
 derived from /boot/device.hints i.e.
 
 ...
 hint.uart.0.flags=0x10
 ...
 
  You can see this from bhyveload by issuing a 'show' command to dump 
 all the environment, including what was read in from device.hints by the 
 loader, or more /boot/device.hints.

So, checking /boot/device.hints shows:
hint.uart.0.at=isa
hint.uart.0.port=0x3F8
hint.uart.0.flags=0x10
hint.uart.0.irq=4
hint.uart.1.at=isa
hint.uart.1.port=0x2F8
hint.uart.1.irq=3

  It may be possible that this file isn't being populated in your image, 
 or being read in by bhyveload.

show from loader:
OK show
LINES=24
boot_serial=1
console=userboot
currdev=disk0p3:
interpret=OK
kernelname=/boot/kernel/kernel
loaddev=disk0p3:
prompt=${interpret}
smbios.bios.vendor=BHYVE

Any ideas?

Oh, I did verify that uart0 is functional by logging into the system
and doing: echo foobar  /dev/cuau0

So, it's not like the console is not working, it's clearly a problem w/
the kernel deciding to use it as console..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: FreeBSD i386 guest doesn't see console...

[John-Mark Gurney]

Peter Grehan wrote this message on Wed, Dec 03, 2014 at 10:30 -0800:
   It may be possible that this file isn't being populated in your image,
 or being read in by bhyveload.
 
 show from loader:
 OK show
 LINES=24
 boot_serial=1
 console=userboot
 currdev=disk0p3:
 interpret=OK
 kernelname=/boot/kernel/kernel
 loaddev=disk0p3:
 prompt=${interpret}
 smbios.bios.vendor=BHYVE
 
 Any ideas?
 
  Looks like device.hints isn't being sourced in by the loader (aka 
 bhyveload in this case). This is done on a stock system from:
 
 % grep device.hints /boot/defaults/loader.conf
 loader_conf_files=/boot/device.hints /boot/loader.conf 
 /boot/loader.conf.local
 
  Any chance on your system that you've overridden loader_conf_files, or 
 are using a custom setup for loader config ?

Nope, my /boot/loader.conf looks like:
aesni_load=YES

That's all...  And /boot/defaults/loader.conf had the line:
loader_conf_files=/boot/device.hints /boot/loader.conf /boot/loader.conf.local

and, no, I don't have a loader.conf.local file..

Hmmm... I just realized that my i386 system doesn't have the beastie
menu when it boots, but my amd64 system does...  This image was
recently built using installworld/distribution w/ -DNO_ROOT, but the
steps to produce the image are exactly the same wrt amd64...

Ok, if I do a: set hint.uart.0.flags=0x10
in loader, and boot... serial console still doesn't work, but the flag
is now seen by uart0:
uart0: 16550 or compatible port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0

So, it's not just that flag that's the issue..

I also just did a diff between the amd64 and i386 /boot dir, and besides
the expected, boot blocks and loader differing, the only differences are
if_faith_load got removed and ata and apm hints are now present..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: FreeBSD i386 guest doesn't see console...

[John-Mark Gurney]

John-Mark Gurney wrote this message on Wed, Dec 03, 2014 at 10:50 -0800:
 I also just did a diff between the amd64 and i386 /boot dir, and besides
 the expected, boot blocks and loader differing, the only differences are
 if_faith_load got removed and ata and apm hints are now present..

Don't know how my diff missed loader.rc missing, but it did...

So, after some investigation, it appears that loader.rc and friends
weren't installed, though loader and many other things were... I
believe I have run into this issue before multiple times, but didn't
know what was the problem, and a new installworld fixed the issue...

So, there is an issue w/ our installworld, but tracking it down will
probably be hard.. I've had this happen on both local UFS, and NFS
backed by ZFS...  It could be related to the fact that bmake doesn't
always error out when an error occurs (reported), and so it appears
that installworld completes successfully, but doesn't...

Also, I was able to reliably reproduce (in different locations) a
failure in installing tests that failed w/ a permission denied...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

FreeBSD i386 guest doesn't see console...

[John-Mark Gurney]

So, I tried to boot a i386 guest under bhyve, and no console was
detected...

an amd64 guest has console fine...

This is w/ a resonably recent current i386 kernel (r275062)...  Any
suggestions?

i386 guest dmesg:
Hypervisor: Origin = bhyve bhyve 
real memory  = 536870912 (512 MB)
avail memory = 495017984 (472 MB)
Event timer LAPIC quality 400
ACPI APIC Table: BHYVE  BVMADT  
[...]
uart0: 16550 or compatible port 0x3f8-0x3ff irq 4 on acpi0

(no second line, no output after loading kernel when running vmrun.sh)...

amd64 guest dmesg:
Hypervisor: Origin = bhyve bhyve 
real memory  = 18253611008 (17408 MB)
avail memory = 16573952000 (15806 MB)
Event timer LAPIC quality 400
ACPI APIC Table: BHYVE  BVMADT  
[...]
uart0: 16550 or compatible port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (9600,n,8,1)

I can provide more if you'd like...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: 'repz' instruction not supported.

[John Baldwin]

On Thursday, November 20, 2014 1:30:41 am Ashutosh Kumar wrote:
 Hi
 We have found that emulation of 'repz' instruction is not supported in 
bhyve. This is causing VM_EXIT for Guest OS. Do we have plans to add support 
for instructions like 'repz'. 

What are you using repz for?  Is it memory access to some device?

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve remote kgdb does not support kernel modules...

[John Baldwin]

On Saturday, November 15, 2014 1:15:23 pm John-Mark Gurney wrote:
 Julian Elischer wrote this message on Sun, Nov 16, 2014 at 00:23 +0800:
  On 11/15/14, 5:43 PM, John-Mark Gurney wrote:
  John-Mark Gurney wrote this message on Sat, Nov 15, 2014 at 01:37 -0800:
  So, I got remote kgdb working w/ bhyve, but kernel modules aren't
  loaded...
  
  I'm getting:
  (kgdb) bt
  #0  kdb_enter (why=0x8102ee88 panic, msg=value optimized 
out)
   at ../../../kern/subr_kdb.c:444
  #1  0x8094aa79 in vpanic (fmt=value optimized out,
   ap=value optimized out) at ../../../kern/kern_shutdown.c:739
  #2  0x8094a8c9 in kassert_panic (fmt=value optimized out)
   at ../../../kern/kern_shutdown.c:634
  #3  0x8093583c in __mtx_lock_flags (c=0xf80002919720, 
opts=0,
   file=0x81c3f0fa 
   
/usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c,
 
line=442) at ../../../kern/kern_mutex.c:217
  #4  0x81c3e3a7 in ?? ()
  #5  0xf80002a474a0 in ?? ()
  #6  0xf80002919720 in ?? ()
  
  Where those should be in the aesni.ko module...  If I run kgdb on a
  local machine, it properly finds the kernel modules... Any hits on
  how to fix this?
  
  works for me..
  the modules I'm debugging are not compiled in.
  you have to have the modules in the standard place ONE THE GDB HOST.
  I make a chroot with an image of the machine being debugged, PLUS the 
  sources.
  and then run kgdb from inside that chroot.
 
 Ahh, that's right, kgdb isn't smart enough to know when you've pointed
 it to the kernel build dir whereh the modules are... :(

gdb isn't smart enough if you run with all the shared libraries in the
current directory either.  Note you can always use 'add-kld /path/to/foo.ko'
when it can't find a useful module path.  Also, you don't have to use a
chroot.  You can use 'solib-absolute-prefix' (it's documented for shared
libraries and kgdb treats modules as shared libraries) and set it to the 
DESTDIR you use when you install the kernel.

That is:

% cd /path/to/kernel/amd64/compile/FOO
% make
% make install BINOWN=you DESTDIR=/someplace
% kgdb kernel.debug
(kgdb) set solib-absolute-prefix /someplace/
(kgdb) target remote foo

(I have a local patch to vmrun.sh on my laptop that always ties a nmdm 
interface on COM2 if nmdm is loaded so I can now attach kgdb to any VM
without having to plan ahead).

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: RFC: Enabling VIMAGE in GENERIC

[John-Mark Gurney]

Alexander V. Chernikov wrote this message on Wed, Nov 19, 2014 at 16:07 +0400:
 Can we have some wiki/man/docs on how particular subsystem should 
 interact with VNET first?

Yes, we need a man page talking about this feature first, how to enable
it, compile it into the kernel, how to manage it, what subsystems it
interacts w/, what sysctl nodes it provides, etc.

W/o man page(s) the feature is not complete.

$ man -k vnet
revnetgroup(8)   - generate reverse netgroup data
$ man -k vimage
XvCreateImage(3), XvShmCreateImage(3) - create an XvImage
XvPutImage(3), XvShmPutImage(3) - display an XvImage

hmm.. nope...  jail has something about vnets, but not nearly enough
to be useful...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

bhyve remote kgdb does not support kernel modules...

[John-Mark Gurney]

So, I got remote kgdb working w/ bhyve, but kernel modules aren't
loaded...

I'm getting:
(kgdb) bt
#0  kdb_enter (why=0x8102ee88 panic, msg=value optimized out)
at ../../../kern/subr_kdb.c:444
#1  0x8094aa79 in vpanic (fmt=value optimized out,
ap=value optimized out) at ../../../kern/kern_shutdown.c:739
#2  0x8094a8c9 in kassert_panic (fmt=value optimized out)
at ../../../kern/kern_shutdown.c:634
#3  0x8093583c in __mtx_lock_flags (c=0xf80002919720, opts=0,
file=0x81c3f0fa 
/usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c,
 line=442) at ../../../kern/kern_mutex.c:217
#4  0x81c3e3a7 in ?? ()
#5  0xf80002a474a0 in ?? ()
#6  0xf80002919720 in ?? ()

Where those should be in the aesni.ko module...  If I run kgdb on a
local machine, it properly finds the kernel modules... Any hits on
how to fix this?

Thanks.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve remote kgdb does not support kernel modules...

[John-Mark Gurney]

John-Mark Gurney wrote this message on Sat, Nov 15, 2014 at 01:37 -0800:
 So, I got remote kgdb working w/ bhyve, but kernel modules aren't
 loaded...
 
 I'm getting:
 (kgdb) bt
 #0  kdb_enter (why=0x8102ee88 panic, msg=value optimized out)
 at ../../../kern/subr_kdb.c:444
 #1  0x8094aa79 in vpanic (fmt=value optimized out,
 ap=value optimized out) at ../../../kern/kern_shutdown.c:739
 #2  0x8094a8c9 in kassert_panic (fmt=value optimized out)
 at ../../../kern/kern_shutdown.c:634
 #3  0x8093583c in __mtx_lock_flags (c=0xf80002919720, opts=0,
 file=0x81c3f0fa 
 /usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c,
  line=442) at ../../../kern/kern_mutex.c:217
 #4  0x81c3e3a7 in ?? ()
 #5  0xf80002a474a0 in ?? ()
 #6  0xf80002919720 in ?? ()
 
 Where those should be in the aesni.ko module...  If I run kgdb on a
 local machine, it properly finds the kernel modules... Any hits on
 how to fix this?

Also, any plans to support the Z2 write-watchpoint packet?

(kgdb) watch *(char *)0xf80002919708
Hardware watchpoint 1: *(char *) 18446735277659625224
(kgdb) c
Continuing.
Can't clear hardware watchpoints without the 'Z2' (write-watchpoint) packet

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve remote kgdb does not support kernel modules...

[John-Mark Gurney]

Julian Elischer wrote this message on Sun, Nov 16, 2014 at 00:23 +0800:
 On 11/15/14, 5:43 PM, John-Mark Gurney wrote:
 John-Mark Gurney wrote this message on Sat, Nov 15, 2014 at 01:37 -0800:
 So, I got remote kgdb working w/ bhyve, but kernel modules aren't
 loaded...
 
 I'm getting:
 (kgdb) bt
 #0  kdb_enter (why=0x8102ee88 panic, msg=value optimized out)
  at ../../../kern/subr_kdb.c:444
 #1  0x8094aa79 in vpanic (fmt=value optimized out,
  ap=value optimized out) at ../../../kern/kern_shutdown.c:739
 #2  0x8094a8c9 in kassert_panic (fmt=value optimized out)
  at ../../../kern/kern_shutdown.c:634
 #3  0x8093583c in __mtx_lock_flags (c=0xf80002919720, opts=0,
  file=0x81c3f0fa 
  
  /usr/home/jmg/freebsd.p4/opencrypto/sys/modules/aesni/../../crypto/aesni/aesni.c,
   line=442) at ../../../kern/kern_mutex.c:217
 #4  0x81c3e3a7 in ?? ()
 #5  0xf80002a474a0 in ?? ()
 #6  0xf80002919720 in ?? ()
 
 Where those should be in the aesni.ko module...  If I run kgdb on a
 local machine, it properly finds the kernel modules... Any hits on
 how to fix this?
 
 works for me..
 the modules I'm debugging are not compiled in.
 you have to have the modules in the standard place ONE THE GDB HOST.
 I make a chroot with an image of the machine being debugged, PLUS the 
 sources.
 and then run kgdb from inside that chroot.

Ahh, that's right, kgdb isn't smart enough to know when you've pointed
it to the kernel build dir whereh the modules are... :(

Thanks for the reminder...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: how to for making bhyve images (using makefs + mkimg)

[John-Mark Gurney]

Alfred Perlstein wrote this message on Thu, Oct 30, 2014 at 22:37 -0700:
 On 10/29/14 9:34 PM, John-Mark Gurney wrote:
 So, Michael Dexter prodded me to document how I'm doing it, so I put
 up a blog post at:
 http://blog.funkthat.com/2014/10/building-bhyve-images-using-makefs-and.html
 
 It's a quick guide on how to build images using makefs and mkimg.  Only
 time you need root is when you run the VM...
 
 btw, thanks for getting bhyve running on AMD! :)
 
 John-Mark, very cool.
 
 Thanks for pointing me at mkimg, will have to look at porting our build 
 to use that as opposed to mdconfig.

np...  It's cool..  though things get built a bit different w/ mkimg
wrt to partiion alignment.. so make sure things are good, i.e. 4k
aligned for most hard drives...  One easy way to do this is to use
-P 4k...

Enjoy!

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

how to for making bhyve images (using makefs + mkimg)

[John-Mark Gurney]

So, Michael Dexter prodded me to document how I'm doing it, so I put
up a blog post at:
http://blog.funkthat.com/2014/10/building-bhyve-images-using-makefs-and.html

It's a quick guide on how to build images using makefs and mkimg.  Only
time you need root is when you run the VM...

btw, thanks for getting bhyve running on AMD! :)

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: NATed or Private Network Setups

[John Baldwin]

On Friday, October 24, 2014 04:08:27 PM Pete Wright wrote:
 Hi All,
 Has anyone deployed bhyve using NAT'd or private network setups?  I've
 been able to deploy bridged interfaces, but I was wondering if anyone
 has done other network topologies.  Is there anything preventing this
 from happening code wise?  I reckon it could be achieved by creating a
 pseudo interface?

I setup a bridge on my laptop and add all the tap interfaces for VMs as 
members to the bridge.  I use a /24 for the internal LAN for these 
interfaces and assign the .1 to the bridge0 interface itself.  I then run 
dnsmasq to provide DHCP/DNS to the VMs and use natd (ipfw_nat would also work) 
to allow the VMs NAT access to the outside world.  There are more details in 
an article in the most recent issue of the FreeBSD Journal, but I'll push that 
into the regular FreeBSD docs at some point as well.

With the dnsmasq setup, I put the vmname as the hostname so that it is sent in 
the dhclient request.  dnsmasq then adds local overrides for VMs while they 
are active.  (So you can 'ssh vm0' on the host, or from another vm.)  The 
'host' entry in /etc/hosts is also snarfed up by dnsmasq so that within a vm I 
can use 'host' as a hostname (e.g. for NFS mounting something off of my 
laptop).

Some config file snippets:

/etc/sysctl.conf:

net.link.tap.up_on_open=1

/etc/rc.conf:

# bhyve setup
autobridge_interfaces=bridge0
autobridge_bridge0=tap*
cloned_interfaces=bridge0 tap0 tap1 tap2
ifconfig_bridge0=inet 192.168.16.1/24
gateway_enable=YES
natd_enable=YES
natd_interface=wlan0
dnsmasq_enable=YES
firewall_enable=YES
firewall_type=/etc/rc.firewall.pippin

/etc/hosts:

192.168.16.1host

/etc/resolvconf.conf:
 
name_servers=127.0.0.1
dnsmasq_conf=/etc/dnsmasq-conf.conf
dnsmasq_resolv=/etc/dnsmasq-resolv.conf

/usr/local/etc/dnsmasq.conf:

domain-needed
bogus-priv
resolv-file=/etc/dnsmasq-resolv.conf
interface=bridge0
dhcp-range=192.168.16.10,192.168.16.200,12h
conf-file=/etc/dnsmasq-conf.conf

/etc/rc.firewall.pippin:

# prevent inbound traffic for our guest /24
add denyall from any to 192.168.16.0/24 via em0
add denyall from any to 192.168.16.0/24 via wlan0

# divert packets between guest and outside world to natd
add divert  natd all from any to any via wlan0

# prevent outbound traffic for our guest /24
add denyall from 192.168.16.0/24 to any via em0
add denyall from 192.168.16.0/24 to any via wlan0

# pass everything else
add allow   all from any to any

(I have not figured out a way to have the NAT prefer em0 if present and fail 
over to wlan0 if not, etc.)

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Rework hypervisor detection

[John Baldwin]

On Monday, October 27, 2014 12:47:13 PM Conrad Meyer wrote:
 Hi John,
 
 I don't have access to phabricator so I'll try and review in email.
 
  +#ifdef XEN
  +int vm_guest = VM_GUEST_XEN;
  +#else
  +int vm_guest = VM_GUEST_NO; /* Running as virtual machine guest? */
  +#endif
 
 The whitespace between 'int' and 'vm_guest' in the second line seems
 wrong (extra space?).

Yep, fixed.
 
 For vmware_hvcall() in x86/include/vmware.h, are you sure the inline
 assembly is correct? Have you tried the detection on a VMware VM?
 VMware's documentation specifies the arguments in a different
 order[0], and at Isilon when we tried the port-based detection in a
 slightly older revision of CURRENT, it didn't work (IIRC).

I don't have VMware handy, but it is a simple copy and paste of existing
code in tsc.c.

-- 
John Baldwin
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: moving from virtualbox to bhyve

[John]

Hi,

On Tue, Oct 21, 2014 at 05:11:07AM -0400, Cory Smelosky wrote:
 
 You can convert the image without data loss. Qemu-img and 
 VBoxManage provide a means to do as such.

I know, I had already converted it to raw. The question was,
how do I launch it, but after some experimentation and reading, 
I *think* I may have sussed it out. So far, the instance boots
and loads its services (it's a web server) but it's hanging halfway
through the loading process, and I can't make it abort loading them
with ctrl-c for example.

cheers,
-- 
John
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

bhyvectl man page?

[John-Mark Gurney]

Can someone write a man page for this tool?

I'm willing to do the formating if someone writes the text...

Thanks.

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

testing bhyve on AMD box...

[John-Mark Gurney]

So, I did, and it failed...

First, if_tap doesn't get automatically loaded.. but that was easy to
spot due to the error message...  loader runs fine, but then after
hitting enter, I get the following:
/tmp/bhyve.4dv5wFK 91: [0008]   FACS Address : 000F27C0
Error6154 - Integer too large for target ^  (000F27C0)

/tmp/bhyve.4dv5wFK 92: [0008]  DSDT Address : 000F2800
Error6154 - Integer too large for target ^  (000F2800)

/tmp/bhyve.4dv5wFK 98: [0008]  Address : 0400
Error6154 - ^ Integer too large for target 
(0400)

/tmp/bhyve.4dv5wFK112: [0008]Address : 0404
Error6154 - ^ Integer too large for target 
(0404)

/tmp/bhyve.4dv5wFK133: [0008]Address : 0408
Error6154 - ^ Integer too large for target 
(0408)

Assertion failed: (error == 0), function main, file 
/usr/src/usr.sbin/bhyve/bhyverun.c, line 864.
Abort trap (core dumped)

I had to reformat it since the terminal wasn't emitting CR's...

This is from:
FreeBSD carbon.funkthat.com 11.0-CURRENT FreeBSD 11.0-CURRENT #9: Tue Oct 21 
10:55:05 PDT 2014 
j...@carbon.funkthat.com:/usr/home/jmg/freebsd.p4/opencrypto/sys/amd64/compile/GENERIC
  amd64

Though this is from my p4 project branch, it is fully synced w/ r273388...

CPU: AMD A10-5700 APU with Radeon(tm) HD Graphics(3393.89-MHz K8-class CPU)
  Origin=AuthenticAMD  Id=0x610f01  Family=0x15  Model=0x10  Stepping=1
  
Features=0x178bfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT
  
Features2=0x3e98320bSSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C
  AMD Features=0x2e500800SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM
  AMD 
Features2=0x1ebbfffLAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC
  Structured Extended Features=0x8BMI1
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 7665848320 (7310 MB)

I'm willing to test any patches you throw my way...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: testing bhyve on AMD box...

[John-Mark Gurney]

Neel Natu wrote this message on Tue, Oct 21, 2014 at 12:01 -0700:
 On Tue, Oct 21, 2014 at 11:50 AM, John-Mark Gurney j...@funkthat.com wrote:
  So, I did, and it failed...
 
  First, if_tap doesn't get automatically loaded.. but that was easy to
  spot due to the error message...  loader runs fine, but then after
  hitting enter, I get the following:
  /tmp/bhyve.4dv5wFK 91: [0008]   FACS Address : 
  000F27C0
  Error6154 - Integer too large for target ^  (000F27C0)
 
  /tmp/bhyve.4dv5wFK 92: [0008]  DSDT Address : 000F2800
  Error6154 - Integer too large for target ^  (000F2800)
 
  /tmp/bhyve.4dv5wFK 98: [0008]  Address : 0400
  Error6154 - ^ Integer too large for target 
  (0400)
 
  /tmp/bhyve.4dv5wFK112: [0008]Address : 0404
  Error6154 - ^ Integer too large for target 
  (0404)
 
  /tmp/bhyve.4dv5wFK133: [0008]Address : 0408
  Error6154 - ^ Integer too large for target 
  (0408)
 
  Assertion failed: (error == 0), function main, file 
  /usr/src/usr.sbin/bhyve/bhyverun.c, line 864.
  Abort trap (core dumped)
 
  I had to reformat it since the terminal wasn't emitting CR's...
 
  This is from:
  FreeBSD carbon.funkthat.com 11.0-CURRENT FreeBSD 11.0-CURRENT #9: Tue Oct 
  21 10:55:05 PDT 2014 
  j...@carbon.funkthat.com:/usr/home/jmg/freebsd.p4/opencrypto/sys/amd64/compile/GENERIC
amd64
 
  Though this is from my p4 project branch, it is fully synced w/ r273388...
 
  CPU: AMD A10-5700 APU with Radeon(tm) HD Graphics(3393.89-MHz K8-class 
  CPU)
Origin=AuthenticAMD  Id=0x610f01  Family=0x15  Model=0x10  Stepping=1

  Features=0x178bfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT

  Features2=0x3e98320bSSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C
AMD Features=0x2e500800SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM
AMD 
  Features2=0x1ebbfffLAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC
Structured Extended Features=0x8BMI1
TSC: P-state invariant, performance statistics
  real memory  = 8589934592 (8192 MB)
  avail memory = 7665848320 (7310 MB)
 
  I'm willing to test any patches you throw my way...
 
 
 What is the output of iasl -v on your system?
 
 This is what's on my AMD machine:
 
 [neel@a ~/head]$ iasl -v
 
 Intel ACPI Component Architecture
 ASL Optimizing Compiler version 20140926-64
 Copyright (c) 2000 - 2014 Intel Corporation

$ iasl -v

Intel ACPI Component Architecture
ASL Optimizing Compiler version 20130823-64
Copyright (c) 2000 - 2013 Intel Corporation

Sorry, I was so excited to try it out that I didn't do a buildworld...
I'll do one now, since I'm still getting a failure after updating to a
current iasl:
Assertion failed: (CPU_ISSET(vcpu, active_cpus)), function vm_loop, file 
/usr/src/usr.sbin/bhyve/bhyverun.c, line 597.
Abort trap (core dumped)

I'll report back success or failure after buildworld/installworld/reboot
is complete...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: moving from virtualbox to bhyve - SOLVED

[John]

On Tue, Oct 21, 2014 at 02:02:17PM +0100, John wrote:
 Hi,
 
 On Tue, Oct 21, 2014 at 05:11:07AM -0400, Cory Smelosky wrote:
  
  You can convert the image without data loss. Qemu-img and 
  VBoxManage provide a means to do as such.
 
 I know, I had already converted it to raw. The question was,
 how do I launch it, but after some experimentation and reading, 
 I *think* I may have sussed it out. So far, the instance boots
 and loads its services (it's a web server) but it's hanging halfway
 through the loading process, and I can't make it abort loading them
 with ctrl-c for example.

I was so busy waiting for the login screen from the console that I'd failed
to notice it *HAD* actually booted. I found this out by trying
to ssh to it for the hell of it. dmesg showed things were normal.
My console, though, looks like this: http://tinyurl.com/q4a3t4e
The ubuntu is 13.10 with security patches.

cheers
-- 
John 
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: moving from virtualbox to bhyve - SOLVED

[John]

On Tue, Oct 21, 2014 at 01:55:26PM -0700, Neel Natu wrote:

 If you are comfortable sharing your disk image then I can try to
 reproduce locally and hopefully get a better grip on what's happening.

That's very nice of you to offer, but I have to decline as I don't own
the data. I think I might have sorted it in any case. I was thrown by 
the login prompt not appearing in the console. 

Really pleased with the performance of ubuntu on bhyve. It doesn't impact
much on the host either. 
-- 
John 
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: moving from virtualbox to bhyve

[John]

On Mon, Oct 20, 2014 at 12:31:18PM -0400, Allan Jude wrote:

 What version of FreeBSD are you using? You might want to try the latest
 10.1 RC (or HEAD if you are so included) instead, as they have much
 improved bhyve.

freebsd version is: 10.1-PRERELEASE #0 r273275
ports is 371224

I'm running 10-STABLE
-- 
John 
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: can a bhyve instance be resized? adding another virtual disk?

[John Nielsen]

On Oct 15, 2014, at 5:56 AM, freebsd-li...@potato.growveg.org wrote:

 Can a bhyve instance be resized? I'm talking about the disk. 
 Say your end user needs more diskspace. They have 32GB. They need 64GB.
 How do you do it? I presume one has to stop the guest, then use truncate.
 What about if the guest OS isn't freebsd, and they use say ext2 or 3? Will
 ext3 start yelling at me because I've resized it?

This isn't specific to FreeBSD or bhyve. Virtio block devices actually can 
support online resizing, but I don't know if bhyve allows that yet (I'm 
assuming it doesn't). In which case, yes, stop the guest and resize whatever 
its volume lives on (if a raw file then truncate would work), then start it up 
again. That's the easy part.

The harder part (but much easier than it used to be) is resizing everything 
else. If using partitions they need to be extended first (and if using GPT the 
backup partition table needs to be moved first of all, a la gpart recover.) 
On FreeBSD this is pretty straightforward with gpart:
sysctl kern.geom.debugflags=16
gpart resize -i $number_of_last_partition $underlying_block_device

You should probably reboot at this point so the kernel forgets about the old 
partition table.

Then you get to resize the filesystem. If you are using ZFS or if you have 
FreeBSD 9.2 or newer and UFS then you can do it while it is mounted. Otherwise 
you may need to boot from another source to do the resize. For UFS use growfs a 
la growfs /dev/$block_special_for_partition. For ZFS use zpool online -e 
$zpool $zdev

For ext[234] on Linux, use resize2fs /dev/$block_special. (If using LVM then 
you need to first resize the LV with lvextend). For XFS use xfs_growfs 
$mountpoint. You can also resize btrfs but I don't know the command off the 
top of my head.

That should be it.

 What if they just want another disk? How does one refer to a 
 newly created virtual disk from a guest? How is it mounted to the guest?

Just add a -d /path/to/new/device to your vmrun.sh or the corresponding -s to 
bhyve when you start the VM. It will show up as a new block device in the guest 
(e.g. /dev/vtbd1), you can partition and/or put filesystems on it as you choose 
and mount them normally and/or add them to /etc/fstab, etc.

JN

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Enabling VIMAGE by default for FreeBSD 11?

[John D. Hendrickson and Sara Darnell]

Alexander V. Chernikov wrote:

On 11 Oct 2014, at 21:58, Craig Rodrigues rodr...@freebsd.org wrote:


Hi,

What action items are left to enable VIMAGE by default for FreeBSD 11?


well the next step is to make it a dependancy so that free bsd won't 
install without it, and to inject it in many binaries that insure 
it's in use.  like ssh !


the key is: 98 117C FE83 22EA B843 3E86  6486 4320 545E 1B2A FA1C

just change anything you don't like in any jails being upgraded and 
tada you may get what's protected!  can i have some?

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: Enabling VIMAGE by default for FreeBSD 11?

[John D. Hendrickson and Sara Darnell]

Alexander V. Chernikov wrote:

On 11 Oct 2014, at 21:58, Craig Rodrigues rodr...@freebsd.org wrote:


Hi,

What action items are left to enable VIMAGE by default for FreeBSD 11?

Are there any tests results showing performance implications on different 
network-related workloads?

Not everyone uses bhyve, so VIMAGE is quite useful when using jails.

--
Craig
___
freebsd-...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org



___
freebsd-a...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-arch
To unsubscribe, send any mail to freebsd-arch-unsubscr...@freebsd.org



i know little about chroot jails or 7 ring processor levels

but let me ask rhetorically ...

do you mean VIMAGE allows a jail to use an iface device for many IPs 
or even MAC?  i thought that was already the case all cards can 
listen - it's only a headers trick per say.


but do you mean a chroot can have access to an iface (which there 
are pkg for setting up if i remember)?  but if a jail is allowed to 
use an iface why not allocate it - meaning: what is the purpose of 
middleman vimage connecting device to jail unless there is a strict 
filter inbetween (ie, strippign headers, or even controlling what 
iface/routes are alllowed)?


i can't see what it's for, but much less making it mandatorily 
injected upon all jailsm, except maybe it may BREAK existing jails 
by allowing net access where there is NOT supposed to be any / 
assumed not to be any


if they old programmers didn't want anyone compiling software who 
logged in: they'd insure there was no compiler.  if they didn't want 
typing at a terminal: they'd take away the keyboard right?


___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: no routing from new bhyve install

[John]

Answered my own question - the machine froze and rebooted. Now bhyve works as 
expected. Looks like it's as fast as bare metal, which is unexpected. 
-- 
John 
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org

Re: bhyve and zfs and speed

[John]

On Wed, Sep 03, 2014 at 12:15:28AM +0200, Nikolay Denev wrote:

 I see you have 108G free memory. Is this due to a recent reboot and
 the ARC hasn't grown much yet, or you are limiting it?

not doing anything to it - it's running defaults. Two virtual machines
are running.

 As for ZFS tuning for bhyve, I'm not aware of anything specific, but
 you should consider that values you want to put for the zvols for
 primarycache and secondarycache. (In your case only primarycache as
 you don't have L2ARC).

ok, thanks.

 Leaving this to the default might be faster but might result in double
 caching: Once in the ARC on the host system, and then on the buffer
 cache of the UFS guest.

ok - will look to see how to turn off double buffering

 For MySQL and databases in general, as far as I can remember the
 general advice was to run them on zfs datasets with lower reclen, like
 8k for example.
 Here you have zvols which default to 8k but the vtbd device reports
 512b sectors, not sure if you try to trick it into using larger values
 here wouldn't help.
 Also different UFS block sizes for different zvol recordlens might be
 interesting test.

Thanks again for the advice

-- 
John 
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org