Re: [smartos-discuss] Smartos bhyve support on AMD Sun Fire X4140

[Ján Poctavek]

Hi,

Just to be sure - so no AMD support again?

Jan

On 3. 7. 2018 11:42, Jorge Schrauwen wrote:

For now bhyve support on SmartOS requires VMX and EPT to work.
So older intel CPU without EPT or AMD CPU that use SVM are not support.

Regards

Jorge

July 3, 2018 11:15 AM, "Paolo Marcheschi" > 
wrote:


Hi

Today I tried to run a bhyve VM on the latest Smartos:

SunOS 5.11 joyent_20180629T143106Z i86pc i386 i86pc

The server is an AMD Opteron Sun Fire X4140 :

psrinfo -vp The physical processor has 6 virtual processors (0-5) x86 
(AuthenticAMD 100F80 family 16 model 8 step 0 clock 2200 MHz) Six-Core AMD 
Opteron(tm) Processor 2427 [ Socket: F(1207) ] The physical processor has 6 
virtual processors (6-11) x86 (AuthenticAMD 100F80 family 16 model 8 step 0 
clock 2200 MHz) Six-Core AMD Opteron(tm) Processor 2427 [ Socket: F(1207) ]

when I try to create a bhyve VM I get:
#vmadm create -f centos.json
Bhyve not supported
Why ?
Thank you
Paolo



*smartos-discuss* | Archives 
 | Modify 
 Your Subscription 
	[Powered by Listbox] 







---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] failure to boot on Intel Core i7 8700 Hexa-Core Processor (Coffee Lake) and Gigabyte Z370XP SLI motherboard

[Ján Poctavek]

Hi dewey,

I think it's worth dropping a line into the github thread. The 
information that you've booted without the need of moddebug and 
disable-xhci might be useful for others.


YanChii


On 13. 2. 2018 0:00, de...@hyltown.com wrote:

Is this information somehow worth adding to the open issue, or does this just
amount to "me too" ?

https://github.com/joyent/smartos-live/issues/727

- On Feb 10, 2018, at 10:47 PM, Dewey de...@hyltown.com wrote:


With os_console set to ttya, following the procedure described here:
https://github.com/joyent/smartos-live/issues/727#issuecomment-342868065

I was able to get all the way to SmartOS Setup. I haven't attempted to do the
actual installation yet, because this isn't the way I want to boot it every
time.

What I found, which differs from what YanChii noticed, is that I need neither
disable-xhci=true nor moddebug to get this far. Typing this into the debugger
is all that is needed for me:

::bp -Dn 1 -c '.+0x95::bp -c "0>dx;::cont" ; ::cont' acpica`AcpiOsWritePort
:c

I am testing with: joyent_20180203T031130Z

Please let me know whether this helps, and how to proceed from here.

- On Feb 10, 2018, at 5:55 PM, Dewey de...@hyltown.com wrote:


There is, indeed (it also has ps2 ports, which I did not realize). I managed
to find what appears to be the proper cabling to go from header to db9; now
to find my old null modem stuff.

Can you point me to the the docs I should start with, or is there something
very specific you'd like me to try first?

- On Feb 10, 2018, at 2:39 PM, Robert Mustacchi r...@joyent.com wrote:


Is there a serial header that we can use for kmdb on that system? It may
be useful to try and use the module auto load / breakpoint system and/or
maybe disable the boot of other CPUs to try and debug.

Robert









---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Create network overlays after boot

[Ján Poctavek]

Hi Dan,

Thank you for your hints, they are very valuable. The -h flag is a 
really good idea. And I'll also look deeper into ikeadm dump *.


Regarding the DPD, lowering the p2_softlife_secs seems to do the job and 
works even after ikeadm flush on the remote node.


Thanks.

Jan

On 8. 2. 2018 16:55, Dan McDonald wrote:

Pardon the shortness here.

One warning up front is that in.iked is closed-source, so IF there are any 
issues with IKE per se, we are not able to fix them.


On Feb 8, 2018, at 6:08 AM, Ján Poctavek <jan.pocta...@erigones.com> wrote:

Hi Dan,

Yes we know about the IKEv2 and we'll be happy to try it.

You might also be interested in our ansible playbook that sets up IPsec and 
overlays:
https://github.com/erigones/esdc-ce/tree/master/ans/overlays
And our simple IPSsec debug scripts:
https://github.com/erigones/esdc-ce/tree/master/bin/debug

Thanks for sharing.


Any feedback on our docs or usage of IPsec is totally welcome. Don't hesitate 
to ask anything.

Thank you.


One more thing - as you are probably right person to ask - currently we are 
trying to set up a reliable Dead Peer Detection so the IPsec can overcome a 
remote node reboot or ipseckey flush. Is there any doc that can point us how to 
do that? I'm a bit lost in all IKE options.

DPD should Just Work on IKE, insofar as I recall.  SAs have IDLE timeouts which 
in.iked monitors. The trick is whether or not one side is behind a NAT or not.  
If both sides are NAT-free, what should happen is:

- IDLE timeout occurs

- SAs (and IKE SA) get deleted.

- Next outbound traffic starts fresh with a PF_KEY ACQUIRE message, and 
a fresh IKE exchange.

If a NAT is in the way, things get tricky, because only the node *behind* the 
NAT can initiate cleanly, UNLESS you're doing static port mapping.

Now I just looked at your debug scripts.  I've noticed some things:

1.) You do not query the highly-queriable in.iked via ikeadm(1M).  You use ikeadm to 
enable full-blown IKE logging, but there's more you can do.  Try "ikeadm dump 
p1", for example.  :)

2.) I would recommend having the option of using the -n flag for your 
invocations of IPsec utilities JUST IN CASE your name services are down.  If 
you're only using files, though, you can ignore this.

3.) You may not be aware of this, but if you utter "ipseckey flush" IKE *also* 
deletes all of its IKE SAs as well.  The ipsec_restart.sh script MAY be redundant, unless 
in.iked is seriously hosed.

Dan





---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Create network overlays after boot

[Ján Poctavek]

Hi Dan,

Yes we know about the IKEv2 and we'll be happy to try it.

You might also be interested in our ansible playbook that sets up IPsec 
and overlays:

https://github.com/erigones/esdc-ce/tree/master/ans/overlays
And our simple IPSsec debug scripts:
https://github.com/erigones/esdc-ce/tree/master/bin/debug

Any feedback on our docs or usage of IPsec is totally welcome. Don't 
hesitate to ask anything.


One more thing - as you are probably right person to ask - currently we 
are trying to set up a reliable Dead Peer Detection so the IPsec can 
overcome a remote node reboot or ipseckey flush. Is there any doc that 
can point us how to do that? I'm a bit lost in all IKE options.


Thank you.

Jan


On 7. 2. 2018 17:51, Dan McDonald wrote:

I'm so glad to see you're using the built-in illumos IPsec features for your 
overlay improvements.  If you don't already know, we're working on IKEv2 for 
illumos-via-SmartOS.  As we move along with it, it would be interesting to have 
your feedback on it.

Thanks for this update, and the documentation links.  I'll need to give them 
(esp. the IPsec-specific bits) a bit of a deeper read.

Thanks!
Dan




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: AW: [smartos-discuss] Live migration

[Ján Poctavek]

Hi Daniel,

Live migration is already in qemu/KVM. I was not reinventing it, just 
making it work by understanding the principles and correcting the way it 
works in the illumos.


On 9. 11. 2017 1:08, Daniel Plominski wrote:


I think the patches are a bit small overall, live migration is quite 
complex


  * 
https://developers.redhat.com/blog/2015/03/24/live-migrating-qemu-kvm-virtual-machines/


If you find something not working, please let me know.


How do you solve the problem with the page (vm memory state) transfer?


Dirty pages tracking. Standard qemu and KVM way.


How do you solve the problem with the storage transfer?

  * a zfs snapshot on an active zvolume, without manual sync (inside
the vm and outside the zfs txg_sync) and freeze of the kvm, i
think that is not a good idea for a consistent block storage state

This is the most tricky part. You can start VM disk migration in the 
background and after you finish, you can do it again using differential 
snapshot transferring just the newly written bits. But the main problem 
is that when you start the destination zone (= qemu waiting for incoming 
migration stream), the qemu opens the destination disk zvols and after 
that point, you are not able to receive zfs incremental stream anymore 
(it fails with device is busy because zvol is open). You have several 
options here:


1. The best one that is not done at the moment: upgrade qemu. The new 
qemu supports custom dirty block tracking maps for disk devices. That 
plays nicely with zfs send/receive. For more info read here:

https://patchwork.kernel.org/patch/9709815/
https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg02219.html

2. You have to pause the source VM before transferring the last 
incremental zfs snapshot and before starting the dest VM. After this 
last zfs transfer, you can start the qemu migration stream. This 
introduces a bit of delay between pausing the source VM and resuming the 
dest VM. Depends on how you implement it, you can have longer or shorter 
delay.


2a: Longer delay (but easier to implement) is doing everything 
sequentially: zfs full send/recv, pause src VM, zfs differential 
send/recv, start dest VM, start migration (the dest VM resumes 
automatically after successful state transfer). The downtime is more or 
less equal to the time needed to tranfer VM RAM contents.


2b: Shorter delay (this is the best way I can think of at the moment. If 
you find better way, please let me know): zfs full send/recv, start qemu 
live migration that transfers the VM state into the temporary file in 
/tmp (=ramdisk) at destination's host and in parallel with that, run 
differential zfs snapshot transfer every 10 seconds (src VM is still 
running at this time). As soon as the qemu migration finishes (src VM is 
paused now), you transfer the last few disk bits using incremental 
snapshot and run the dest VM + immediately push the content of the 
temporary migration file from /tmp (after this is done, the VM resumes 
automatically). State restore from /tmp into the qemu takes very short 
time so the downtime is minimized.


How do you solve the problem with unequal kvm configs, for example 
cpu_type HOST (on the source host) and QEMU(on the target host)?


You (or the migration script) are responsible for creating a destination 
VM with a compatible config. If your new config differs too much (e.g. 
different device count, RAM size, etc.), I expect qemu to stop/fail the 
incoming migration. But you don't want to create incompatible configs 
anyway.


How do you solve the problem with starting the second kvm zone on the 
other host (this reserves the same MAC / IP addresses on the network) 
at the same time?


Yes, it does start the zone with the same zone network config. But the 
qemu inside the destination zone is not running the VM yet. AFAIK the 
zone network config is mostly about restrictions on possible IP/MAC 
addresses. My tests show that as soon as I unpause the destination VM, 
it is reachable immediately.


I hope it helped to get an overview.
Cheers

Jan


Mit freundlichen Grüßen

*DANIEL PLOMINSKI*

Leiter – IT / Head of IT

Telefon 09265 808-151  |  Mobil 0151 58026316  | d...@ass.de 
<mailto:d...@ass.de>


PGP Key: http://pgp.ass.de/2B4EB20A.key

cid:C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain

ASS-Einrichtungssysteme GmbH

ASS-Adam-Stegner-Straße 19  |  D-96342 Stockheim

Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß

Amtsgericht Coburg HRB 3395  |  Ust-ID: DE218715721

cid:E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain

*Von:*Ján Poctavek [mailto:jan.pocta...@erigones.com]
*Gesendet:* Mittwoch, 8. November 2017 10:55
*An:* smartos-discuss@lists.smartos.org
*Betreff:* Re: [smartos-discuss] Live migration

Hi,

Sure, the sources are here:

https://github.com/YanChii/illumos-kvm/tree/live-migration
https://github.com/YanChii/illumos-kvm-cmd/tree/live_migration

The (small) number of changes does not reflect the 

Re: [smartos-discuss] Live migration

[Ján Poctavek]

Hi,

Sure, the sources are here:

https://github.com/YanChii/illumos-kvm/tree/live-migration
https://github.com/YanChii/illumos-kvm-cmd/tree/live_migration

The (small) number of changes does not reflect the effort invested into 
this. All dead ends and debugs are not present in the final patch :).


I just rebased and testing my changes to integrate latest commit into 
illumos-kvm (regarding the coexistence with bhyve):

https://github.com/YanChii/illumos-kvm/tree/lm-merge

Jan


On 7. 11. 2017 21:03, Daniel Plominski wrote:

Hi Jan,

are the danube kvm patches public available on github?
Von meinem iPhone gesendet

Am 07.11.2017 um 11:32 schrieb Ján Poctavek <jan.pocta...@erigones.com 
<mailto:jan.pocta...@erigones.com>>:



Hi Matt,

KVM live migration is possible and we are working on it right now. 
The hardest part (the kernel & qemu support) is already done and 
currently I'm able to successfully migrate the VMs. Now we need two 
things:
1. integrate the patches into the illumos-kvm and illumos-kvm-cmd 
repos (requires testing, reviews, etc.)

2. some userspace support (live migration script).

After merging the patches into SmartOS, we can focus on the userspace 
part (or you can write your own migration script).


Jan

PS: in parallel with the above, the live migration will be present in 
the Danube Cloud v3.0.0 expected in this December.


On 6. 11. 2017 17:05, Matthew Law wrote:
De-lurking for a moment to ask a question: is live migration of 
zones and KVM VMs possible? I recall it has been mentioned on this 
list before and I can find ageing references to it, e.g:


https://github.com/joyent/smartos-live/blob/master/src/vm/README.migration

I usually get by with vmadm send and vmadm receive after first 
stopping the VM and that works great but it’s a bit of a chore 
stopping and migrating dozens of VMs off a busy host when it needs 
to go down for maintenance.



Cheers,

Matt.


*smartos-discuss* | Archives 
<https://www.listbox.com/member/archive/184463/=now> 
<https://www.listbox.com/member/archive/rss/184463/28549989-eb719454> 
| Modify 
<https://www.listbox.com/member/?;> 
Your Subscription 	[Powered by Listbox] <http://www.listbox.com>







---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Live migration

[Ján Poctavek]

Hi Matt,

KVM live migration is possible and we are working on it right now. The 
hardest part (the kernel & qemu support) is already done and currently 
I'm able to successfully migrate the VMs. Now we need two things:
1. integrate the patches into the illumos-kvm and illumos-kvm-cmd repos 
(requires testing, reviews, etc.)

2. some userspace support (live migration script).

After merging the patches into SmartOS, we can focus on the userspace 
part (or you can write your own migration script).


Jan

PS: in parallel with the above, the live migration will be present in 
the Danube Cloud v3.0.0 expected in this December.



On 6. 11. 2017 17:05, Matthew Law wrote:
De-lurking for a moment to ask a question: is live migration of zones 
and KVM VMs possible? I recall it has been mentioned on this list 
before and I can find ageing references to it, e.g:


https://github.com/joyent/smartos-live/blob/master/src/vm/README.migration

I usually get by with vmadm send and vmadm receive after first 
stopping the VM and that works great but it’s a bit of a chore 
stopping and migrating dozens of VMs off a busy host when it needs to 
go down for maintenance.



Cheers,

Matt.
*smartos-discuss* | Archives 
 
 
| Modify 
 
Your Subscription 	[Powered by Listbox] 







---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] adding xserver to smartos?

[Ján Poctavek]

Hi,


On 1. 11. 2017 15:16, Lonnie Cumberland wrote:

Hi Jan,

I started working on this a bit more today after spending a bit of 
time reviewing the SmartOS docs to get a better feel for things but 
have a couple of questions (in-line)


X server environment can be installed using pkgsrc. But in
SmartOS, you are very limited. You can make it running but
expecting even a working web browser is a bit daring. Here is what
you need to do:


Mostly, I just want to startup a VNC or Spice client and try to 
connect to a running zone with maybe a simple linux VM running, perhaps.
For VNC, there's a good VNC/RDP client called vinagre in pkgsrc. For 
connecting to zones you just need some sort of xterm.


- bootstrap pkgsrc into the global zone

For this part, do you mean to install pkgin as I did this by following:

https://wiki.smartos.org/pages/viewpage.action?pageId=756297

That's and old link. Use this: https://pkgsrc.joyent.com/install-on-illumos/
And this download link: 
http://pkgsrc.joyent.com/packages/SmartOS/bootstrap/bootstrap-{{ 
pkgin_release }}-x86_64.tar.gz

With pkgin_release="2017Q3"



and:

Install pkgin

From http://wiki.smartos.org/display/DOC/Installing+pkgin

This is the SmartOS package manager you will find the latest version 
at http://pkgsrc.joyent.com/packages/SmartOS/bootstrap/ in this case 
its bootstrap-2013Q2-x86_64.tar.gz


cd /
curl -k 
http://pkgsrc.joyent.com/packages/SmartOS/bootstrap/bootstrap-2013Q2-x86_64.tar.gz 
| gzcat | tar -xf -

pkg_admin rebuild
pkgin -y up
--

For the next part, I want to install everything in a zone, if 
possible, and am looking at:


From https://gist.github.com/3050224

From 
http://wiki.smartos.org/display/DOC/How+to+create+a+zone+%28+OS+virtualized+machine+%29+in+SmartOS



imgadm update
imgadm avail | grep base64
# copy the UUID "9eac5c0c-a941-11e2-a7dc-57a6b041988f" from the latest 
version (look at the date)
# Note there exist a node image too, but its always outdated and it 
contains software that you most likely don't need


imgadm import 9eac5c0c-a941-11e2-a7dc-57a6b041988f
# you should also see a ASCII progress bar. If instead you see:
#   60a3b1fa-0674-11e2-abf5-cb82934a8e24 doesnt exist. continuing with 
install
# you are mostlikely running an older SmartOS version, don't worry it 
is downloading it just dosn't show any progress


# /zones/ is the permanent directory if you store something elsewhere 
it will be deleted when SmartOS power off.

mkdir /zones/defs
touch /zones/defs/base.json

# dataset_uuid: the copied UUD
# nics[0].ip: In smartos I typed `ifconfig -a` and found `e1000g0` 
(172.16.136.129).
#   so for the SmartMashine I use `172.16.136.100` (I just picked 100 
randomly)
# nics[0].gateway, resolvers[0]: I typed `netstat -r` and find the 
default gateway value (172.16.136.2).

# ram: My VM of smartos is given 1024 so I give my SmartMachine 512

echo '
{
  "brand": "joyent",
  "ram": 512,
  "autoboot": false,
  "dataset_uuid": "9eac5c0c-a941-11e2-a7dc-57a6b041988f",
  "resolvers": [
    "172.16.136.2",
    "8.8.8.8"
  ],
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "172.16.136.100",
      "netmask": "255.255.255.0",
      "gateway": "172.16.136.2"
     }
  ]
}' > /zones/defs/base.json

vmadm create -f /zones/defs/base.json

Of course, I will be chaning the IP, gateway, etc.., then start the VM 
and zlogin:


vmadm list
# output:
# UUID                                  TYPE  RAM   STATE            
 ALIAS

# 463647dc-57f8-45ac-bdc2-18b0cc4c0be6  OS    512   running           -
# copy the UUID

vmadm start 463647dc-57f8-45ac-bdc2-18b0cc4c0be6
zlogin 463647dc-57f8-45ac-bdc2-18b0cc4c0be6

- basic X org:
pkgin install modular-xorg-server xf86-video-vesa
xf86-input-keyboard xf86-input-mouse xhost xauth dejavu-ttf xinit


I think that this will need to be done after creating the unprivileged 
user in the next step below, right.

-

- create an unprivileged user (make sure the user and homedir
persist after reboot)


This part is a bit confusing and all that I could find on this process 
is, of course will change the user name:


https://docs.joyent.com/sdc6/managing-customers

User Accounts on SmartMachines

The SmartMachine owner can create user accounts by logging in to the 
SmartMachine as root and using the useradd tool. This command creates 
a user account for the user jill:


[root@smartos ~]# useradd -d /home/jill -m -c "Jill Joyent" jill
By default, new user accounts are locked. Mark the user account as "no 
login" to unlock it. This means that the user cannot log in to the 
user account with a password. She will be able to log in to the user 
account using SmartLogin. Use the passwd command like this to mark the 
account "no login":


[root@smartos ~]# passwd -N jill
passwd: password information changed for jill
You will need to store the users public key in her 

Re: [smartos-discuss] adding xserver to smartos?

[Ján Poctavek]

Hi Lonnie,

X server environment can be installed using pkgsrc. But in SmartOS, you 
are very limited. You can make it running but expecting even a working 
web browser is a bit daring. Here is what you need to do:


- bootstrap pkgsrc into the global zone

- basic X org:
pkgin install modular-xorg-server xf86-video-vesa xf86-input-keyboard 
xf86-input-mouse xhost xauth dejavu-ttf xinit


- create an unprivileged user (make sure the user and homedir persist 
after reboot)


- choose a window manager (for example, I've seen icewm and fluxbox working)
pkgin install fluxbox
echo fluxbox > ~/.xinitrc # run this as unprivileged user
- or
pkgin install icewm
echo icewm-session > ~/.xinitrc # run this as unprivileged user

startx # run this as unprivileged user

And don't use 2016Q4 repo. AFAIK the X mouse integration is broken 
there. Use newer (2017Q2 worked for me).


Jan

On 25. 10. 2017 0:19, Lonnie Cumberland wrote:

Greetings All,

Hope that everyone is doing well.

While still working on my smartos-based cluster, which is coming along 
more and more, I decided to explore and investigate various Illumos 
based dialects.


In particular, I was looking for the smallest footprint and stable 
release that I could find. In my search, I came across a number of 
Illumos based distributions (OpenIndiana, OmniOS, Nexenta, etc.) but 
all of them seem to have, or not have as the case may be, some core 
features that I was seeking which may have brought me back to SmartOS 
for this general exploration.


Basically, it seems that SmartOS has the smallest footprint, from what 
I can tell, and seems to be better, at least for what is does as a 
hypervisor which is what I am interested in as well.


it also seems that all, or most of the distrobutions, like OpenIndiana 
can use the SmartOS package system and repositories and I think that I 
read that all of them can also use it as they are all Illumos-based.


Long story short, I think that I would like to take a copy of SmartOS 
and see about adding a very small XServer and just one X-application 
to the global zone.


With this in mind, I was wondering if anyone has done something 
similar, or might be able to offer some guidance as I am really coming 
from the Windows/Linux world and my Solaris background is very limited.


Thanks in advance,
Lonnie
*smartos-discuss* | Archives 
 
 
| Modify 
 
Your Subscription 	[Powered by Listbox] 







---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Hopefully simple question on IPF and svcadm

[Ján Poctavek]

Hi,

2. use the "svcadm clear ipfilter" command to clear the service 
maintenance state


Jan


On 12. 10. 2017 2:04, Marc Phillips wrote:

I have, what I suspect are, remedial smartOS questions which I haven't been 
able to google an answer too.  Just getting back into smartOS and I've never 
used IPF before.

1. I have IPF set to log and put local0.* /var/log/ipf.log in both syslog.conf 
and rsyslog.conf (rsyslog seems to be what's running; so I assume it reads 
/etc/rsyslog.conf).
That log file doesn't get touched after kill HUPing the service or rebooting the box.  I have 
"block in quick log first on  all" in my ipf configs.
Not sure what I'm missing based on reading the IPF docos but not getting logs.  
IPF itself is working fine otherwise on both IPv4 and IPv6.

2. When ipfilter errors out (for syntax errors in config), I have to reboot in 
order to make it reread the config.  refresh, disable/enable, restart seem to 
have no effect if it's failed a load.  I see the refresh notifications in the 
log, but it doesn't seem to try to reload/restart unless and until I reboot.  
Not sure if there's a trick to make it do that via svcadm that I'm missing, but 
I'd like to know if there is.  Reboot seems rather brute force.

I'm running SmartOS  base64 14.3.0 in Joyent's public cloud.

R.  Marc




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: AW: [smartos-discuss] smartos in dedicated hosting

[Ján Poctavek]

I see. You are using the admin nictag as external interface and then you add 
additional vnic with provider-assigned failover subnet. 

On 12 September 2017 14:01:23 CEST, Daniel Plominski <d...@ass.de> wrote:
>Hi Jan,
>
>but exactly this Setup works at Hetzner on our Rrootservers.
>
>Hetzner  sends the complete traffic to the MAIN IP:
>admin_ip=138.XXX.XX.XXA
>The vnic0 gets the first IP from the SUBNET: 88.XXX.XXX.XXA
>
>For IPv6 use vnic1 and plumb 2a01:::::EEEA/64
>Hetzner specifically you push the entire traffic over fe80::1
>
>[root@root1 /zones/ass.de/template]# cat vm01-root1-fw1-opnsense.json
>
>{
>  "brand": "kvm",
>  "alias": "root1-fw1-opnsense",
>  "resolvers": [
>"8.8.8.8",
>"8.8.4.4"
>  ],
>  "ram": "4096",
>  "vcpus": "2",
>  "nics": [
>{
>  "__comment" : "hetzner: 88.XXX.XXX.XXB",
>  "nic_tag": "admin",
>  "allowed_ips": [
>"2a01:::::B:"
>  ],
>  "ip": "88.XXX.XXX.XXB",
>  "ips": ["88.XXX.XXX.XXB/29", "addrconf"],
>  "netmask": "255.255.255.248",
>  "gateway": "88.XXX.XXX.XXA",
>  "model": "virtio",
>  "primary": true
>},
>{
>  "__comment" : "internal: 10.XXX.XXX.XXD",
>  "nic_tag": "vswitch0",
>  "ip": "10.XXX.XXX.XXD",
>  "ips": ["10.XXX.XXX.XXD/22", "addrconf"],
>  "netmask": "255.255.252.0",
>  "gateway": "10.XXX.XXX.XXE",
>  "model": "virtio"
>}
>  ],
>  "disks": [
>{
>  "boot": true,
>  "model": "virtio",
>  "compression": "lz4",
>  "size": 16384,
>  "block_size": 8192
>}
>  ]
>}
>
>[root@root1 /zones/ass.de/template]#
>For security reasons create firewall rules at Hetzner Robot for the
>Root Servers. (DROP all traffic to the MAIN IP, but allow all other for
>the Subnet IPs)
>vmadm update UUID vnc_port=ZZZA (this only activates the vnc port on
>the MAIN IP) / to disable the vnc access -> use: vmadm update UUID
>vnc_port=-1
>And with ssh -p  -i /home/fuu/.ssh/id_bar -L :
>138.XXX.XX.XXA:ZZZA r...@138.xxx.xx.xxa<mailto:r...@138.xxx.xx.xxa> you
>can tunnel the plain vnc access locally
>Works like a charm.
>
>From linux I recognize that you can rewrite mac addresses on the bridge
>(proxyarp) but I did not try this under smartos.
>
>I have used a lot of network stuff in my LXC-to-GO Project:
>https://github.com/plitc/lxc-to-go/blob/master/content/README.DIAGRAM.md
>
>Or crazy stuff on FreeBSD with up to 256 Bridges:
>https://blog.plitc.eu/2014/freebsd-10-komplexe-bridge-zones-mit-lacp-uplink/
>
>But my impression is, the more one uses complicated techniques, the
>more cumbersome it becomes to the conclusion to debug (like proxyarp,
>multiple source & destination nat between vms on the same host)
>
>
>
>Mit freundlichen Grüßen
>
>
>DANIEL PLOMINSKI
>Leiter – IT / Head of IT
>
>Telefon 09265 808-151  |  Mobil 0151 58026316  | 
>d...@ass.de<mailto:d...@ass.de>
>PGP Key: http://pgp.ass.de/2B4EB20A.key
>
>
>[cid:C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain]
>
>ASS-Einrichtungssysteme GmbH
>ASS-Adam-Stegner-Straße 19  |  D-96342 Stockheim
>
>Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß
>Amtsgericht Coburg HRB 3395  |  Ust-ID: DE218715721
>
>[cid:E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain]
>
>Von: Ján Poctavek [mailto:jan.pocta...@erigones.com]
>Gesendet: Dienstag, 12. September 2017 13:08
>An: smartos-discuss@lists.smartos.org
>Betreff: Re: AW: [smartos-discuss] smartos in dedicated hosting
>
>
>Thank you Daniel for sharing your setup. I use your scenario in some
>installations, also with etherstubs and GZ routing.
>But:
>
>1. this is exactly I'd like to avoid - need for creating an own custom
>script for networking
>
>2. you are creating a vnic0 interface over e1000g0. It will not work
>with e.g. Hetzner or OVH because you are changing the external MAC.
>
>Jan
>On 12. 9. 2017 11:17, Daniel Plominski wrote:
>
>Hi Poctavek,
>
>
>
>Example: DATACENTER <=> DC Switch <=> Rootserver (SmartOS + VMs)
>
>
>
>SmartOS has 1 ADMIN interfac e wi

Re: AW: [smartos-discuss] smartos in dedicated hosting

[Ján Poctavek]

Thank you Daniel for sharing your setup. I use your scenario in some 
installations, also with etherstubs and GZ routing.

But:

1. this is exactly I'd like to avoid - need for creating an own custom 
script for networking


2. you are creating a vnic0 interface over e1000g0. It will not work 
with e.g. Hetzner or OVH because you are changing the external MAC.


Jan


On 12. 9. 2017 11:17, Daniel Plominski wrote:


Hi Poctavek,

Example: DATACENTER óDC Switch óRootserver (SmartOS + VMs)

SmartOS has 1 ADMIN interfac e with an additional /29 Subnet

[root@root1 /usbkey]# cat config
#
# This file was auto-generated and must be source-able by bash.
#
### ### ### ASS // ### ### ###

admin_nic=AA:BB:CC:DD:EE:00
admin_ip=dhcp
headnode_default_gateway=138.XXX.XX.XXF

dns_resolvers=8.8.8.8,8.8.4.4
dns_domain=ass.de

ntp_hosts=0.smartos.pool.ntp.org
compute_node_ntp_hosts=dhcp

... … …

### ### ### // ASS ### ### ###
# EOF
[root@root1 /usbkey]#

[root@root1 /opt/custom/smf]# cat subnet-routing-setup.xml
'/usr/share/lib/xml/dtd/service_bundle.dtd.1'>


  
    
    
    restart_on='error' type='service'>

  
    
    restart_on='error' type='service'>

  
    
    exec='/opt/custom/scripts/subnet-routing-setup' timeout_seconds='60'>

  
    
    
  
    
  
    
    timeout_seconds='60'>

  
    
  
    
    timeout_seconds='60'>

  
    
  
    
    
  
  
    
    
    
    
  
    subnet-routing-setup
  
    
  

[root@root1 /opt/custom/smf]#

[root@root1 /opt/custom/scripts]# cat subnet-routing-setup
#!/bin/sh

. /lib/svc/share/smf_include.sh

#// disable services
svcadm disable svc:/network/rpc/bind:default

#// HOST: ipv6
#/dladm create-vnic -l e1000g0 vnic1
ifconfig e1000g0 inet6 plumb
ifconfig e1000g0 inet6 addif 2a01:::::EEEA/64 up
route add -inet6 fe80::1 2a01:::::EEEA -interface
route add -inet6 default fe80::1
svcadm enable ipv6-forwarding
routeadm -e ipv6-forwarding
routeadm -e ipv6-routing
routeadm -u

#// VM: ipv4
dladm create-vnic -l e1000g0 vnic0
ifconfig vnic0 plumb 88.XXX.XXX.XXA netmask 255.255.255.248 up
svcadm enable route
routeadm -e ipv4-forwarding
routeadm -e ipv4-routing
routeadm -u

#// VM: internal vswitch (intern / ass vpn)
# create a etherstub
dladm create-etherstub vswitch0
dladm set-linkprop -p mtu=1500 vswitch0

#// VM: internal vswitch (intern / coorp vpn)
# create a etherstub
dladm create-etherstub vswitch1
dladm set-linkprop -p mtu=1500 vswitch1

exit $SMF_EXIT_OK

[root@root1 /opt/custom/scripts]#

Now use the SmartOS IP: 88.XXX.XXX.XXAas default gateway for ALL your
Zone / KVM Machines

Another method would be: NAT

Mit freundlichen Grüßen

*DANIEL PLOMINSKI*

Leiter – IT / Head of IT

Telefon 09265 808-151  |  Mobil 0151 58026316  | d...@ass.de
<mailto:d...@ass.de>

PGP Key: http://pgp.ass.de/2B4EB20A.key

cid: C17DB6FB-5F79-4BCC-AAB4-CAB59266BC29@localdomain

ASS-Einrichtungssysteme GmbH

ASS-Adam-Stegner-Straße 19  |  D-96342 Stockheim

Geschäftsführer: Matthias Stegner, Michael Stegner, Stefan Weiß

Amtsgericht Coburg HRB 3395  |  Ust-ID: DE218715721

cid: E40AEC87-91EE-472A-901A-ECAD3F5801FB@localdomain

-Ursprüngliche Nachricht-
Von: Ján Poctavek [mailto:jan.pocta...@erigones.com]
Gesendet: Dienstag, 12. September 2017 10:45
An: smartos-discuss@lists.smartos.org
Betreff: [smartos-discuss] smartos in dedicated hosting

Hi guys,

I have a bit of complications installing SmartOS in the dedicated hosting.

Many hosting providers have an additional security (network filter)
that allows a dedicated server to communicate to the internet only by
using the assigned IP address *together* with the default MAC address.
But when I configure the external interface with IP address in the
config file, the IP address is created over external0 vnic. And this
new vnic has a new MAC address that is different from default HW NIC
address. As a result, all communication gets dropped.

Is there a way to solve this using a config file?

The workarounds I can come with:

1.  add a new SMF service that manually adds the IP address over the
physical NIC

2.  modify the network/physical script

3.  add _preserve_mac config property to add IP address
directly to physical NIC

The thing is that the first two options do not scale and I don't want
to implement the third if it already exists.

Thanks for hints.

Jan

*smartos-discuss* | Archives
<https://www.listbox.com/member/archive/184463/=now>
<https://www.listbox.com/member/archive/rss/184463/28549989-eb719454>
| Modify 
<https://www.listbox.com/member/?;> 
Your Subscription [Powered by Listbox] <http://www.listbox.com>






---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?mem

[smartos-discuss] smartos in dedicated hosting

[Ján Poctavek]

Hi guys,

I have a bit of complications installing SmartOS in the dedicated hosting.

Many hosting providers have an additional security (network filter) that 
allows a dedicated server to communicate to the internet only by using 
the assigned IP address *together* with the default MAC address. But 
when I configure the external interface with IP address in the config 
file, the IP address is created over external0 vnic. And this new vnic 
has a new MAC address that is different from default HW NIC address. As 
a result, all communication gets dropped.


Is there a way to solve this using a config file?

The workarounds I can come with:

1. add a new SMF service that manually adds the IP address over the 
physical NIC


2. modify the network/physical script

3. add _preserve_mac config property to add IP address directly 
to physical NIC


The thing is that the first two options do not scale and I don't want to 
implement the third if it already exists.


Thanks for hints.

Jan



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] KVM live migration debug

[Ján Poctavek]

On 29. 7. 2017 1:28, Robert Mustacchi wrote:

On 7/21/17 8:33 , Ján Poctavek wrote:

Hi,

as advised, I've changed the guest OS to SmartOS. I was able to migrate
the OS to the second qemu. Before the migration, I've started debugger
using "mdb -K". Now I have two VMs:

1. The source VM running mdb.
2. The destination VM also running mdb without a problem.

When I resume the OS by exiting mdb in the source VM, everything is
running fine.
When I do the same in the second VM, the processes start to crash with
SEGV.

The contents of the two VMs are expected to be absolutely identical..
but apparently they are not. Anyway, with running mdb on both VMs, it
seems as an ideal debug setup to me - just compare and find the
difference. I believe it should be easy to look on the memory.

Unfortunately, even after 2 days spent reading various mdb manuals and
handbooks, I don't know how to actually do it.

Can you please help me with that?

When this occurs, do you get a kernel crash dump?

Hi, Robert,

Sorry for my late reply.
Unlike Linux, SmartOS does not crash after migration. But it goes into 
an infinite loop of starting login prompt that immediately crashes again 
with SEGV. I'm unable to enter any commands in this state. I have only a 
dump right after the migration before resuming the migrated VM. Is it 
possible to set a breakpoint that dumps the kernel at first occurrence 
of SEGV? It would get us exactly to the point of failure.


Thank you.
Jan



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Contributing to smartos-live

[Ján Poctavek]

Hi Lonnie,

thank you for pointing out that we don't have this comparison, we'll try 
to fix that. But this is not about competition anyway. Our focus is a 
bit different.


To answer your question: first of all, Danube Cloud (DC) is trying to be 
user friendly and easy to use in the first place. You don't need the 
dedicated headnode (the management are just few VMs that can be migrated 
anywhere). On top of that, DC includes full Zabbix install for automated 
monitoring of all nodes and hosted VMs, full featured VM backups using 
incremental ZFS snapshots, DNS server, multi-tenant IP address and 
networks management, etc.


All this stuff is in community edition. Enterprise version is about SLA 
support, not about features (all our new work goes into the community 
edition).


Cheers.
Jan

On 9. 8. 2017 13:11, Lonnie Cumberland wrote:

Hi Daniel,

I just started looking at the Danube release and it looks like you
have brought things together nicely, but was trying to locate what
major changes, improvements, or modifications the Danube release has
over Triton and could not find a list anywhere to look over.

I am sure that you have made some changes as your post describes to
some degree, but was looking to try and get a feel for why I might be
interested in installing your Danube release over Triton. One thing
that I did immediately see on your site is that you have a Community
release which seems to have less features than your Enterprise release
while also offering the Danube Cloud while Joyent seems to have their
single Triton release while focusing on the Triton Cloud services.

This was just meant to be quick observation and I have not looked
heavily into the Danube software while still very new to the Triton
Data Center as well so I'm not really one to do any type of
evaluations or comparisons at this point, but I am all for making
things easier so that one can setup a solidly functioning private
cloud so as to be able to focus on containers and their uses while
minimizing the concerns about the infrastructure.

Personally, I am setting up a private cloud to learn from and test,
with a major focus on running Docker containers, micro-services and
possibly Kubernetes at some future point.

Cheers and have a good day,
Lonnie

On Wed, Aug 9, 2017 at 3:33 AM, Daniel Kontsek > wrote:

Dear Joyent and SmartOS Community,

we have been successfully using and integrating SmartOS at Danube
Cloud (Erigones) for about 5 years now. At the beginning we've
built an IaaS platform, which we've transformed into a
full-featured software solution - Danube Cloud (former Erigones
SDDC) [1]. During this time, there were occasional moments where
we thought that a Linux hypervisor would maybe be a better
choice... But the strengths of SmartOS / illumos gave ourselves
repeatedly arguments that - YES - it was the right choice (Zones,
ZFS, Crossbow, DTrace... you know the perks...). We would like to
say to all illumos and SmartOS contributors: THANK YOU for the
amazing work.

Although, we would like to use the SmartOS platform as it is, we
have to maintain some changes to the illumos-joyent and
smartos-live repositories. This is mostly because of support for
installation/boot from hard drive (contributed by Juraj Lutter),
installer (prompt-config) script location and other smaller
changes to the installer. As far as my understanding goes, Joyent
would not want to support installation to hard drive but maybe
some other (smaller) features would be beneficial for Joyent and
the SmartOS community. We can create pull requests for that, but
there are a few topics I would like to discuss first:

-  AFAIK we should open merge requests here: https://cr.joyent.us/
and not on GitHub, but we should create an issue on GitHub first,
is this correct, or can we just create CRs in Gerrit?

-  Shell scripts coding style (mainly svc/methods and
prompt_config) is a problem. We are seeing mixing of bash coding
patterns, even in scripts where new bash features are used. (e.g.
$var vs ${var} vs "${var}", `` vs $(), [ vs [[). I assume lots of
these are just Solaris heritage, but some scripts are new and yet
we see these strange inconsistencies. I'm not going to argue about
line length and tabs vs spaces (although please don't mix them).
But as we are saying: at least do it consistently wrong :) We are
happily using shellcheck [2] for most of our bash scripts and it
does solve these kind of problems. Is there a coding style guide
for shell scripts?

-  For example: we would love to rewrite the
smartos_prompt_config.sh script so it does not use global
variables. Would you accept such change?

Daniel

[1]  https://github.com/erigones/esdc-ce/wiki

 https://github.com/koalaman/shellcheck


*smartos-discuss* | Archives

Re: [smartos-discuss] KVM live migration debug

[Ján Poctavek]

Hi,

as advised, I've changed the guest OS to SmartOS. I was able to migrate 
the OS to the second qemu. Before the migration, I've started debugger 
using "mdb -K". Now I have two VMs:


1. The source VM running mdb.
2. The destination VM also running mdb without a problem.

When I resume the OS by exiting mdb in the source VM, everything is 
running fine.

When I do the same in the second VM, the processes start to crash with SEGV.

The contents of the two VMs are expected to be absolutely identical.. 
but apparently they are not. Anyway, with running mdb on both VMs, it 
seems as an ideal debug setup to me - just compare and find the 
difference. I believe it should be easy to look on the memory.


Unfortunately, even after 2 days spent reading various mdb manuals and 
handbooks, I don't know how to actually do it.


Can you please help me with that?

Thank you.

Jan

On 13. 7. 2017 2:40, Robert Mustacchi wrote:

Hi Ján,

Unfortunately, I'm not very familiar with the internals of Linux. What I
might suggest trying to do is to maybe test with migrating SmartOS, only
because it'll hopefully be easier for us to look at dumps, get a crash
dump, and debug.

I suspect that likely what's going on here is that some part of the
migrated state has not been correctly saved and/or restored, especially
given that we never really focused on bring up at the time.

Sorry I don't have a more actionable next step for you.

Robert




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] KVM live migration debug

[Ján Poctavek]

Hi Robert ,

thank you for the hints. I'll try with the SmartOS guest and come back 
with results.


Jan


On 13. 7. 2017 2:40, Robert Mustacchi wrote:

On 7/1/17 13:35 , Ján Poctavek wrote:

Hi,

I'm trying to get a KVM/qemu live migration working on SmartOS. My
starting point was the same problem as in this post:
https://www.listbox.com/member/archive/184463/2012/04/sort/time_rev/page/2/entry/24:101/20120417112635:B4169A4C-88A1-11E1-9C88-F96B3BAD9C1B/


I have dtraced the EIVALs and I have identified two problems -
unimplemented ioctls: KVM_GET_IRQCHIP and KVM_GET_CLOCK.

The first one can be (at least temporarily) solved by adding
"-no-kvm-irqchip" to qemu flags.

With the second one, I have implemented ioctl calls for KVM_GET_CLOCK
and KVM_SET_CLOCK in the KVM kernel module.

After this, I am able to do migration without qemu complaining. More
importantly, I am able to successfully migrate the VM in GRUB prompt
(using "migrate" qemu command).

But when migrating linux (booted into the initrd target for simplicity),
it panicks after pressing "enter" in the console:

[   28.337953] double fault:  [#1] SMP
[   28.337953] Modules linked in: ext4 mbcache jbd2 sd_mod crc_t10dif
sr_mod cdrom crct10dif_generic crct10dif_common ata_generic pata_acpi
ata_piix serio_raw libata floppy
[   28.337953] CPU: 0 PID: 195 Comm: sh Not tainted
3.10.0-514.16.1.el7.x86_64 #1
[   28.337953] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[   28.337953] task: 88001f46 ti: 88001f74c000 task.ti:
88001f74c000
[   28.337953] RIP: 0010:[] []
do_page_fault+0xb/0x90
[   28.337953] RSP: 0008:7ffc606b9000  EFLAGS: 00010097
[   28.337953] RAX: 8168e8ec RBX: 0001 RCX:
8168e8ec
[   28.337953] RDX: cdc0 RSI:  RDI:
7ffc606b9018
[   28.337953] RBP: 7ffc606b9008 R08: 000a R09:
7f7a161bf740
[   28.337953] R10: 0008 R11: 0246 R12:

[   28.337953] R13:  R14: 0002 R15:
7ffc606ba860
[   28.337953] FS:  7f7a161bf740() GS:88001fc0()
knlGS:
[   28.337953] CS:  0010 DS:  ES:  CR0: 8005003b
[   28.337953] CR2: 7ffc606b8ff8 CR3: 1f71a000 CR4:
06f0
[   28.337953] DR0:  DR1:  DR2:

[   28.337953] DR3:  DR6: 0ff0 DR7:
0400
[   28.337953] Stack:
[   28.337953]   7ffc606b90f8 8168eb88
7ffc606ba860
[   28.337953]  0002  
7ffc606b90f8
[   28.337953]  7ffc606b9108 0246 0008
7f7a161bf740
[   28.337953] Call Trace:
[   28.337953] Code: 89 de 4c 89 ef e8 7c ca fe ff e9 5c fd ff ff 31 c0
e9 01 ff ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5
41 55 <41> 54 49 89 f4 53 48 89 fb 48 83 ec 08 0f 20 d0 0f 1f 40 00 0f
[   28.337953] RIP  [] do_page_fault+0xb/0x90
[   28.337953]  RSP <7ffc606b9000>
[   28.337953] ---[ end trace b556ad308185dda4 ]---
[   28.337953] Kernel panic - not syncing: Fatal exception

Can somebody give me a hint how can I debug this?

Hi Ján,

Unfortunately, I'm not very familiar with the internals of Linux. What I
might suggest trying to do is to maybe test with migrating SmartOS, only
because it'll hopefully be easier for us to look at dumps, get a crash
dump, and debug.

I suspect that likely what's going on here is that some part of the
migrated state has not been correctly saved and/or restored, especially
given that we never really focused on bring up at the time.

Sorry I don't have a more actionable next step for you.

Robert




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

[smartos-discuss] KVM live migration debug

[Ján Poctavek]

Hi,

I'm trying to get a KVM/qemu live migration working on SmartOS. My 
starting point was the same problem as in this post:

https://www.listbox.com/member/archive/184463/2012/04/sort/time_rev/page/2/entry/24:101/20120417112635:B4169A4C-88A1-11E1-9C88-F96B3BAD9C1B/

I have dtraced the EIVALs and I have identified two problems - 
unimplemented ioctls: KVM_GET_IRQCHIP and KVM_GET_CLOCK.


The first one can be (at least temporarily) solved by adding 
"-no-kvm-irqchip" to qemu flags.


With the second one, I have implemented ioctl calls for KVM_GET_CLOCK 
and KVM_SET_CLOCK in the KVM kernel module.


After this, I am able to do migration without qemu complaining. More 
importantly, I am able to successfully migrate the VM in GRUB prompt 
(using "migrate" qemu command).


But when migrating linux (booted into the initrd target for simplicity), 
it panicks after pressing "enter" in the console:


[   28.337953] double fault:  [#1] SMP
[   28.337953] Modules linked in: ext4 mbcache jbd2 sd_mod crc_t10dif 
sr_mod cdrom crct10dif_generic crct10dif_common ata_generic pata_acpi 
ata_piix serio_raw libata floppy
[   28.337953] CPU: 0 PID: 195 Comm: sh Not tainted 
3.10.0-514.16.1.el7.x86_64 #1

[   28.337953] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[   28.337953] task: 88001f46 ti: 88001f74c000 task.ti: 
88001f74c000
[   28.337953] RIP: 0010:[] [] 
do_page_fault+0xb/0x90

[   28.337953] RSP: 0008:7ffc606b9000  EFLAGS: 00010097
[   28.337953] RAX: 8168e8ec RBX: 0001 RCX: 
8168e8ec
[   28.337953] RDX: cdc0 RSI:  RDI: 
7ffc606b9018
[   28.337953] RBP: 7ffc606b9008 R08: 000a R09: 
7f7a161bf740
[   28.337953] R10: 0008 R11: 0246 R12: 

[   28.337953] R13:  R14: 0002 R15: 
7ffc606ba860
[   28.337953] FS:  7f7a161bf740() GS:88001fc0() 
knlGS:

[   28.337953] CS:  0010 DS:  ES:  CR0: 8005003b
[   28.337953] CR2: 7ffc606b8ff8 CR3: 1f71a000 CR4: 
06f0
[   28.337953] DR0:  DR1:  DR2: 

[   28.337953] DR3:  DR6: 0ff0 DR7: 
0400

[   28.337953] Stack:
[   28.337953]   7ffc606b90f8 8168eb88 
7ffc606ba860
[   28.337953]  0002   
7ffc606b90f8
[   28.337953]  7ffc606b9108 0246 0008 
7f7a161bf740

[   28.337953] Call Trace:
[   28.337953] Code: 89 de 4c 89 ef e8 7c ca fe ff e9 5c fd ff ff 31 c0 
e9 01 ff ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 
41 55 <41> 54 49 89 f4 53 48 89 fb 48 83 ec 08 0f 20 d0 0f 1f 40 00 0f

[   28.337953] RIP  [] do_page_fault+0xb/0x90
[   28.337953]  RSP <7ffc606b9000>
[   28.337953] ---[ end trace b556ad308185dda4 ]---
[   28.337953] Kernel panic - not syncing: Fatal exception

Can somebody give me a hint how can I debug this?

Thank you.

Jan



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vmadm and overlays

[Ján Poctavek]

On 8. 6. 2017 22:29, Ján Poctavek wrote:

- note that overlays as nic_tag need to be referenced by name and a
(random) number after slash

The number in this case was designed to be a vxlan (or other
encapsulation protocol) identifier.
Is this implemented in some search plugin? I only see the vnetid 
hardcoded in the overlay definition. 
Just for completeness - yes, it's implemented for all overlay types. 
Just omit "-p vnetid=..." from overlay_rules.json. Then vmadm "nic_tag" 
parameter will specify vxlan number of overlay.


Jan



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vmadm and overlays

[Ján Poctavek]

Hi Robert,

thank you for the clarification. Now, with all bits together, it seems 
quite logical. Some additional comments in-line:



On 8. 6. 2017 2:40, Robert Mustacchi wrote:

We never really figured out what a good interface for this would be for
normal SmartOS, so you've mostly found all the bits in Triton that
automate it there and make more sense there. I'll try and explain some
of the things you saw, but if you have ideas on what might make sense,
that'd be useful.


The hardcoded location of overlay_rules.json that has to reside on the 
non-permament storage is a bit weird. One has to create a service early 
in the boot list (definitely before starting zones) just to put the file 
in place. Maybe if there was some alternative (permanent) location that 
can be looked up if not found in the first location. It also could be 
loaded into /run at boot e.g. from /opt/custom/overlays.


Or, alternatively, dladm create-overlay could have some "persist" flag 
that can add new overlay into the overlay_rules.json

- note that overlays as nic_tag need to be referenced by name and a
(random) number after slash

The number in this case was designed to be a vxlan (or other
encapsulation protocol) identifier.
Is this implemented in some search plugin? I only see the vnetid 
hardcoded in the overlay definition.


Jan


---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vmadm and overlays

[Ján Poctavek]

Just to add credits, I've done it with great help of jurajlutter.

Thanks


On 7. 6. 2017 23:59, Ján Poctavek wrote:
Okay, seems I've figured it out. But it was not an easy one. The 
documentation about this is nonexistent.


When using overlays with VMs, the overlays are created automatically 
and they don't need to be created by dladm command.


Here's the thing:

- on each server, you need to create this file with parameters how to 
(automatically) create an overlay:

# mkdir -p /var/run/smartdc/networking/
# vim /var/run/smartdc/networking/overlay_rules.json
{
"myoverlay": "-e vxlan -s direct -p vxlan/listen_ip=158.69.227.172 
-p direct/dest_ip=95.168.205.35 -p direct/dest_port=4789 -p vnetid=666 
-p mtu=1400"

}
- on the other server, swap listen and dest IP
- test that your overlay is recognized:
# nictagadm list | grep myoverlay
- and when creating a VM, reference the nic tag this way:
  "nics": [
{
  "interface": "net0",
  "nic_tag": "myoverlay/7",
  "vlan_id": 555,
  ...
}
- note that overlays as nic_tag need to be referenced by name and a 
(random) number after slash


- and also note that "-s direct" overlay can be created only between 
two (and no more) SmartOS servers


Hope it will help somebody.

Jan

On 7. 6. 2017 14:55, Ján Poctavek wrote:

Hi guys,

I'm trying to work with SmartOS overlays and it seems I'd really
utilize some help here. I have created a simple point-to-point
(direct) overlay between two servers:

-  node1:
dladm create-overlay -e vxlan -s direct -p
vxlan/listen_ip=158.69.227.172 -p direct/dest_ip=95.168.205.35 -p
direct/dest_port=4789 -v 666 olay0
dladm create-vnic -l olay0 -v 555 olayvlan0
ipadm create-if olayvlan0
ipadm create-addr -t -T static -a 10.79.79.101 olayvlan0/olay
 node2:
dladm create-overlay -e vxlan -s direct -p
direct/dest_ip=158.69.227.172 -p vxlan/listen_ip=95.168.205.35 -p
direct/dest_port=4789 -v 666 olay0
dladm create-vnic -l olay0 -v 555 olayvlan0
ipadm create-if olayvlan0
ipadm create-addr -t -T static -a 10.79.79.100 olayvlan0/olay

To this point, everything is working fine and I can ping the
   addresses.

But I'm unable to figure out how can I use the overlays in vmadm.
Nictagadm refuses to add nic tag on overlay (as it does with 
etherstubs).


Any hints?

Thank you.

Jan







---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] vmadm and overlays

[Ján Poctavek]

Okay, seems I've figured it out. But it was not an easy one. The 
documentation about this is nonexistent.


When using overlays with VMs, the overlays are created automatically and 
they don't need to be created by dladm command.


Here's the thing:

- on each server, you need to create this file with parameters how to 
(automatically) create an overlay:

# mkdir -p /var/run/smartdc/networking/
# vim /var/run/smartdc/networking/overlay_rules.json
{
"myoverlay": "-e vxlan -s direct -p vxlan/listen_ip=158.69.227.172 
-p direct/dest_ip=95.168.205.35 -p direct/dest_port=4789 -p vnetid=666 
-p mtu=1400"

}
- on the other server, swap listen and dest IP
- test that your overlay is recognized:
# nictagadm list | grep myoverlay
- and when creating a VM, reference the nic tag this way:
  "nics": [
{
  "interface": "net0",
  "nic_tag": "myoverlay/7",
  "vlan_id": 555,
  ...
}
- note that overlays as nic_tag need to be referenced by name and a 
(random) number after slash


- and also note that "-s direct" overlay can be created only between two 
(and no more) SmartOS servers


Hope it will help somebody.

Jan

On 7. 6. 2017 14:55, Ján Poctavek wrote:

Hi guys,

I'm trying to work with SmartOS overlays and it seems I'd really
utilize some help here. I have created a simple point-to-point
(direct) overlay between two servers:

-  node1:
dladm create-overlay -e vxlan -s direct -p
vxlan/listen_ip=158.69.227.172 -p direct/dest_ip=95.168.205.35 -p
direct/dest_port=4789 -v 666 olay0
dladm create-vnic -l olay0 -v 555 olayvlan0
ipadm create-if olayvlan0
ipadm create-addr -t -T static -a 10.79.79.101 olayvlan0/olay
 node2:
dladm create-overlay -e vxlan -s direct -p
direct/dest_ip=158.69.227.172 -p vxlan/listen_ip=95.168.205.35 -p
direct/dest_port=4789 -v 666 olay0
dladm create-vnic -l olay0 -v 555 olayvlan0
ipadm create-if olayvlan0
ipadm create-addr -t -T static -a 10.79.79.100 olayvlan0/olay

To this point, everything is working fine and I can ping the
   addresses.

But I'm unable to figure out how can I use the overlays in vmadm.
Nictagadm refuses to add nic tag on overlay (as it does with etherstubs).

Any hints?

Thank you.

Jan




---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

[smartos-discuss] vmadm and overlays

[Ján Poctavek]

Hi guys,

I'm trying to work with SmartOS overlays and it seems I'd really utilize 
some help here. I have created a simple point-to-point (direct) overlay 
between two servers:


- node1:
dladm create-overlay -e vxlan -s direct -p 
vxlan/listen_ip=158.69.227.172 -p direct/dest_ip=95.168.205.35 -p 
direct/dest_port=4789 -v 666 olay0

dladm create-vnic -l olay0 -v 555 olayvlan0
ipadm create-if olayvlan0
ipadm create-addr -t -T static -a 10.79.79.101 olayvlan0/olay
- node2:
dladm create-overlay -e vxlan -s direct -p direct/dest_ip=158.69.227.172 
-p vxlan/listen_ip=95.168.205.35 -p direct/dest_port=4789 -v 666 olay0

dladm create-vnic -l olay0 -v 555 olayvlan0
ipadm create-if olayvlan0
ipadm create-addr -t -T static -a 10.79.79.100 olayvlan0/olay

To this point, everything is working fine and I can ping the 
10.79.79.10[01] addresses.


But I'm unable to figure out how can I use the overlays in vmadm. 
Nictagadm refuses to add nic tag on overlay (as it does with etherstubs).


Any hints?

Thank you.

Jan



---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com

Re: [smartos-discuss] Vnc

[Ján Poctavek]

If you need only to check if webserver is working, you can do it also 
from command line:


#> telnet  80
GET / HTTP/1.1
Host: www.yourweb.tld
(end with two enters)

You should see a long html printing or 302 (redirect). In case of 302, 
adjust the path from "/" to whatever you need in url.


If you have https, use this to connect:
#> openssl s_client -connect www.yourweb.tld:443

Jan


On 10. 5. 2017 18:53, cristian pancià wrote:
Can someone explain how to view the X stuff on a single Workstation 
running smartOS as it boot from console with no Graphics stuff,owning 
no supplementary tablet smartphone or pc with the possibilities to 
view X?

how can i figure out if a webserver for a client its set in a decent way?
Is fbida source code possible to install on my wkstation?Anyone tried 
that?

Thank you
*smartos-discuss* | Archives 
 
 
| Modify 
 
Your Subscription 	[Powered by Listbox] 







---
smartos-discuss
Archives: https://www.listbox.com/member/archive/184463/=now
RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=25769125_secret=25769125-7688e9fb
Powered by Listbox: http://www.listbox.com