Bug#452721: [Pkg-xen-devel] Bug#452721: irt: Bug#452721 notes from explorations

2023-07-31 Thread zithro

On 31 Jul 2023 03:39, Elliott Mitchell wrote:


Presently I hope to convince the Xen core to allow full Python in domain
configuration files, but no news on that front so far.  This would mean
/etc/default/xendomains would need to change to match Python syntax.


There was an answer today on xen-devel: the ability to use scripts in 
domU cfg files has been explicitely removed for various reasons.
This does not prevent you from "source"-ing teh cfg files in your 
script(s) if they are proper Python syntax. Or you could simply 
parse/regex the values you want.
And as Marek suggested in his answer, you can also put any arbitrary 
settings in the comments.


Although ...


My thinking for adding to domain configuration files would be something
along these lines:

init = {
'tool': 'xendomains-ng',
'version': 0,
'order': 9,
'startwait': 60,
'stopaction': 'save',
}


The problem with adding this to a domU config file is that it could 
cause problems for (live) migrations. The start/stop order is "per 
dom0", and may be different on another one.
Imagine two dom0s, one storing the domain files "locally", while the 
other uses NFS. Only in the second case the domU should wait for the NFS 
server/domain to be available.


To me, the start/stop logic should be in a dom0 config file.


'startwait' would tell the script to wait that long before starting
subsquent domains.


A time-based wait may be useful for when everything goes well, but what 
about when there are problems ?
If you want to be sure a domain is up (ie. ready to serve), you would 
need to peek at the related "service".
For example, to be sure a DNS domU is up, you would have to try a DNS 
request, as a ping or "xl list" would not be enough.
Also, domains in xen/auto are started with a mix of serialization AND 
parallelization, as "xl create" returns once the domain has started (ie. 
in the Xen point of view, not the user's).



'stopaction' would allow different actions if the machine was to stop.
The 3 options which come to mind are 'stop' (shutdown), 'save' (save to
specified storage location), and 'migrate'.


Then, each time you do NOT want to follow the usual action, you'd have 
to edit -each- domU cfg file ?



If full Python doesn't become available, this might take the format:
init = 'tool=xendomains-ng,version=0,order=9,startwait=60,stopaction=save'
Not needing to parse the string though does make one's life simpler.


Well, it makes -your- life easier, not the maintainers' one ;)


I'm basically certain writing a new xendomains script in Python is the
way to go.  Now to get an answer as to whether full Python in domain
configuration files could be reenabled.


I'm not sure a Python script would solve anything, as (ba)sh variables 
are imported from other files.
(see for example 
https://salsa.debian.org/xen-team/debian-xen/-/blob/master/tools/hotplug/Linux/xendomains.in)


Everything considered, I'm not sure why Xen should provide such 
functionnality.
I think custom scripts can handle all the various use cases, don't you 
think ?
PS: as mentionned by diederik, the "dependency" logic is already handled 
by Qubes since years, and it never made it to Xen (I don't know the 
reasons though).


But I agree the shutdown sequence could be adapted to :
1. first shutdown the domains NOT in xen/auto
2. then shutdown the domains in xen/auto, in reverse order

For fine grained start/stop order, maybe having a dom0 config file 
handling this could be added, like:


# START/STOP ORDER
# domains not in these lists will be started after and stopped
# before the ones here
start-order=(list of domU names)
stop-order=(list of domU names)

But then again, this only ensures "domains" start order, not "services 
availability" in said domains.


--
zithro / Cyril



Bug#452721: irt: Bug#452721 notes from explorations

2023-07-30 Thread Elliott Mitchell
Even though there hasn't been any discussion recently, bug #452721 is
very much still of major concern to me.

First issue is how to parse domain configuration files.  Reason being a
foo.cfg file might have the configuration 'name = "bar"'.  This would
also let the script retrieve the UUID if that has been set.

Turns out while Python in domain configuration files isn't supportted,
the syntax is still a proper subset of the Python language.  This makes
Python the ideal programming language for a replacement script.  Only
weakness is being able to have full Python syntax in configuration files
might make the task simpler.

Presently I hope to convince the Xen core to allow full Python in domain
configuration files, but no news on that front so far.  This would mean
/etc/default/xendomains would need to change to match Python syntax.


My thinking for adding to domain configuration files would be something
along these lines:

init = {
'tool': 'xendomains-ng',
'version': 0,
'order': 9,
'startwait': 60,
'stopaction': 'save',
}

Mainly a Python dictionary holding key values.  Thought being the 'tool'
and 'version' values, is to hope for some form of compatibility if such
scripts were to become common.

My thinking is 'order' would indicate sequence.  Domains with higher
order get started first (same order would nominally allow parallel
start).  If a domain.cfg file didn't define order then its order is 0.

'startwait' would tell the script to wait that long before starting
subsquent domains.

'stopaction' would allow different actions if the machine was to stop.
The 3 options which come to mind are 'stop' (shutdown), 'save' (save to
specified storage location), and 'migrate'.


If full Python doesn't become available, this might take the format:

init = 'tool=xendomains-ng,version=0,order=9,startwait=60,stopaction=save'

Not needing to parse the string though does make one's life simpler.


Other concerns include:
Sometimes you may want to take a distinct action during stop.  Ie if
you're doing restarts for kernel updates, you'll want to override and
have domains reboot.

It may be handier to have distinct options for 'restart'.  Full restarts
can follow proper order, or could simply involve bouncing domains based
on order.  Notably with HVM domains and Qemu updates, you could do:

order 0 down, order 1 down, order 9 down, order 9 up, order 2 up, order 0 up

Or you could do:

order 9 down, order 9 up, order 1 down, order 1 up, order 0 down, order 0 up


I'm basically certain writing a new xendomains script in Python is the
way to go.  Now to get an answer as to whether full Python in domain
configuration files could be reenabled.


-- 
(\___(\___(\__  --=> 8-) EHM <=--  __/)___/)___/)
 \BS (| ehem+sig...@m5p.com  PGP 87145445 |)   /
  \_CS\   |  _  -O #include  O-   _  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445