Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Fri, Mar 13, 2015 at 10:32:41AM +, Ian Campbell wrote: (culling the cc list a lot) On Thu, 2015-03-12 at 11:54 +, Jan Beulich wrote: On 12.03.15 at 11:21, wei.l...@citrix.com wrote: == Linux == Wouldn't it make sense to move external projects down, and have our core pieces (hypervisor, tool stack, maybe qemu) be near the top? When I originally started tracking external things back in whichever release I was RM for there was only a couple of big ticket external items. It seems that today there are dozens of things ranging from the small to the large which are obscuring the work which is actually happening within the Xen release itself (the list is now so long that is a proper chore to read through it and pay attention). Added to that is the fact that the list in general has become something of a laundry list of everything anyone has ever thought of or someone once considered working on, rather than things which are of some importance to the 4.6 release. Along with the fact that it must be a load of work to maintain I think there is a danger nobody will read it because it is so overwhelming. Wei, perhaps one or more of these could be applied: * raise the bar for inclusion in the list for external projects to be only the very largest most important items; * Stop tracking external projects altogether unless they have a direct interaction with the 4.6 release; * raise the bar for inclusion in the list for all projects a bit, i.e. not every little change to xen.git needs to be tracked; * be more aggressive about garbage collecting old ideas which aren't seeing any actual progress in this release window. I've been actively doing #4 by dropping lots of stuffs since the beginning. I've removed those up for grabs items because they can be / should be tracked in bug tracker. I also removed some Linux projects which don't seem to require interaction with Xen. Wei. WRT the third one, I wonder if it is necessary for the RM to track everything which is going on in the world WRT Xen. Just tracking those items which we as a community have decided we want to get into 4.6 and for which we feel there is a realistic chance of that happening would make the list more manageable, which reduces the burden on you as well as those trying to read the list. I'm not saying that only things which are considered blockers for 4.6 should be on the list, but perhaps raise the bar from including every possible wishlist item e.g. to only include important stuff, and drop the nice to haves? Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, 12 Mar 2015 10:21:56 + wei.l...@citrix.com wrote: We are now two months into 4.6 development window. This is an email to keep track of all the patch series I gathered. It is by no means complete and / or acurate. Feel free to reply this email with new projects or correct my misunderstanding. = Timeline = We are planning on a 9-month release cycle, but we could also release a bit earlier if everything goes well (no blocker, no critical bug). * Development start: 6 Jan 2015 === We are here === * Feature Freeze: 10 Jul 2015 * RCs: TBD * Release Date: 9 Oct 2015 (could release earlier) The RCs and release will of course depend on stability and bugs, and will therefore be fairly unpredictable. Bug-fixes, if Acked-by by maintainer, can go anytime before the First RC. Later on we will need to figure out the risk of regression/reward to eliminate the possiblity of a bug introducing another bug. = Prognosis = The states are: none - fair - ok - good - done none - nothing yet fair - still working on it, patches are prototypes or RFC ok - patches posted, acting on review good - some last minute pieces done - all done, might have bugs = Bug Fixes = Bug fixes can be checked in without a freeze exception throughout the freeze, unless the maintainer thinks they are particularly high risk. In later RC's, we may even begin rejecting bug fixes if the broken functionality is small and the risk to other functionality is high. Document changes can go in anytime if the maintainer is OK with it. These are guidelines and principles to give you an idea where we're coming from; if you think there's a good reason why making an exception for you will help us make Xen better than not doing so, feel free to make your case. [...] I have been meaning to write this email for a while now, just to let everyone know we're working on a couple more patches related to VM introspection. They are not as big as our initial ones, but they do bring in new functionality. Răzvan Cojocaru will post a series based on Tamas' changes, as soon as it stabilizes some more or (even better) gets merged. I'll let him choose the right moment. I will also pick up some of my [oldish] patches with reviews from Andrew and Ian and also try to tackle some of the x86 emulator challenges that we talked about some months ago. We've adjusted our plans and aim for a to-the-point set of changes that allow one to built a security solution (at least) the we envision it, in terms of isolation and monitoring capability. Regards, -- Mihai Donțu pgpGS4KQ6pQ3j.pgp Description: OpenPGP digital signature ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Monday 16 March 2015 18:18:22 Razvan Cojocaru wrote: On 03/16/2015 06:00 PM, Lars Kurth wrote: On 16 Mar 2015, at 13:01, Mihai Donțu mdo...@bitdefender.com wrote: On Thu, 12 Mar 2015 10:21:56 + wei.l...@citrix.com wrote: We are now two months into 4.6 development window. This is an email to keep track of all the patch series I gathered. It is by no means complete and / or acurate. Feel free to reply this email with new projects or correct my misunderstanding. = Timeline = We are planning on a 9-month release cycle, but we could also release a bit earlier if everything goes well (no blocker, no critical bug). * Development start: 6 Jan 2015 === We are here === * Feature Freeze: 10 Jul 2015 * RCs: TBD * Release Date: 9 Oct 2015 (could release earlier) The RCs and release will of course depend on stability and bugs, and will therefore be fairly unpredictable. Bug-fixes, if Acked-by by maintainer, can go anytime before the First RC. Later on we will need to figure out the risk of regression/reward to eliminate the possiblity of a bug introducing another bug. = Prognosis = The states are: none - fair - ok - good - done none - nothing yet fair - still working on it, patches are prototypes or RFC ok - patches posted, acting on review good - some last minute pieces done - all done, might have bugs = Bug Fixes = Bug fixes can be checked in without a freeze exception throughout the freeze, unless the maintainer thinks they are particularly high risk. In later RC's, we may even begin rejecting bug fixes if the broken functionality is small and the risk to other functionality is high. Document changes can go in anytime if the maintainer is OK with it. These are guidelines and principles to give you an idea where we're coming from; if you think there's a good reason why making an exception for you will help us make Xen better than not doing so, feel free to make your case. [...] I have been meaning to write this email for a while now, just to let everyone know we're working on a couple more patches related to VM introspection. They are not as big as our initial ones, but they do bring in new functionality. Mihai, it would make Wei's life easier if you could provide headlines for those patches. That way they can be tracked before you post them For my part, the patches are: 1. xen: Add support for XSETBV vm_events This is basically VMX support for sending out an event on VMEXIT / EXIT_REASON_XSETBV. Additional information sent out is the XCR (the value of ECX). 2. xen: Support hybernating guests This patch cover two areas: A) send data (just a regular blob / buffer) back to the HV in the vm_event response, and B) have that data returned by the read function when emulating an instruction. Unless we do this, monitored guests won't be able to properly wake up from hybernation. 3. xen: Support for VMCALL-based vm_events This is a modification of the VMCALL patch in my original RFC series, which got dropped last year. The modification takes into account Andrew Cooper's suggestion to just use a hypercall: http://lists.xen.org/archives/html/xen-devel/2014-07/msg01677.html 4. xen: Deny MSR writes if refused by the vm_event reply Preempt MSR writes that the monitoring application decides are evil. 5. xen: Implement actual write of CR values on xc_vcpu_setcontext() Although libxc's API leads one to believe that all info in the context will be set for the guest, the CR values were actually ignored for HVM guests. This patch addresses that problem. Hope this helps, Mihai will complete the picture with the rest. 6. xmalloc: pool integrity This is a simple mechanism that gives an early heads-up that the xen 'heap' has been corrupted. 7. tools: coding-style auto-adjustments This is not really related to memory introspection, it's just a script around clang-format which I put together while working on the introspection. It received positive feedback, but is a bit problematic for users of 'older' distributions (clang-format is rather new). 8. x86_emulate: dump the opcodes of unsupported instructions This is something I have not yet started to work, but I feel it will help me to easily extend the emulator, but dumping the opcodes of unsupported instructions (in debug mode). This way I can pasted them, dump the instruction with ndisasm or the like, and then proceed to making the proper adjustments. There was an even better idea from Andrew to ask the KVM folks to join forces into a common x86 emulator (since they've started from an older version of Xen's), but we're a bit time constrained now. :-( 9. x86_emulate: add support for unsupported instructions This should make Jan and the other folks happy, since they rejected our initial idea of a generic workaround and instead wanted us to
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 16 Mar 2015, at 13:01, Mihai Donțu mdo...@bitdefender.com wrote: On Thu, 12 Mar 2015 10:21:56 + wei.l...@citrix.com wrote: We are now two months into 4.6 development window. This is an email to keep track of all the patch series I gathered. It is by no means complete and / or acurate. Feel free to reply this email with new projects or correct my misunderstanding. = Timeline = We are planning on a 9-month release cycle, but we could also release a bit earlier if everything goes well (no blocker, no critical bug). * Development start: 6 Jan 2015 === We are here === * Feature Freeze: 10 Jul 2015 * RCs: TBD * Release Date: 9 Oct 2015 (could release earlier) The RCs and release will of course depend on stability and bugs, and will therefore be fairly unpredictable. Bug-fixes, if Acked-by by maintainer, can go anytime before the First RC. Later on we will need to figure out the risk of regression/reward to eliminate the possiblity of a bug introducing another bug. = Prognosis = The states are: none - fair - ok - good - done none - nothing yet fair - still working on it, patches are prototypes or RFC ok - patches posted, acting on review good - some last minute pieces done - all done, might have bugs = Bug Fixes = Bug fixes can be checked in without a freeze exception throughout the freeze, unless the maintainer thinks they are particularly high risk. In later RC's, we may even begin rejecting bug fixes if the broken functionality is small and the risk to other functionality is high. Document changes can go in anytime if the maintainer is OK with it. These are guidelines and principles to give you an idea where we're coming from; if you think there's a good reason why making an exception for you will help us make Xen better than not doing so, feel free to make your case. [...] I have been meaning to write this email for a while now, just to let everyone know we're working on a couple more patches related to VM introspection. They are not as big as our initial ones, but they do bring in new functionality. Mihai, it would make Wei's life easier if you could provide headlines for those patches. That way they can be tracked before you post them Regards Lars ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 16 Mar 2015, at 10:43, Wei Liu wei.l...@citrix.com wrote: On Fri, Mar 13, 2015 at 10:32:41AM +, Ian Campbell wrote: Wei, perhaps one or more of these could be applied: * raise the bar for inclusion in the list for external projects to be only the very largest most important items; * Stop tracking external projects altogether unless they have a direct interaction with the 4.6 release; * raise the bar for inclusion in the list for all projects a bit, i.e. not every little change to xen.git needs to be tracked; * be more aggressive about garbage collecting old ideas which aren't seeing any actual progress in this release window. I've been actively doing #4 by dropping lots of stuffs since the beginning. I've removed those up for grabs items because they can be / should be tracked in bug tracker. I also removed some Linux projects which don't seem to require interaction with Xen. Wei. Wei, I copied the stuff that has been done to http://wiki.xenproject.org/wiki/Xen_Project_Hypervisor_Roadmap/4.6#Xen_4.6_Completed_Features based on this thread (and some of the replies) Regards Lars ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
* vsyscall in Linux (fair) - Konrad Rzeszutek Wilk Not done. Still in 'git stash'. * Convert tasklet to per-cpu tasklets (fair) RFC posted - Konrad Rzeszutek Wilk Pff.. I've the patches but would need to post them and redo them. Too busy right now. Might as well move them to Xen 4.7 or so. Eventually I will get the time. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, Mar 12, 2015 at 11:54:15AM +, Jan Beulich wrote: On 12.03.15 at 11:21, wei.l...@citrix.com wrote: == Linux == Wouldn't it make sense to move external projects down, and have our core pieces (hypervisor, tool stack, maybe qemu) be near the top? * Convert tasklet to per-cpu tasklets (fair) RFC posted - Konrad Rzeszutek Wilk Is this still a valid item? I think we went another route, albeit that work is still incomplete iirc. Konrad? Right. I want to finish up the another route before I go back to this. I think it is still valid - as it can help with other tasklet uses. However I would need to do instrumentation/perf testing before I post them. * Further tmem cleanups/fixes (16TB etc) (fair) - Bob Liu As we can now support more than 16Tb, I don't think this number should be mentioned anymore. * amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch) Which one? * Feature masking MSR support (enhancement) (in master) What is this about? Or what status does in master actually refer to? That they are done. In 'master' branch. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 13/03/15 18:02, Konrad Rzeszutek Wilk wrote: * vsyscall in Linux (fair) - Konrad Rzeszutek Wilk Not done. Still in 'git stash'. * Convert tasklet to per-cpu tasklets (fair) RFC posted - Konrad Rzeszutek Wilk Pff.. I've the patches but would need to post them and redo them. Too busy right now. xen-unstable currently has the broken implementation in, as they were reverted out of 4.5 at the 11th hour, but not out of master which has branched by that point. If you don't have time to fix it, then they need reverting out of unstable as well. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
== Hypervisor == * Alternate p2m: support multiple copies of host p2m (ok) - Ed White I'm hoping to see some progress on getting this restarted in the next 2 or 3 weeks, with additional Intel resources. Ed ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Fri, Mar 13, 2015 at 06:05:15PM +, Andrew Cooper wrote: On 13/03/15 18:02, Konrad Rzeszutek Wilk wrote: * vsyscall in Linux (fair) - Konrad Rzeszutek Wilk Not done. Still in 'git stash'. * Convert tasklet to per-cpu tasklets (fair) RFC posted - Konrad Rzeszutek Wilk Pff.. I've the patches but would need to post them and redo them. Too busy right now. xen-unstable currently has the broken implementation in, as they were reverted out of 4.5 at the 11th hour, but not out of master which has branched by that point. If you don't have time to fix it, then they need reverting out of unstable as well. Correct. I am not referring to that one. As I mentioned in my reply to Jan - I need to fix the broken one in staging - before I even contemplate doing anything of this mentioned above. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, Mar 12, Daniel Kiper wrote: IIRC, 4.3 release (and probably earlier ones) was a mess. Olaf did the work for 4.5. However, as he pointed out in another email last one is incorrect. Olaf, are you going to fix it for 4.6? I suppose it is quite simple. There are still many hardcoded /var related strings in the code. Last time I looked at this it was too cumbersome to change all of them to depend on --localstatedir (or whatever). I remember the remainders are consistent in itself so I left them alone. The only thing I would touch for 4.6 is to make the odd /var/xen/dump tuneable via configure. We patch the path since many years. So yes, I will put this on my TODO list. Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
* Intel memory bandwidth monitoring for VMs (fair) v9 posted - Chao Peng Wei, this one is already merged. Thanks, Chao ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
(culling the cc list a lot) On Thu, 2015-03-12 at 11:54 +, Jan Beulich wrote: On 12.03.15 at 11:21, wei.l...@citrix.com wrote: == Linux == Wouldn't it make sense to move external projects down, and have our core pieces (hypervisor, tool stack, maybe qemu) be near the top? When I originally started tracking external things back in whichever release I was RM for there was only a couple of big ticket external items. It seems that today there are dozens of things ranging from the small to the large which are obscuring the work which is actually happening within the Xen release itself (the list is now so long that is a proper chore to read through it and pay attention). Added to that is the fact that the list in general has become something of a laundry list of everything anyone has ever thought of or someone once considered working on, rather than things which are of some importance to the 4.6 release. Along with the fact that it must be a load of work to maintain I think there is a danger nobody will read it because it is so overwhelming. Wei, perhaps one or more of these could be applied: * raise the bar for inclusion in the list for external projects to be only the very largest most important items; * Stop tracking external projects altogether unless they have a direct interaction with the 4.6 release; * raise the bar for inclusion in the list for all projects a bit, i.e. not every little change to xen.git needs to be tracked; * be more aggressive about garbage collecting old ideas which aren't seeing any actual progress in this release window. WRT the third one, I wonder if it is necessary for the RM to track everything which is going on in the world WRT Xen. Just tracking those items which we as a community have decided we want to get into 4.6 and for which we feel there is a realistic chance of that happening would make the list more manageable, which reduces the burden on you as well as those trying to read the list. I'm not saying that only things which are considered blockers for 4.6 should be on the list, but perhaps raise the bar from including every possible wishlist item e.g. to only include important stuff, and drop the nice to haves? Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, Mar 12, Daniel Kiper wrote: On Thu, Mar 12, 2015 at 10:21:56AM +, wei.l...@citrix.com wrote: * Rearrange and cleanup installation destination directories (/var - var/lib/xen) (fair) - Daniel Kiper If Olaf did all work then we can remove this one. What does this mean anyway? Does it mean the --prefix changes I did for 4.5? Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 03/12/2015 10:21 AM, wei.l...@citrix.com wrote: * Credit2: introduce per-vcpu hard and soft affinity (fair) - Justin T. Weaver The most recent patches looked pretty good -- I'd be very surprised if these didn't make it in by July. I'd change this to good. * Default to credit2 (none) cpu pinning, numa affinity and cpu reservation - George Dunlap I think before actually doing a release with credit2 as the default, we want almost an entire development cycle with credit2 as the default, to shake out any latent bugs; and probably an entire release with credit2 listed as production-ready. So maybe this goal would be more helpfully stated as credit2 production ready, so that when we open the next development window we can change the default immediately? * pvUSB support in libxl (fair) - Chunyan Liu You can add in hvmUSB support in libxl here, with my name; and call it fair. * Blktap2 support (none) - George Dunlap I'm planning on doing it, but I haven't spec'd the actual work yet. Is fair the best description for that? -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] Xen 4.6 Development Update (two months reminder)
Hi all
We are now two months into 4.6 development window. This is an email to keep
track of all the patch series I gathered. It is by no means complete and / or
acurate. Feel free to reply this email with new projects or correct my
misunderstanding.
= Timeline =
We are planning on a 9-month release cycle, but we could also release a bit
earlier if everything goes well (no blocker, no critical bug).
* Development start: 6 Jan 2015
=== We are here ===
* Feature Freeze: 10 Jul 2015
* RCs: TBD
* Release Date: 9 Oct 2015 (could release earlier)
The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.
Bug-fixes, if Acked-by by maintainer, can go anytime before the First
RC. Later on we will need to figure out the risk of regression/reward
to eliminate the possiblity of a bug introducing another bug.
= Prognosis =
The states are: none - fair - ok - good - done
none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs
= Bug Fixes =
Bug fixes can be checked in without a freeze exception throughout the
freeze, unless the maintainer thinks they are particularly high
risk. In later RC's, we may even begin rejecting bug fixes if the
broken functionality is small and the risk to other functionality is
high.
Document changes can go in anytime if the maintainer is OK with it.
These are guidelines and principles to give you an idea where we're coming
from; if you think there's a good reason why making an exception for you will
help us make Xen better than not doing so, feel free to make your case.
== Linux ==
* PV domain with memory 512GB (fair)
- Juergen Gross
* Block driver multiqueue support (fair)
RFC posted
- Bob Liu
* Block driver multi-page ring support (fair)
- Bob Liu
* Preemptable privcmd hypercalls (good)
v5 posted
- David Vrabel
* Linux ARM - Device assigment (PCI) (none)
Depends on Xen pieces which are on the Xen 4.6 list.
- Manish Jaggi
* pvUSB in Linux (fronted and backend) (Fair)
- Juergen Gross
* VPMU - 'perf' support in Linux (ok)
Depends on Xen patches
Acked by David Vrabel
- Boris Ostrovsky
* vNUMA in Linux (ok)
v6 posted
git://gitorious.org/vnuma/linux_vnuma.git
- Elena Ufimtseva
* vsyscall in Linux (fair)
- Konrad Rzeszutek Wilk
* COLO Agent in Linux (fair)
- Gui Jianfeng
- Yang Hongyang
- Dong, Eddie
* ARM64 - support 64K guest (none)
- Julien Grall
== OpenStack ==
* setup CI loop for OpenStack (fair)
- Anthony Perard
== FreeBSD ==
* PVH FreeBSD dom0 (ok)
FreeBSD 11 goal. Toolstack side done in Xen 4.5
- Roger Pau Monné
== Other OSes (MiniOS, QNX) ==
* PV drivers for automotive kernels (fair)
- Artem Mygaiev
* mini-os: xenbus changes for rump kernels (ok)
git://xenbits.xen.org/people/iwj/rumpuser-xen.git
branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
v2 posted
- Ian Jackson
== GRUB2 ==
* GRUB2 multiboot2 (fair)
- Daniel Kiper
== OSSTEST ==
* OSSTest: studom test case (none)
- Wei Liu
* OSSTest: libvirt migration (fair)
- Wei Liu
* OSSTest: upgrade to Debian Jessie (none)
- Wei Liu
* OSSTest: performance test (fair)
- Dario Faggioli
* CPU pool test case (fair)
- Dario Faggioli
* Add a FreeBSD host (fair)
- Roger Pau Monné
* Nested virt test case (fair)
- Robert Hu
== QEMU ==
* Linux-based QEMU upstream stub domain (fair)
RFC posted
- Eric Shelton
* Using qemu-upstream in a stubdomain (none)
Will use rump kernels.
- Wei Liu
* AMD Radeon PCI GPU passthrough (none)
Focusing on Xen 4.2 and qemu-traditional
- Kelly Zytaruk
* Intel IGD PCI GPU passthrough (ok)
v5 posted
- Chen, Tiejun
== Up for grabs ==
* save/restore/migrate PVHVM guest with 32 vcpus
http://lists.xen.org/archives/html/xen-devel/2015-02/msg00244.html
* PoD fixes
if you boot with memory = maxmem we have a size estimation bug
* TLB flushing without locks in Xen
* xl does not support specifying virtual function for passthrough device
http://bugs.xenproject.org/xen/bug/22
* PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with
PCI/GPU passthrough
http://bugs.xenproject.org/xen/bug/28
* libx{c,l} error handling cleanup
* Adding missing 'xend' features in libxl
Need to define what is missing
* xl list -l doesn't contain tty console port
* xl: passing more defaults in configuration in xl.conf
There are a number of options for which it might be useful to pass a default
in xl.conf. For example, if we could have a default backend parameter for
vifs, then it would be easy to switch back and forth between a backend in a
driver domain and a backend in dom0.
* PVH - PVH working with shadow.
Based on Tim's work
* PVH - PCI passthrough for DomU.
* AMD performance regressions
* Performance due to hypercall
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
Il 12/03/2015 11:54, George Dunlap ha scritto: On 03/12/2015 10:21 AM, wei.l...@citrix.com wrote: * Credit2: introduce per-vcpu hard and soft affinity (fair) - Justin T. Weaver The most recent patches looked pretty good -- I'd be very surprised if these didn't make it in by July. I'd change this to good. * Default to credit2 (none) cpu pinning, numa affinity and cpu reservation - George Dunlap I think before actually doing a release with credit2 as the default, we want almost an entire development cycle with credit2 as the default, to shake out any latent bugs; and probably an entire release with credit2 listed as production-ready. So maybe this goal would be more helpfully stated as credit2 production ready, so that when we open the next development window we can change the default immediately? * pvUSB support in libxl (fair) - Chunyan Liu You can add in hvmUSB support in libxl here, with my name; and call it fair. You did a new patchset version? The latest I saw, rebased and tested (including compatibility with actual spice usb redirection) was long time ago and I not saw newer posted in xen-devel. * Blktap2 support (none) - George Dunlap I'm planning on doing it, but I haven't spec'd the actual work yet. Is fair the best description for that? -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 03/12/2015 11:34 AM, Fabio Fantoni wrote: Il 12/03/2015 11:54, George Dunlap ha scritto: On 03/12/2015 10:21 AM, wei.l...@citrix.com wrote: * Credit2: introduce per-vcpu hard and soft affinity (fair) - Justin T. Weaver The most recent patches looked pretty good -- I'd be very surprised if these didn't make it in by July. I'd change this to good. * Default to credit2 (none) cpu pinning, numa affinity and cpu reservation - George Dunlap I think before actually doing a release with credit2 as the default, we want almost an entire development cycle with credit2 as the default, to shake out any latent bugs; and probably an entire release with credit2 listed as production-ready. So maybe this goal would be more helpfully stated as credit2 production ready, so that when we open the next development window we can change the default immediately? * pvUSB support in libxl (fair) - Chunyan Liu You can add in hvmUSB support in libxl here, with my name; and call it fair. You did a new patchset version? The latest I saw, rebased and tested (including compatibility with actual spice usb redirection) was long time ago and I not saw newer posted in xen-devel. No, and I'm not going to post one until the pvUSB stuff gets in, since that work will involve refactoring the pvUSB code into PV and DEVICEMODEL paths, and I don't fancy having to repeat that work every time there's another revision. :-) The core technology is working just fine; it's only the library interface that really needed work in the last patch, and that will be set by the pvUSB series. So once that's in, it should be a relatively small amount of work. -George ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 3/12/2015 6:54 AM, Jan Beulich wrote: * amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch) Which one? This is in reference to the patches to microcode_amd and it's all 'done' in 4.5 itself. I think when we were tracking this, commit 8b24b07e was not in master branch. Hence the status. We can remove this from the current list of features to be tracked for 4.6 IMO. * Feature masking MSR support (enhancement) (in master) What is this about? Or what status does in master actually refer to? This is in reference to commit e74de9c0. And this is also complete, and can be removed from the list. Before I knew what status values to apply to a feature, I had used 'in master' to convey that patches are in master branch. The feature tracking list has maintained this remnant. In fact, all these patches in the list are 'done' as of 4.5 itself and can be removed IMO. *Data breakpoint Extension support (new-feat) (in master) *Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master) *fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups) *multiple AMD container files appended together in initrd (early initramfs) -Aravind and Suravee Thanks, -Aravind. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 3/12/2015 at 06:21 PM, in message
e1yw0fy-0002p7...@ukmail1.uk.xensource.com, wei.l...@citrix.com wrote:
Hi all
We are now two months into 4.6 development window. This is an email to keep
track of all the patch series I gathered. It is by no means complete and /
or
acurate. Feel free to reply this email with new projects or correct my
misunderstanding.
= Timeline =
We are planning on a 9-month release cycle, but we could also release a bit
earlier if everything goes well (no blocker, no critical bug).
* Development start: 6 Jan 2015
=== We are here ===
* Feature Freeze: 10 Jul 2015
* RCs: TBD
* Release Date: 9 Oct 2015 (could release earlier)
The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.
Bug-fixes, if Acked-by by maintainer, can go anytime before the First
RC. Later on we will need to figure out the risk of regression/reward
to eliminate the possiblity of a bug introducing another bug.
= Prognosis =
The states are: none - fair - ok - good - done
none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs
= Bug Fixes =
Bug fixes can be checked in without a freeze exception throughout the
freeze, unless the maintainer thinks they are particularly high
risk. In later RC's, we may even begin rejecting bug fixes if the
broken functionality is small and the risk to other functionality is
high.
Document changes can go in anytime if the maintainer is OK with it.
These are guidelines and principles to give you an idea where we're coming
from; if you think there's a good reason why making an exception for you
will
help us make Xen better than not doing so, feel free to make your case.
== Linux ==
* PV domain with memory 512GB (fair)
- Juergen Gross
* Block driver multiqueue support (fair)
RFC posted
- Bob Liu
* Block driver multi-page ring support (fair)
- Bob Liu
* Preemptable privcmd hypercalls (good)
v5 posted
- David Vrabel
* Linux ARM - Device assigment (PCI) (none)
Depends on Xen pieces which are on the Xen 4.6 list.
- Manish Jaggi
* pvUSB in Linux (fronted and backend) (Fair)
- Juergen Gross
* VPMU - 'perf' support in Linux (ok)
Depends on Xen patches
Acked by David Vrabel
- Boris Ostrovsky
* vNUMA in Linux (ok)
v6 posted
git://gitorious.org/vnuma/linux_vnuma.git
- Elena Ufimtseva
* vsyscall in Linux (fair)
- Konrad Rzeszutek Wilk
* COLO Agent in Linux (fair)
- Gui Jianfeng
- Yang Hongyang
- Dong, Eddie
* ARM64 - support 64K guest (none)
- Julien Grall
== OpenStack ==
* setup CI loop for OpenStack (fair)
- Anthony Perard
== FreeBSD ==
* PVH FreeBSD dom0 (ok)
FreeBSD 11 goal. Toolstack side done in Xen 4.5
- Roger Pau Monné
== Other OSes (MiniOS, QNX) ==
* PV drivers for automotive kernels (fair)
- Artem Mygaiev
* mini-os: xenbus changes for rump kernels (ok)
git://xenbits.xen.org/people/iwj/rumpuser-xen.git
branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
v2 posted
- Ian Jackson
== GRUB2 ==
* GRUB2 multiboot2 (fair)
- Daniel Kiper
== OSSTEST ==
* OSSTest: studom test case (none)
- Wei Liu
* OSSTest: libvirt migration (fair)
- Wei Liu
* OSSTest: upgrade to Debian Jessie (none)
- Wei Liu
* OSSTest: performance test (fair)
- Dario Faggioli
* CPU pool test case (fair)
- Dario Faggioli
* Add a FreeBSD host (fair)
- Roger Pau Monné
* Nested virt test case (fair)
- Robert Hu
== QEMU ==
* Linux-based QEMU upstream stub domain (fair)
RFC posted
- Eric Shelton
* Using qemu-upstream in a stubdomain (none)
Will use rump kernels.
- Wei Liu
* AMD Radeon PCI GPU passthrough (none)
Focusing on Xen 4.2 and qemu-traditional
- Kelly Zytaruk
* Intel IGD PCI GPU passthrough (ok)
v5 posted
- Chen, Tiejun
== Up for grabs ==
* save/restore/migrate PVHVM guest with 32 vcpus
http://lists.xen.org/archives/html/xen-devel/2015-02/msg00244.html
* PoD fixes
if you boot with memory = maxmem we have a size estimation bug
* TLB flushing without locks in Xen
* xl does not support specifying virtual function for passthrough device
http://bugs.xenproject.org/xen/bug/22
* PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with
PCI/GPU passthrough
http://bugs.xenproject.org/xen/bug/28
* libx{c,l} error handling cleanup
* Adding missing 'xend' features in libxl
Need to define what is missing
* xl list -l doesn't contain tty console
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, 2015-03-12 at 10:21 +, wei.l...@citrix.com wrote: * Repurpose SEDF Scheduler for Real-time (fair) RFC patch posted (v2) - Joshua Whitehead, Robert VanVossen This was superceded by the RTDS stuff, wasn't it? === Hypervisor ARM === * Add support for Xilinx ZynqMP SoC (fair) - Edgar E. Iglesias Done * Add support for Huawei hip04-d01 platform (ok) - Frediano Ziglio Done * Thunder X platform support (ok) - Vijay Kilari Done, with one exception * ARM - SMMU resync of Linux's one (ok) - Julien Grall Done * Rearrange and cleanup installation destination directories (/var - var/lib/xen) (fair) - Daniel Kiper What is this? I've never heard about it. My local tree has these after make dist: dist/install/var dist/install/var/lib dist/install/var/lib/xen dist/install/var/lib/xenstored dist/install/var/lib/xen/xenpaging dist/install/var/log dist/install/var/log/xen dist/install/var/xen dist/install/var/xen/dump which all seems proper and correct to me. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, 2015-03-12 at 15:07 +, Ian Campbell wrote: On Thu, 2015-03-12 at 10:21 +, wei.l...@citrix.com wrote: * Repurpose SEDF Scheduler for Real-time (fair) RFC patch posted (v2) - Joshua Whitehead, Robert VanVossen This was superceded by the RTDS stuff, wasn't it? I haven't head from Joshua and Robert in a while, so I don't really know whether they're still working on this, but I think they're not. And yes, we have a new and modern real-time scheduling now, so I would rather direct all the effort toward it (to move it out from 'experimental' status), and start thinking at ways to deprecate/get rid of SEDF. Regards, Dario I apologize for our lack of communication and not keeping the list up to date on this. This did get pushed to the back burner for a while as other responsibilities came along and we were also waiting to see what the RT-Xen folks were able to come up with as it looked quite promising. However we (Robbie and I) have been discussing the last week or so here how we'd like to proceed on this. At one point it had been put forward that we would submit our own complete scheduler and then look at combining the two (ours and rt-xen's) into one coherent scheduler, but we're not longer sure that's the best route forward. We did have a couple ideas that may be more easily implemented starting from the SEDF code, but we're still investigating some of that. We still have to discuss it with our managers, so I don't want to commit to anything just yet, but I think Robbie and I are leaning toward contributing bug fixes and improvements to sched_rt to move it out of experimental/tech preview status as well as contributing one or more new deadline algorithms (CBS being the obvious first choice if that's still desired by the community). We've had some new projects in the Xen arena begin here at DornerWorks (supporting Xen on the new Xilinx Zynq UltraScale+ MPSoC being the big one) and we hope to making contributions in several areas to the project in the next few months. I will push the discussion of this topic in particular today and try to get back to everyone on this scheduler topic by the end of today or tomorrow. Thanks for your time and again I apologize for the lack of communication on this topic. - Joshua ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, Mar 12, Ian Campbell wrote: dist/install/var/xen/dump which all seems proper and correct to me. Except the last one, which should be /var/lib/xen/dump or whatever dumpdir the OS/FHS provides. Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, 2015-03-12 at 15:07 +, Ian Campbell wrote: On Thu, 2015-03-12 at 10:21 +, wei.l...@citrix.com wrote: * Repurpose SEDF Scheduler for Real-time (fair) RFC patch posted (v2) - Joshua Whitehead, Robert VanVossen This was superceded by the RTDS stuff, wasn't it? I haven't head from Joshua and Robert in a while, so I don't really know whether they're still working on this, but I think they're not. And yes, we have a new and modern real-time scheduling now, so I would rather direct all the effort toward it (to move it out from 'experimental' status), and start thinking at ways to deprecate/get rid of SEDF. Regards, Dario signature.asc Description: This is a digitally signed message part ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
All, I expect to have a patch out soon for the RTDS scheduler improvement. Regards, Dagaen Golomb On Thu, Mar 12, 2015 at 12:01 PM, Olaf Hering o...@aepfle.de wrote: On Thu, Mar 12, Ian Campbell wrote: dist/install/var/xen/dump which all seems proper and correct to me. Except the last one, which should be /var/lib/xen/dump or whatever dumpdir the OS/FHS provides. Olaf ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, Mar 12, 2015 at 03:07:33PM +, Ian Campbell wrote: On Thu, 2015-03-12 at 10:21 +, wei.l...@citrix.com wrote: [...] * Rearrange and cleanup installation destination directories (/var - var/lib/xen) (fair) - Daniel Kiper What is this? I've never heard about it. My local tree has these after make dist: dist/install/var dist/install/var/lib dist/install/var/lib/xen dist/install/var/lib/xenstored dist/install/var/lib/xen/xenpaging dist/install/var/log dist/install/var/log/xen dist/install/var/xen dist/install/var/xen/dump which all seems proper and correct to me. IIRC, 4.3 release (and probably earlier ones) was a mess. Olaf did the work for 4.5. However, as he pointed out in another email last one is incorrect. Olaf, are you going to fix it for 4.6? I suppose it is quite simple. Daniel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
2015-03-12 11:39 GMT-04:00 Dario Faggioli dario.faggi...@citrix.com: On Thu, 2015-03-12 at 15:07 +, Ian Campbell wrote: On Thu, 2015-03-12 at 10:21 +, wei.l...@citrix.com wrote: * Repurpose SEDF Scheduler for Real-time (fair) RFC patch posted (v2) - Joshua Whitehead, Robert VanVossen This was superceded by the RTDS stuff, wasn't it? I haven't head from Joshua and Robert in a while, so I don't really know whether they're still working on this, but I think they're not. And yes, we have a new and modern real-time scheduling now, so I would rather direct all the effort toward it (to move it out from 'experimental' status), and start thinking at ways to deprecate/get rid of SEDF. Dagaen, Chong and I are working on improving the RTDS scheduler right now. Dagaen will change it from quantum driven to timer based scheduler; Chong will extend the xl toolstack to support per-vcpu setting/getting based on our old patch set. I'm working on eliminating the scheduler overhead for the dedicated VCPU (A VCPU is dedicated VCPU if it has full capacity, and its hard affinity has only one CPU and user choose to set it as dedicated.). Once a VCPU is dedicated to a CPU, no other VCPUs will run on that CPU. So there is no point to trigger the scheduler on that CPU, which cause around 1500cycles schedule() overhead. I'm working on the code and then evaluating its performance. Once I can confirm this will definitely benefit the RTDS scheduler, I will send a separate email to the ML to explain the goal/design/implementation in detail. Thanks, Meng -- --- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
-Original Message- From: wei.l...@citrix.com [mailto:wei.l...@citrix.com] * extending mem_access support to PV domain (fair) RFC v2 - Aravindh Puthiyaparambil (aravindp) We did some internal reprioritizing and decided to focus on HVM and PVH domains. This can be placed in deep freeze for now. Thanks, Aravindh ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, Mar 12, 2015 at 10:21:56AM +, wei.l...@citrix.com wrote: [...] == GRUB2 == * GRUB2 multiboot2 (fair) - Daniel Kiper RFC patches were posted (see: http://lists.xen.org/archives/html/xen-devel/2015-01/msg03982.html). Weeding out bugs found during testing. I am going to post new patch series at the beginning of April. [...] == Hypervisor == [...] * Xen Boot Information (xbi) (ok) Dependency for GRUB2 + EFI work http://lists.xen.org/archives/html/xen-devel/2014-10/msg02068.html v4, No go for full patchset. Only some of the patches. No ARM EFI hardware (yet) available to test them. - Daniel Kiper Move this to after 4.6. It depends on multiboot2 protocol support. * Xen multiboot2-EFI support (fair) Needed for GrUB2 I am not sure what does it mean. Please drop this line above. Depends on Xen Boot info (rework multiboot and other structs) Not valid right now. Please drop this line above. See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html RFC posted - Daniel Kiper RFC patches were posted (see: http://lists.xen.org/archives/html/xen-devel/2015-01/msg03962.html). Weeding out bugs found during testing. I am going to post new patch series at the beginning of April. [...] == Xen toolstack == [...] * Rearrange and cleanup installation destination directories (/var - var/lib/xen) (fair) - Daniel Kiper If Olaf did all work then we can remove this one. * libxl/xl - xm compatibility mode for mem-max and mem-set; (ok) - Daniel Kiper Bob and I will be working on this. However, this probably will not come into 4.6. So, let's move this to after 4.6. Daniel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
* Clean-up of mem-event subsystem (good) v5 posted - Tamas K Lengyel v7 will be posted today, status is good. === Hypervisor ARM === I've posted v13 of the mem_access for ARM series that should be added to this tracker. Status should be good. Thanks, Tamas ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On Thu, 2015-03-12 at 10:54 +, George Dunlap wrote: On 03/12/2015 10:21 AM, wei.l...@citrix.com wrote: * Credit2: introduce per-vcpu hard and soft affinity (fair) - Justin T. Weaver The most recent patches looked pretty good -- I'd be very surprised if these didn't make it in by July. I'd change this to good. +1 * Default to credit2 (none) cpu pinning, numa affinity and cpu reservation - George Dunlap I think before actually doing a release with credit2 as the default, we want almost an entire development cycle with credit2 as the default, to shake out any latent bugs; Indeed. We also want to discuss and define some criteria/requirement for a scheduler to fulfill in order to be considered as the new default one. We can work on this as well, during this dev cycle, and have it ready for next time, if we want to try doing the switch. and probably an entire release with credit2 listed as production-ready. I agree. We want to make it no longer experimental, and that is probably doable within this dev cycle. So maybe this goal would be more helpfully stated as credit2 production ready, so that when we open the next development window we can change the default immediately? +1 Regards, Dario signature.asc Description: This is a digitally signed message part ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Xen 4.6 Development Update (two months reminder)
On 12.03.15 at 11:21, wei.l...@citrix.com wrote: == Linux == Wouldn't it make sense to move external projects down, and have our core pieces (hypervisor, tool stack, maybe qemu) be near the top? * Convert tasklet to per-cpu tasklets (fair) RFC posted - Konrad Rzeszutek Wilk Is this still a valid item? I think we went another route, albeit that work is still incomplete iirc. Konrad? * Further tmem cleanups/fixes (16TB etc) (fair) - Bob Liu As we can now support more than 16Tb, I don't think this number should be mentioned anymore. * amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch) Which one? * Feature masking MSR support (enhancement) (in master) What is this about? Or what status does in master actually refer to? * Support controlling the max C-state sub-state (fair) v3 posted Hadn't see the patch reposted. - Ross Lagerwall I'm afraid we have a hard time settling on a reasonable model for specifying the maximum sub-state. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
