[Bug 1633207] Re: VM fails to start with dac security driver added

[Christian Ehrhardt ]

Local creation with a apparmor seclabel fails the same as the migration,
so we can ignore all "migration specials".

To test that add:


to /usr/share/uvtool/libvirt/template.xml
And run uvt-kvm create again

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[Launchpad Bug Tracker]

This bug was fixed in the package libvirt - 2.1.0-1ubuntu9.1

---
libvirt (2.1.0-1ubuntu9.1) yakkety; urgency=medium

  * d/p/u/apparmor-fix-other-seclabels.patch fixes an issue parsing non
apparmor security labels (LP: #1633207).

 -- Christian Ehrhardt   Thu, 01 Dec
2016 09:44:12 +0100

** Changed in: libvirt (Ubuntu Yakkety)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Since there was no reply to verify in a week I felt I had to clear the
queue and tested it myself again.

@bugproxy: In general - a.k.a. for next time - I'd really like to have
3rd party verification. Not to save me the 15 minutes, but to make sure
it really addresses your issue and get further verification if anything
else was broken by the SRU. If it doesn't fit with your current plans
that is fine, but then let me know an estimate when you expect you get
to it.

** Tags removed: verification-needed
** Tags added: verification-done

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[Brian Murray]

Hello bugproxy, or anyone else affected,

Accepted libvirt into yakkety-proposed. The package will build now and
be available at
https://launchpad.net/ubuntu/+source/libvirt/2.1.0-1ubuntu9.1 in a few
hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: libvirt (Ubuntu Yakkety)
   Status: Triaged => Fix Committed

** Tags added: verification-needed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Prepared SRU Template and Uploaded into the (Y) SRU review queue.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

** Description changed:

+ [Impact]
+ 
+  * Due to an upstream change in libvirt 2.0 users of libvirt >=2.0
+(that is >=Yakkety) can't use non apparmor security labels anymore.
+ 
+  * That means old guest definitions that should still work fail to start 
+now
+ 
+  * The issue was in virt-aa-helper, the proposed fix was tested and then 
+brought upstream. This is a backport of the upstream accepted fix.
+ 
+ [Test Case]
+ 
+  * Testcase with virt-aa-helper on a minimal xml:
+   $ cat << EOF > /tmp/test.xml
+   
+ test-seclabel
+ 12345678-9abc-def1-2345-6789abcdef00
+ 1
+ hvm
+ 
+ 
+   
+   EOF
+   $ /usr/lib/libvirt/virt-aa-helper -d -r -p 0 \
+ -u libvirt-12345678-9abc-def1-2345-6789abcdef00 < /tmp/test.xml
+ 
+   Current Result:
+ virt-aa-helper: error: could not parse XML
+ virt-aa-helper: error: could not get VM definition
+   Expected Result is to emit a valid apparmor profile
+ 
+ * The more complex test is to create a guest (whatever way you like) and 
+   add an empty dac security label (as shown above) to then start the 
+   guest.
+   
+   Current Result:
+ error: Failed to start domain yakkety-doubleseclabel
+ error: internal error: cannot load AppArmor profile 
'libvirt-8746b00d-aad1-4346-8784-2d4331465153'
+   Expected Result:
+ properly staring the guest
+ 
+ [Regression Potential]
+ 
+  * The change is in the parsing of domain info in domain.conf. While no 
+local nor upstream tests broke anything one could think of very special
+xml configuation that now might fail parsing. OTOH the new change now 
+skips some of the parsing, so even if we miss to consider something it 
+shouldn't fail, but instead "forget" to read some data correctly. The 
+part that we skip are seclabels which are created dynamically anyway.
+ 
+  * Also the changed flag is local to virt-aa-helper.c so and guarded by 
+that flag in domain_conf.c so it should be a no-op to anybody but virt-
+aa-helper for sure.
+ 
+ [Other Info]
+  
+  * Anything else you think is useful to include
+  * Anticipate questions from users, SRU, +1 maintenance, security teams and 
the Technical Board
+  * and address these questions in advance
+ 
+ 
  ---Problem Description---
  VM fails to start with dac security driver added
-   
+ 
  ---uname output---
  Linux ltc-test-ci1 4.4.0-9136-generic #55-Ubuntu SMP Fri Aug 26 05:56:24 UTC 
2016 ppc64le ppc64le ppc64le GNU/Linux
-  
- Machine Type = power 8 ppc64le 
-   
+ 
+ Machine Type = power 8 ppc64le
+ 
  ---Steps to Reproduce---
-  
+ 
  VM fails to start with dac security driver added
- 1. Define a VM with both apparmor and dac security driver( Used XML as below) 
+ 1. Define a VM with both apparmor and dac security driver( Used XML as below)
  #virsh dumpxml virt-tests-vm1
  
-   virt-tests-vm1
-   0491f0cd-eb14-4992-be4c-53a1adf1d314
-   33554432
-   33554432
-   32
-   
- /machine
-   
-   
- hvm
- 
-   
-   
- 
-   
-   
-   destroy
-   restart
-   restart
-   
- /usr/bin/kvm
- 
-   
-   
-   
-   
- 
- 
-   
- 
- 
- 
-   
- 
- 
-   
-   
-   
-   
- 
- 
-   
-   
- 
- 
-   
-   
- 
- 
-   
- 
- 
-   
-   
-   
+   virt-tests-vm1
+   0491f0cd-eb14-4992-be4c-53a1adf1d314
+   33554432
+   33554432
+   32
+   
+ /machine
+   
+   
+ hvm
+ 
+   
+   
+ 
+   
+   
+   destroy
+   restart
+   restart
+   
+ /usr/bin/kvm
+ 
+   
+   
+   
+   
+ 
+ 
+   
+ 
+ 
+ 
+   
+ 
+ 
+   
+   
+   
+   
+ 
+ 
+   
+   
+ 
+ 
+   
+   
+ 
+ 
+   
+ 
+ 
+   
+   
+   
  
  
  2. virsh start virt-tests-vm1
  #virsh start virt-tests-vm1
  error: Failed to start domain virt-tests-vm1
  error: internal error: cannot load AppArmor profile 
'libvirt-0491f0cd-eb14-4992-be4c-53a1adf1d314'--NOK
  
- 
  3. After removing dac line from xml() VM started fine
  #virsh start virt-tests-vm1
  Domain virt-tests-vm1 started
  
+ Userspace tool common name: ii  libvirt-bin
+ 2.1.0-1ubuntu5  ppc64el  programs for the
+ libvirt library
  
-  
- Userspace tool common name: ii  libvirt-bin  
2.1.0-1ubuntu5  ppc64el  programs for the libvirt 
library 
-  
- The userspace tool has the following bit modes: both 
+ The userspace tool has the following bit modes: both
  
  Userspace package: ii  libvirt-bin
  2.1.0-1ubuntu5  ppc64el  programs for the
  libvirt library

** Description changed:

  [Impact]
  
-  * Due to an upstream change in libvirt 2.0 users of libvirt >=2.0
-(that is >=Yakkety) can't use non apparmor security labels anymore.
+  * Due to an upstream change in libvirt 2.0 users of libvirt >=2.0
+    (that is >=Yakkety) can't use non apparmor 

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

The refreshed upstream accepted fix is now available in Zesty as 2.1.0-1ubuntu14
With that ready now preparing the SRU into Yakkety.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

FYI - The backport SRU to Yakkety will have to wait until we have a
upstream accepted solution.

** Also affects: libvirt (Ubuntu Yakkety)
   Importance: Undecided
   Status: New

** Changed in: libvirt (Ubuntu Yakkety)
   Status: New => Triaged

** Changed in: libvirt (Ubuntu Yakkety)
   Importance: Undecided => Low

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[Launchpad Bug Tracker]

This bug was fixed in the package libvirt - 2.1.0-1ubuntu13

---
libvirt (2.1.0-1ubuntu13) zesty; urgency=medium

  * drop d/p/ubuntu/fix-ftbfs-for-gnutls-3-5-6.patch as the offending change
in gnutls has been reverted (LP: #1641615)
  * Build depend on gnutls >= 3.5.6-4ubuntu2 to build after the gnutls fix
migrated

 -- Christian Ehrhardt   Thu, 17 Nov
2016 08:43:10 +0100

** Changed in: libvirt (Ubuntu)
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

FYI - v2 of the patch in discussion upstream
https://www.redhat.com/archives/libvir-list/2016-November/msg00991.html

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

What worked last week doesn't have to this week - I ran into an FTBFS -
please wait a bit until resolved.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Unfortunately upstream response is super slow on this.
I think the patch is right and therefore I'm willing to put it into zesty as 
being a dev release in development.
That will also give us more coverage if there is anything we might have missed.


That said pushed it to Zesty now the way it was tested.by me and the reporter.

Since it is not an issue for Xenial there is not SRU need there but for Yakkety 
I'd only consider an SRU once upstream discussion settled and accepted it.
That said @IBM - if you want to request an SRU on this into Yakkety I'd ask you 
to join the upstream discussion on libvirt to give it some weight by a third 
party pushing for it.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

** Changed in: libvirt (Ubuntu)
   Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Thank you a lot for verifying the ppa.

Since this isn't critically urgent I'll wait with a fix upload to the
package until the upstream discussion settled (better than to revert in
two weeks again).

Once https://www.redhat.com/archives/libvir-list/2016-October/msg01297.html
followed in November by thread 
https://www.redhat.com/archives/libvir-list/2016-November/msg00229.html 
conclude I'll go forward on this.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

FYI discussion started at https://www.redhat.com/archives/libvir-
list/2016-October/msg01297.html

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Thanks a lot Guido for your feedback - it helped me better "reading the
case".

I see the same issue throughout latest libvirt upstream as of today.
So I'm gonna submit the fix upstream for discussion as I could easily overlook 
something here.
E.g. parseOpaque is quite close as it is passed up to virDomainDefParseXML, but 
I think that would be even more misuse than a new flag.

If accepted there (one way or the other) I intend to create a diff to
upload for latest Debian and Ubuntu and consider SRUs from there.

The created domain while active has both seclabels and valid content in them 
just as it had back on libvirt 1.3:
[...]
  
libvirt-956134c4-d91d-417e-b68f-1d8d492419d6
libvirt-956134c4-d91d-417e-b68f-1d8d492419d6
  
  
+112:+116
+112:+116
  

@AGX - I'll set you on cc on that upstream discussion.

@IBM - I have a new version (2.1.0-1ubuntu10~ppa5) in the ppa that works
for me, It would be great if you could verify this one for you as well.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[Guido Günther]

dfbc9a83 was necessary since libvirt changed the paths of the monitor
socket in a89f05ba8df095875f5ec8a9065a585af63a010b. We had to switch
from VIR_DOMAIN_DEF_PARSE_INACTIVE to active since we need the domain id
(ctl->def->id) as it is part of the socket path now. It would probably
o.k. to skip validation but we need to parse the active domain config to
get the id.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

I subscribed agx, the author of the conflicting patch upstream.

Questions:
agx - Could you please comment on my finding?
agx - Please describe your case that let you write dfbc9a83?
IBM - I don't think it helps yet, but if you can please try to verify the ppa I 
provide at https://launchpad.net/~paelzer/+archive/ubuntu/libvirt-bug-1633207

I'm available on freenode e.g. in #ubuntu-server as cpaelzer.
This likely is an extended weekend for both of us, but please feel free to try 
to catch me there.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

I made an experimental fix available at
https://launchpad.net/~paelzer/+archive/ubuntu/libvirt-bug-1633207

In the pure aa-helper tests that continues to work with all my usual minor 
tests and it fixes dac and dac+apparmor label issues.
But I seem to run into issues with doing full guests:
qemu-system-x86_64: -object 
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-yakkety-sec-testfix/master-key.aes:
 Unable to read 
/var/lib/libvirt/qemu/domain-2-yakkety-sec-testfix/master-key.aes: Failed to 
open file '/var/lib/libvirt/qemu/domain-2-yakkety-sec-testfix/master-key.aes': 
Permission denie

That is due to an apparmor deny and might be that this was the issue
that was tried to be fixed with the breaking change in libvirt?

I'll run more tests on it on my own.
But I'd really like to coordinate with the author of the former change what the 
test case was that made him create the patch.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

TL;DR:
- a dac sec label is parsed
- it has no label, but due to a bug it searches one
- label can't be found for an inactive domain
- exit with Error
- expected fix is reverting part of dfbc9a83

Debug-Analysis:

Interesting part of the call chain:
get_definition -> virDomainDefParseString -> virDomainDefParse -> 
virDomainDefParseNode -> virDomainDefParseXML -> virSecurityLabelDefsParseXML 
-> virSecurityLabelDefParseXML

Compiled -O0 -g to see more to see where it is failing.
The code itself (of that failing function) didn't change since 1.3.1 (Xenial).

gdb ~/libvirt-2.1.0/debian/tmp/usr/lib/libvirt/virt-aa-helper
set env LD_LIBRARY_PATH 
/home/ubuntu/libvirt-2.1.0/debian/tmp/usr/lib/x86_64-linux-gnu/
set solib-search-path 
/home/ubuntu/libvirt-2.1.0/debian/tmp/usr/lib/x86_64-linux-gnu/
b virSecurityLabelDefsParseXML
run -d -r -p 0 -u libvirt-6e082f89-902c-413c-9d9e-f609089d3374 < 
yakkety-sec-dac.xml

virSecurityLabelDefParseXML (ctxt=0x557ddaf0, flags=1024) at 
../../../src/conf/domain_conf.c:6384
n (number of labels) is 1
single def parse in virSecurityLabelDefParseXML
1. type dynamic = VIR_DOMAIN_SECLABEL_DYNAMIC
2. relabel yes
3-5 useless if/jumps
6. fails at parsing the actual label
   it doesn't find a label, but thinks it needs one
   check:
   6.1 seclabel->type == VIR_DOMAIN_SECLABEL_STATIC   => it is not
   6.2 !(flags & VIR_DOMAIN_DEF_PARSE_INACTIVE) &&=> true
   6.3 seclabel->type != VIR_DOMAIN_SECLABEL_NONE => true
=> There is no label for the currently off machine, so it fails to find one and 
goes to error path

The function does right, but the flags suggest it would be alive.
Definiton:
/* Parse only parts of the XML that would be present in an inactive libvirt
 * XML. Note that the flag does not imply that ABI incompatible
 * transformations can be used, since it's used to strip runtime info when
 * restoring save images/migration. */
VIR_DOMAIN_DEF_PARSE_INACTIVE= 1 << 1,

The flag comes from the first in the call chain "get_definition"
   ctl->def = virDomainDefParseString(xmlStr, ctl->caps, ctl->xmlopt, 
VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE);

That exactly is a diff of the Ubuntu versions on that call:
 ctl->def = virDomainDefParseString(xmlStr,
ctl->caps, ctl->xmlopt,
-   VIR_DOMAIN_DEF_PARSE_INACTIVE);
+   VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE);

Almost all other changes do OR it in:
-int domainflags = VIR_DOMAIN_DEF_PARSE_INACTIVE;
+int domainflags = VIR_DOMAIN_DEF_PARSE_INACTIVE |
+  VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE;

Check upstream for the reasons:

commit b394af162a3871575d9f9c28f72331f198aafa25
Author: Peter Krempa 
Date:   Thu May 26 15:58:53 2016 +0200
conf: Add infrastructure for adding configuration validation

On the critical place before there was a 0, so setting fix was like ORing in.
Why was there a 0 and not VIR_DOMAIN_DEF_PARSE_INACTIVE like in the past?

That was the reason there was a 0 before b394af16 came in:
commit dfbc9a8382adc0495bf0e034ae6add92bed4822b
Author: Guido Günther 
Date:   Sat Apr 2 12:49:28 2016 +0200
apparmor: QEMU monitor socket moved

That changed the call from VIR_DOMAIN_DEF_PARSE_INACTIVE to 0 for issues
starting with apparmor but provides no further detail.


The patch to fix would be as easy as:
--- libvirt-2.1.0.orig/src/security/virt-aa-helper.c
+++ libvirt-2.1.0/src/security/virt-aa-helper.c
@@ -708,6 +708,7 @@ get_definition(vahControl * ctl, const c
 
 ctl->def = virDomainDefParseString(xmlStr,
ctl->caps, ctl->xmlopt,
+   VIR_DOMAIN_DEF_PARSE_INACTIVE |
VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE);
 
 if (ctl->def == NULL) {

I checked some related cases on apparmor instead of dac labels:
- if dumpxml runs on an running instance with apparmor labels it adds the label 
to the output, so next load works as it can find it
- if a uuid is not yet defined it creates new labels and works
- if a uuid is defined, but no lavel in xml aa-helper fails on apparmor 
seclabels with the same issue (fixed by the same patch)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Again at:
sudo virsh start yakkety-doubleseclabel
error: Failed to start domain yakkety-doubleseclabel
error: internal error: cannot load AppArmor profile 
'libvirt-8746b00d-aad1-4346-8784-2d4331465153'

In the log I found the related:
Okt 27 13:45:50 horsea libvirtd[10370]: internal error: Child process 
(LIBVIRT_LOG_OUTPUTS=3:stderr /usr/lib/libvirt/virt-aa-helper -p 0 -r -u 
libvirt-8746b00d-aad1-4346-8784-2d4331465153) unexpected exit status 1: 
2016-10-27 13:45:20.873+: 10640: info : libvirt version: 2.1.0, package: 
1ubuntu10~ppa3 (Christian Ehrhardt  Mon, 24 
Oct 2016 14:21:36 +0200)
2016-10-27 13:45:20.873+: 10640: 
info : hostname: horsea
2016-10-27 13:45:20.873+: 10640: 
error : virSecurityLabelDefParseXML:6473 : XML error: security label is missing
virt-aa-helper: error: could not parse 
XML
virt-aa-helper: error: could not get VM 
definition
Okt 27 13:45:50 horsea libvirtd[10370]: internal error: cannot load AppArmor 
profile 'libvirt-8746b00d-aad1-4346-8784-2d4331465153'
Okt 27 13:45:50 horsea virtlogd[7706]: End of file while reading data: 
Input/output error

I also found that adding dac alone is enough to trigger:

$ virsh dumpxml yakkety-doubleseclabel | grep -A 20 ' Failing

$ virsh dumpxml yakkety-sec-app | grep -A 20 seclabel
  

=> Working

$ virsh dumpxml yakkety-sec-dac | grep -A 20 seclabel
  

=> Failing just as much as case 1, maybe because apparmor is default on.

Trying to check the /usr/lib/libvirt/virt-aa-helper in those cases, but
since it is not meant to be called directly that is a bit tricky.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

After a bit of twiddling I found a somewhat reasonable repro with the
virt-aa-helper tool.

diff -Naur yakkety-sec-dac.xml yakkety-sec-nodac.xml
--- yakkety-sec-dac.xml 2016-10-27 14:32:39.565995840 +
+++ yakkety-sec-nodac.xml   2016-10-27 14:32:45.097973456 +
@@ -60,6 +60,5 @@
   
 
   
-  
 

So the only diff is if the dac seclabel is here or not.

$ sudo /usr/lib/libvirt/virt-aa-helper -d -r -p 0 -u 
libvirt-6e082f89-902c-413c-9d9e-f609089d3374 < yakkety-sec-dac.xml 
virt-aa-helper: error: could not parse XML
virt-aa-helper: error: could not get VM definition

$ sudo /usr/lib/libvirt/virt-aa-helper -d -r -p 0 -u 
libvirt-6e082f89-902c-413c-9d9e-f609089d3374 < yakkety-sec-nodac.xml 
virt-aa-helper:
/etc/apparmor.d/libvirt/libvirt-6e082f89-902c-413c-9d9e-f609089d3374.files
virt-aa-helper:
  "/var/log/libvirt/**/yakkety-sec-dac.log" w,
  "/var/lib/libvirt/qemu/domain-yakkety-sec-dac/monitor.sock" rw,
  "/var/lib/libvirt/qemu/domain--1-yakkety-sec-dac/*" rw,
  "/var/lib/libvirt/qemu/channel/target/domain--1-yakkety-sec-dac/*" rw,
  "/var/run/libvirt/**/yakkety-sec-dac.pid" rwk,
  "/run/libvirt/**/yakkety-sec-dac.pid" rwk,
  "/var/run/libvirt/**/*.tunnelmigrate.dest.yakkety-sec-dac" rw,
  "/run/libvirt/**/*.tunnelmigrate.dest.yakkety-sec-dac" rw,
  "/var/lib/uvtool/libvirt/images/yakkety-sec-dac.qcow" rw,
  
"/var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZC5kYWlseTpzZXJ2ZXI6MTYuMTA6YW1kNjQgMjAxNjEwMjI="
 r,
  "/var/lib/uvtool/libvirt/images/yakkety-sec-dac-ds.qcow" rw,
  # for qemu guest agent channel
  owner "/var/lib/libvirt/qemu/channel/target/domain-yakkety-sec-dac/**" rw,
  /dev/vhost-net rw,

New running debuild locally on xenial and yakkety libvirt to have the
packaged aa-helper in a debuggable and recompilable fashion.

** Changed in: libvirt (Ubuntu)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Once more confirmed that it worked in Xenial - adding regression-release

** Tags added: regression-release

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Ok, I found why those templates didn't get on my BM test system to begin with.
They were conffiles and I had none of them modified, but some more in the same 
directories.
So while not that clear still the usual "protect custom conffiles" mechanism 
that blocked me.
A full purge + manual extra conffile clean + re-install made it working again.
Overall feels a bit touchy atm :-/

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Three way check on fresh installs:
dpkg -S $((find /etc/apparmor.d/ -name '*libvirt*' && find /etc/apparmor.d/ 
-name '*TEMPLATE*' )| xargs) | sort
X   Y   
X-Y upgrade
libvirt-bin: /etc/apparmor.d/abstractions/libvirt-lxc   
libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-lxc 
libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-lxc
libvirt-bin: /etc/apparmor.d/abstractions/libvirt-qemu  
libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-qemu
libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-qemu
libvirt-bin: /etc/apparmor.d/libvirt
libvirt-daemon-system: /etc/apparmor.d/libvirt  
libvirt-daemon-system: /etc/apparmor.d/libvirt
libvirt-bin: /etc/apparmor.d/libvirt/TEMPLATE.lxc   
libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.lxc 
libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.lxc
libvirt-bin: /etc/apparmor.d/libvirt/TEMPLATE.qemu  
libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.qemu
libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.qemu

libvirt-daemon-system: /etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper 
libvirt-daemon-system: /etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper
libvirt-bin: /etc/apparmor.d/local/usr.sbin.libvirtd
libvirt-daemon-system: /etc/apparmor.d/local/usr.sbin.libvirtd  
libvirt-daemon-system: /etc/apparmor.d/local/usr.sbin.libvirtd
libvirt-bin: /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper 
libvirt-daemon-system: /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper   
libvirt-daemon-system: /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
libvirt-bin: /etc/apparmor.d/usr.sbin.libvirtd  
libvirt-daemon-system: /etc/apparmor.d/usr.sbin.libvirtd
libvirt-daemon-system: /etc/apparmor.d/usr.sbin.libvirtd

In this case Y and the X-Y upgrade was equal.
Also the formerly missing TEMPLATE files were here.
Note that I still have that cae on my phys box - no matter how often I 
reinstall.

Checking content between X and Y:
Equal:
5f6aa836ced6b474dabfce46a8bfb5e4  /etc/apparmor.d/libvirt/TEMPLATE.lxc
b0dfa704c6297fd9a4e68f0137c6be88  /etc/apparmor.d/libvirt/TEMPLATE.qemu
7166fa490aaf905b7f71cb5407ef0696  /etc/apparmor.d/local/usr.sbin.libvirtd
No functional diff (only comments/reordering):
/etc/apparmor.d/abstractions/libvirt-lxc
/etc/apparmor.d/abstractions/libvirt-qemu
/etc/apparmor.d/usr.lib.libvirt.virt-aa-helper
New but non-functional (empty to carry overwrites)
/etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper
Changed:
/etc/apparmor.d/usr.sbin.libvirtd added "/usr/sbin/virtlogd pix"

Here the easy in container test stops, as the apparmor security driver really 
can't work in there.
I need to go back to my BM system and understand/fix why it does run into 
"error: unsupported configuration: Unable to find security driver for model 
apparmor" now.
While only partially related I still wanted to document here to find it later 
if needed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

While debugging I found the first level of oddities that I'll continue
on and hopefully gives us a solution (or at least eliminate one
roadblock).

I think I found that things work with the error described in the bug on 
Xenial->Yakkety upgraded systems. But on all others I see:
error: unsupported configuration: Unable to find security driver for model 
apparmor

That would explain the reproducibility fuzz a bit.

After realizing that I checked logs:
 internal error: template '/etc/apparmor.d/libvirt/TEMPLATE.qemu' does not exist
 internal error: template '/etc/apparmor.d/libvirt/TEMPLATE.qemu' does not exist
 unsupported configuration: Security driver apparmor not enabled
 internal error: template '/etc/apparmor.d/libvirt/TEMPLATE.qemu' does not exist

Now checking for those files is even more strange.

$ dpkg -S /etc/apparmor.d/libvirt/TEMPLATE.qemu
libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.qemu
sudo apt-get install --reinstall libvirt-daemon-system
ll /etc/apparmor.d/libvirt/TEMPLATE.qemu
ls: cannot access '/etc/apparmor.d/libvirt/TEMPLATE.qemu': No such file or 
directory

I guess we have those things here:
1. no proper handling of conffile changes due to the switch to the upstream 
provided apparmor profiles
2. on upgraded systems old somehow conflict
3. on new Yakkety apparmor seclabel doesn't work at all

Going on with debugging tomorrow.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

I realized that part of my former verification was caused by the kvm-in-
lxd env I use to avoid needing too much metal. So I retried on x86 again
as these code paths shouldn't be arch specific at all. And now I was
able to recreate on x86 as well.

The summary looks like this now:
* - xenial - works
ppc4el - yakkety - fail
x86 - yakkety - fail
x86 - yakkety with 4.4 kernel - fail

Going on with debugging, but I'm on the Road the next few days - so it
might take a bit unless someone else jumps in.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Ha - got my container trick working again.
So testing on Yakkety, adding the double seclabel.

Finally - able to reproduce - yeah!
Looking deeper into that now...

** Changed in: libvirt (Ubuntu)
   Status: Incomplete => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

2nd level kvm failed me as well :-/

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

I have to report that my usual trick to run KVM from inside a container
doesn't work the same way on ppc64el. It might take a while for me to
get a Yakkety ppc64el BM system, so more than before I'm dependent on
you reporting the extended logs as I requested.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Thanks satheera for the reply.
I wonder why it works for me than as I explicitly tested ppc as well just as 
you do ... ?

It works fine on x86 with Yakkety.
As well as fine on ppc64el with Xenial.
I don't have a Yakkety around yet and machines are scarce.

I assume the xml is how avocado creates it for you.
I compared our libvirt xml files and tried to remove any remaining delta.
That changes from mine to yours were:
+ resource partition
+ adding topology
- cpu features
+ on crash destroy -> restart
+ adding spapr-vio scsi controller

Still starting fine.

>From you it would be great if you could enable debugging for libvirt
service and virsh, run the failing start of the guest again and report
the qemu log file and libvirt from journalctl here. See
https://libvirt.org/logging.html for more.

It seems I have to try getting a ppc64el on Yakkety for this test next
... working on that ...

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1633207] Re: VM fails to start with dac security driver added

[ChristianEhrhardt]

Hi,
I tested a simple guest as created with uvt-kvm:
$ uvt-kvm create --memory 2048 --cpu 4 --password=ubuntu 
paelzer-yakkety-test-libvirt release=yakkety arch=ppc64el label=daily
plus the two lines:
  
  

That works on:
Xenial: ok
Yakkety: ok

I did the same on ppc64el, but only had a Xenial host available there.
Yet this worked just fine as well.

Quoting the report: "Linux ltc-test-ci1 4.4.0-9136-generic #55-Ubuntu SMP Fri 
Aug 26 05:56:24"
Since that seems to be a pre-release yakkety, could I ask you to retest with at 
least the released levels and report the versions of qemu involved for 
you (dpkg -l '*qemu*' '*libvirt*')?

** Changed in: libvirt (Ubuntu)
   Status: New => Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207

Title:
  VM fails to start with dac security driver added

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs