[lxc-devel] [lxc-ci/master] jenkins/jobs: Update Apertis releases

2020-10-08 Thread monstermunchkin on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxc-ci/pull/193

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===
Signed-off-by: Thomas Hipp 
From 054ff0d6217bd82df3ea4c1353c2f075a490635d Mon Sep 17 00:00:00 2001
From: Thomas Hipp 
Date: Tue, 6 Oct 2020 12:01:30 +0200
Subject: [PATCH] jenkins/jobs: Update Apertis releases

Signed-off-by: Thomas Hipp 
---
 jenkins/jobs/image-apertis.yaml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/jenkins/jobs/image-apertis.yaml b/jenkins/jobs/image-apertis.yaml
index 7a4748a..1202851 100644
--- a/jenkins/jobs/image-apertis.yaml
+++ b/jenkins/jobs/image-apertis.yaml
@@ -18,8 +18,8 @@
 name: release
 type: user-defined
 values:
-- 'v2019.3'
-- 'v2020.1'
+- 'v2019.4'
+- 'v2020.2'
 
 - axis:
 name: variant
___
lxc-devel mailing list
lxc-devel@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-devel


[lxc-devel] [lxd/master] lxd/instance/drivers: Add USB controller to VMs

2020-10-08 Thread monstermunchkin on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxd/pull/7998

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===

From 87e44ddcf8dfaa843b7375601aba550182217020 Mon Sep 17 00:00:00 2001
From: Thomas Hipp 
Date: Thu, 8 Oct 2020 09:22:56 +0200
Subject: [PATCH 1/2] lxd/instance/drivers: Enable USB for VMs

This enables USB for VMs.

Signed-off-by: Thomas Hipp 
---
 lxd/instance/drivers/driver_qemu_templates.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lxd/instance/drivers/driver_qemu_templates.go 
b/lxd/instance/drivers/driver_qemu_templates.go
index 6950919840..2e825176b0 100644
--- a/lxd/instance/drivers/driver_qemu_templates.go
+++ b/lxd/instance/drivers/driver_qemu_templates.go
@@ -23,7 +23,7 @@ type = "pseries"
 type = "s390-ccw-virtio"
 {{end -}}
 accel = "kvm"
-usb = "off"
+usb = "on"
 graphics = "off"
 
 {{if eq .architecture "x86_64" -}}

From a6c11f1114c67cb1c18a2c9ba83a5feeb92e5820 Mon Sep 17 00:00:00 2001
From: Thomas Hipp 
Date: Thu, 8 Oct 2020 10:06:49 +0200
Subject: [PATCH 2/2] lxd/instance/drivers: Add USB controller to QEMU config

This adds a USB controller to the QEMU config.

Signed-off-by: Thomas Hipp 
---
 lxd/instance/drivers/driver_qemu.go   | 11 +++
 lxd/instance/drivers/driver_qemu_templates.go | 11 +++
 2 files changed, 22 insertions(+)

diff --git a/lxd/instance/drivers/driver_qemu.go 
b/lxd/instance/drivers/driver_qemu.go
index 62c1de7117..0b697cbf58 100644
--- a/lxd/instance/drivers/driver_qemu.go
+++ b/lxd/instance/drivers/driver_qemu.go
@@ -1749,6 +1749,17 @@ func (vm *qemu) generateQemuConfigFile(busName string, 
devConfs []*deviceConfig.
return "", err
}
 
+   devBus, devAddr, multi = bus.allocate(busFunctionGroupGeneric)
+   err = qemuUSB.Execute(sb, map[string]interface{}{
+   "bus":   bus.name,
+   "devBus":devBus,
+   "devAddr":   devAddr,
+   "multifunction": multi,
+   })
+   if err != nil {
+   return "", err
+   }
+
devBus, devAddr, multi = bus.allocate(busFunctionGroupNone)
err = qemuSCSI.Execute(sb, map[string]interface{}{
"bus":   bus.name,
diff --git a/lxd/instance/drivers/driver_qemu_templates.go 
b/lxd/instance/drivers/driver_qemu_templates.go
index 2e825176b0..c1b383bc6b 100644
--- a/lxd/instance/drivers/driver_qemu_templates.go
+++ b/lxd/instance/drivers/driver_qemu_templates.go
@@ -497,3 +497,14 @@ x-vga = "on"
 multifunction = "on"
 {{- end }}
 `))
+
+var qemuUSB = template.Must(template.New("qemuUSB").Parse(`
+# USB controller
+[device "qemu_usb"]
+driver = "qemu-xhci"
+bus = "{{.devBus}}"
+addr = "{{.devAddr}}"
+{{if .multifunction -}}
+multifunction = "on"
+{{- end }}
+`))
___
lxc-devel mailing list
lxc-devel@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-devel


[lxc-devel] [lxd/master] lxd/apparmor: Fix devPaths in QEMU profile

2020-10-08 Thread monstermunchkin on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxd/pull/7999

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===

From 9165a2c1baacd875522cab545fbe70257a18b371 Mon Sep 17 00:00:00 2001
From: Thomas Hipp 
Date: Thu, 8 Oct 2020 15:11:57 +0200
Subject: [PATCH] lxd/apparmor: Fix devPaths in QEMU profile

Use the devPaths instead of inst.DevPaths(), as the former evaluates
symlinks.

Signed-off-by: Thomas Hipp 
---
 lxd/apparmor/instance.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lxd/apparmor/instance.go b/lxd/apparmor/instance.go
index 77d266befd..6fd1a2a11e 100644
--- a/lxd/apparmor/instance.go
+++ b/lxd/apparmor/instance.go
@@ -176,7 +176,7 @@ func instanceProfile(state *state.State, inst instance) 
(string, error) {
}
 
err = qemuProfileTpl.Execute(sb, map[string]interface{}{
-   "devPaths":inst.DevPaths(),
+   "devPaths":devPaths,
"exePath": util.GetExecPath(),
"libraryPath": 
strings.Split(os.Getenv("LD_LIBRARY_PATH"), ":"),
"logPath": inst.LogPath(),
___
lxc-devel mailing list
lxc-devel@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-devel


[lxc-devel] [lxd/master] Retry harder upon transient database errors

2020-10-08 Thread freeekanayaka on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxd/pull/8000

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===
This fixes the new failures reported in #6439.
From 6da737988a5fc96724446109a32c10e82bc4154b Mon Sep 17 00:00:00 2001
From: Free Ekanayaka 
Date: Thu, 8 Oct 2020 16:16:36 +0200
Subject: [PATCH 1/5] db: Retry transient errors for longer

We used to retry for a littl more than a second, which is not enough in some
cases.

We now retry for about 25 seconds and with a higher attempt frequency.

Signed-off-by: Free Ekanayaka 
---
 lxd/db/query/retry.go | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/lxd/db/query/retry.go b/lxd/db/query/retry.go
index a67f3dc5ac..8c62026843 100644
--- a/lxd/db/query/retry.go
+++ b/lxd/db/query/retry.go
@@ -5,12 +5,16 @@ import (
"strings"
"time"
 
+   "github.com/Rican7/retry/jitter"
+   "github.com/canonical/go-dqlite/driver"
"github.com/mattn/go-sqlite3"
"github.com/pkg/errors"
 
"github.com/lxc/lxd/shared/logger"
 )
 
+const maxRetries = 250
+
 // Retry wraps a function that interacts with the database, and retries it in
 // case a transient error is hit.
 //
@@ -18,7 +22,7 @@ import (
 func Retry(f func() error) error {
// TODO: the retry loop should be configurable.
var err error
-   for i := 0; i < 5; i++ {
+   for i := 0; i < maxRetries; i++ {
err = f()
if err != nil {
// No point in re-trying or logging a no-row error.
@@ -29,8 +33,12 @@ func Retry(f func() error) error {
// Process actual errors.
logger.Debugf("Database error: %#v", err)
if IsRetriableError(err) {
+   if i == maxRetries {
+   logger.Warnf("Give up retring database 
error: %v", err)
+   break
+   }
logger.Debugf("Retry failed db interaction 
(%v)", err)
-   time.Sleep(250 * time.Millisecond)
+   time.Sleep(jitter.Deviation(nil, 0.8)(100 * 
time.Millisecond))
continue
}
}

From 0978355a1ba22a6552ab8cb13bce99808dfcb014 Mon Sep 17 00:00:00 2001
From: Free Ekanayaka 
Date: Thu, 8 Oct 2020 16:18:10 +0200
Subject: [PATCH 2/5] db: Always retry driver.ErrBusy, regardless of the error
 message

Signed-off-by: Free Ekanayaka 
---
 lxd/db/query/retry.go | 4 
 1 file changed, 4 insertions(+)

diff --git a/lxd/db/query/retry.go b/lxd/db/query/retry.go
index 8c62026843..b8a3fb1116 100644
--- a/lxd/db/query/retry.go
+++ b/lxd/db/query/retry.go
@@ -56,6 +56,10 @@ func IsRetriableError(err error) bool {
return false
}
 
+   if err, ok := err.(driver.Error); ok && err.Code == driver.ErrBusy {
+   return true
+   }
+
if err == sqlite3.ErrLocked || err == sqlite3.ErrBusy {
return true
}

From 4ba063876dcec1ea8659ab74bd66dbec58fb4a33 Mon Sep 17 00:00:00 2001
From: Free Ekanayaka 
Date: Thu, 8 Oct 2020 16:23:23 +0200
Subject: [PATCH 3/5] db: Retry failed rollbacks if they are due to transient
 errors

This avoids leaving the connection in a bad state, where a transaction is
already in progress and another one can't be started.

Signed-off-by: Free Ekanayaka 
---
 lxd/db/query/transaction.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lxd/db/query/transaction.go b/lxd/db/query/transaction.go
index 0cde98614d..77bac41403 100644
--- a/lxd/db/query/transaction.go
+++ b/lxd/db/query/transaction.go
@@ -30,7 +30,7 @@ func Transaction(db *sql.DB, f func(*sql.Tx) error) error {
 // succeeds the given error is returned, otherwise a new error that wraps it
 // gets generated and returned.
 func rollback(tx *sql.Tx, reason error) error {
-   err := tx.Rollback()
+   err := Retry(tx.Rollback)
if err != nil {
logger.Warnf("Failed to rollback transaction after error (%v): 
%v", reason, err)
}

From f68785b88259f34af84c4360f06c33d69cbc4281 Mon Sep 17 00:00:00 2001
From: Free Ekanayaka 
Date: Thu, 8 Oct 2020 16:49:56 +0200
Subject: [PATCH 4/5] db: Explicitly rollback leftover transactions when a new
 one can't be started

Signed-off-by: Free Ekanayaka 
---
 lxd/db/query/transaction.go | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/lxd/db/query/transaction.go b/lxd/db/query/transaction.go
index 77bac41403..901177b0e8 100644
--- a/lxd/db/query/transaction.go
+++ b/lxd/db/query/transaction.go
@@ -2,6 +2,7 @@ package query
 
 import (
"database/sql"
+   "strings"
 
"gi

[lxc-devel] [lxd/master] More graceful shutdown fixes

2020-10-08 Thread stgraber on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxd/pull/8001

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===

From f5d8caf7988e3d6e6fd0a772540d696c92e2bc7d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= 
Date: Thu, 8 Oct 2020 11:18:34 -0400
Subject: [PATCH 1/2] lxd/operations: Fix timeout
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber 
---
 lxd/operations.go | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lxd/operations.go b/lxd/operations.go
index aa982b7034..e5d934c9ed 100644
--- a/lxd/operations.go
+++ b/lxd/operations.go
@@ -97,7 +97,7 @@ func waitForOperations(s *state.State, chCancel chan 
struct{}) {
if execConsoleOps > 0 {
logger.Info("Timeout reached, continuing with 
shutdown")
}
-
+   return
case <-logTick:
// Print log message every minute.
logger.Infof("Waiting for %d operation(s) to finish", 
runningOps)

From 3a6bf54b1ac3dc9156f267a7694aa4eb3f9b896c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= 
Date: Thu, 8 Oct 2020 13:06:49 -0400
Subject: [PATCH 2/2] lxd/daemon: Allow more operations during shutdown
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber 
---
 lxd/daemon.go | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/lxd/daemon.go b/lxd/daemon.go
index a422ef8b20..e04cafc397 100644
--- a/lxd/daemon.go
+++ b/lxd/daemon.go
@@ -461,7 +461,25 @@ func (d *Daemon) createCmd(restAPI *mux.Router, version 
string, c APIEndpoint) {
// - internal calls, e.g. lxd shutdown
// - events endpoint as this is accessed when running `lxd 
shutdown`
// - /1.0 endpoint
-   if version != "internal" && c.Path != "events" && c.Path != "" 
&& d.ctx.Err() == context.Canceled {
+   // - /1.0/operations endpoints
+   // - GET queries
+   allowedDuringShutdown := func() bool {
+   if version == "internal" {
+   return true
+   }
+
+   if c.Path == "" || c.Path == "events" || c.Path == 
"operations" || strings.HasPrefix(c.Path, "operations/") {
+   return true
+   }
+
+   if r.Method == "GET" {
+   return true
+   }
+
+   return false
+   }
+
+   if d.ctx.Err() == context.Canceled && !allowedDuringShutdown() {
response.Unavailable(fmt.Errorf("LXD is shutting 
down")).Render(w)
return
}
___
lxc-devel mailing list
lxc-devel@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-devel


[lxc-devel] [lxd/master] lxd/include: Relocate ifndef for NEWCGROUP

2020-10-08 Thread stgraber on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxd/pull/8002

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===
Signed-off-by: Stéphane Graber 
From 985e7cafd3e9afbe94b4781c736a84f957099c9d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= 
Date: Thu, 8 Oct 2020 15:23:37 -0400
Subject: [PATCH] lxd/include: Relocate ifndef for NEWCGROUP
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber 
---
 lxd/include/macro.h | 4 
 lxd/main_forksyscall.go | 4 
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/lxd/include/macro.h b/lxd/include/macro.h
index 6986a1f685..fc6dd72b55 100644
--- a/lxd/include/macro.h
+++ b/lxd/include/macro.h
@@ -278,4 +278,8 @@ enum {
 #define CLONE_NEWTIME 0x0080
 #endif
 
+#ifndef CLONE_NEWCGROUP
+#define CLONE_NEWCGROUP0x0200
+#endif
+
 #endif /* __LXC_MACRO_H */
diff --git a/lxd/main_forksyscall.go b/lxd/main_forksyscall.go
index b4a62a3b92..4b770e73e7 100644
--- a/lxd/main_forksyscall.go
+++ b/lxd/main_forksyscall.go
@@ -199,10 +199,6 @@ static void mknod_emulate(void)
}
 }
 
-#ifndef CLONE_NEWCGROUP
-#define CLONE_NEWCGROUP0x0200
-#endif
-
 const static int ns_flags[] = { CLONE_NEWUSER, CLONE_NEWPID, CLONE_NEWUTS, 
CLONE_NEWIPC, CLONE_NEWNET, CLONE_NEWCGROUP };
 
 static bool change_creds(int pidfd, int ns_fd, cap_t caps, uid_t nsuid, gid_t 
nsgid, uid_t nsfsuid, gid_t nsfsgid)
___
lxc-devel mailing list
lxc-devel@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-devel


[lxc-devel] [lxd/master] doc: Remove stray \_ escapes in security.md

2020-10-08 Thread sparkiegeek on Github
The following pull request was submitted through Github.
It can be accessed and reviewed at: https://github.com/lxc/lxd/pull/8003

This e-mail was sent by the LXC bot, direct replies will not reach the author
unless they happen to be subscribed to this list.

=== Description (from pull-request) ===

From 6c3a01f9e1a533dbc4283f02eb3b95ea17c5a8dd Mon Sep 17 00:00:00 2001
From: Adam Collard 
Date: Thu, 8 Oct 2020 21:02:29 +0100
Subject: [PATCH] doc: Remove stray \_ escapes in security.md

Signed-off-by: Adam Collard 
---
 doc/security.md | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/doc/security.md b/doc/security.md
index ad12291be4..a6721daf08 100644
--- a/doc/security.md
+++ b/doc/security.md
@@ -236,11 +236,11 @@ instance is using, or can be added to individual 
instances, as shown below.
 
 The following security features are available for `bridged` NICs:
 
-Key  | Type  | Default   | Required  | 
Description
-:--  | :--   | :--   | :--   | :--
-security.mac\_filtering  | boolean   | false | no| Prevent 
the instance from spoofing another's MAC address
-security.ipv4\_filtering | boolean   | false | no| Prevent 
the instance from spoofing another's IPv4 address (enables mac\_filtering)
-security.ipv6\_filtering | boolean   | false | no| Prevent 
the instance from spoofing another's IPv6 address (enables mac\_filtering)
+Key | Type  | Default   | Required  | 
Description
+:-- | :--   | :--   | :--   | :--
+security.mac_filtering  | boolean   | false | no| Prevent 
the instance from spoofing another's MAC address
+security.ipv4_filtering | boolean   | false | no| Prevent 
the instance from spoofing another's IPv4 address (enables mac\_filtering)
+security.ipv6_filtering | boolean   | false | no| Prevent 
the instance from spoofing another's IPv6 address (enables mac\_filtering)
 
 One can override the default `bridged` NIC settings from the profile on a 
per-instance basis using:
 
@@ -258,13 +258,13 @@ different MAC address (i.e using bridged or macvlan NICs).
 The IP filtering features block ARP and NDP advertisements that contain a 
spoofed IP, as well as blocking any
 packets that contain a spoofed source address.
 
-If `security.ipv4\_filtering` or `security.ipv6\_filtering` is enabled and the 
instance cannot be allocated an IP
+If `security.ipv4_filtering` or `security.ipv6_filtering` is enabled and the 
instance cannot be allocated an IP
 address (because `ipvX.address=none` or there is no DHCP service enabled on 
the bridge) then all IP traffic for
 that protocol is blocked from the instance.
 
-When `security.ipv6\_filtering` is enabled IPv6 router advertisements are 
blocked from the instance.
+When `security.ipv6_filtering` is enabled IPv6 router advertisements are 
blocked from the instance.
 
-When `security.ipv4\_filtering` or `security.ipv6\_filtering` is enabled, any 
Ethernet frames that are not ARP,
+When `security.ipv4_filtering` or `security.ipv6_filtering` is enabled, any 
Ethernet frames that are not ARP,
 IPv4 or IPv6 are dropped. This prevents stacked VLAN QinQ (802.1ad) frames 
from bypassing the IP filtering.
 
 ### Routed NIC security
___
lxc-devel mailing list
lxc-devel@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-devel