A little bit off-topic, but there is hope:
This week, I almost saturated a 1 Gbit network link between two brand new
Dell Servers with 3.2 GHz Xeon E5-2667v3 CPUs. I got 105 MB/sec using
standard SSH/SCP. So we finally have single-thread-performance that is fast
enough for encryption on gigabit.
However now I think it is going to be even easier to reproduce in a
VM. My guess if you install a VM with virtual drive on DRBD or MD RAID
and cache=none, create ext3 or ext4 partition inside this VM and run
my code inside then you will get inconsistence. May be you need to run
it not once
Your test code was a really interesting example, and lead to
really unexpected results (at least for me). But as Eneko already
mentioned, nobody would write such code. It is simply not thread save,
and I think qemu does it correctly.
I have written that code only because nobody wanted to test
On May 28, 2015 8:12 PM, Stanislav German-Evtushenko ginermail@
ginerm...@gmail.comgmail.com ginerm...@gmail.com wrote:
Your test code was a really interesting example, and lead to
really unexpected results (at least for me). But as Eneko already
mentioned, nobody would write such code. It
On 05/27/2015 12:15 PM, Dietmar Maurer wrote:
IMHO this way of bypassing os-prober is cleaner than adding a 'Conflict'
in our zfs-grub package, since it minimizes the packages conflicts when
adding our proxmox repo on top of debian's.
[1]
I don't think it is wise to play with security-related software in
the stack. If OpenBSD and Debian (or for the matter all the other
distros) haven't applied those patches, I'm sure there is some
reason, although maybe it being only uncertainty.
Yes, is true.
But I think that from an
Alexandre,
The important point is whether O_DIRECT is used with Ceph or not. Don't you
know?
qemu rbd access is only userland, so host don't have any cache or buffer.
If RBD device does not use host cache then it is very likely that RBD
utilizes O_DIRECT. I am not sure if there are other ways
Hi Stanislav,
On 28/05/15 13:10, Stanislav German-Evtushenko wrote:
Alexandre,
The important point is whether O_DIRECT is used with Ceph or not.
Don't you know?
qemu rbd access is only userland, so host don't have any cache or
buffer.
If RBD device does not use host cache then it is very
Hi,
I'm not kernel/IO expert in any way, but I think this test program has a
race condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on userspace.
Cheers
Eneko
On 28/05/15 11:27, Wolfgang Bumiller
---
www/manager5/dc/Config.js | 4
www/manager5/dc/Log.js| 4
www/manager5/dc/Tasks.js | 4
www/manager5/form/ViewSelector.js | 3 +++
www/manager5/panel/StatusPanel.js | 3 +++
www/manager5/tree/ResourceTree.js | 3 +++
6 files changed, 21
This patch serie allows to load the bottom Statuspanel, and add some comment
headers to the classes already added in manager5/
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
---
PVE/ExtJSIndex5.pm| 3 +++
www/manager5/Workspace.js | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/PVE/ExtJSIndex5.pm b/PVE/ExtJSIndex5.pm
index 34952ac..0f1eb89 100644
--- a/PVE/ExtJSIndex5.pm
+++ b/PVE/ExtJSIndex5.pm
@@ -31,6 +31,7 @@ _EOD
script
Hi Eneko,
Writing in QEMU-KVM are not thread-safe. I don't know if this is by
design or just a bug but proving this information here is necessary to
show that we should find a solution or work around for Proxmox.
General problem is that using Proxmox VE with default settings can make any
of
Not tested but what about this:
fork()
# in child
put current pid into cgroup
exec kvm
Yes, I think it should work, if we put the pid of the forked process in cgroups.
Other child threads should go automaticaly to the parent cgroup.
I have done tests with hotplug virtio-net with vhost
qemu rbd access is only userland, so host don't have any cache or buffer.
If RBD device does not use host cache then it is very likely that RBD
utilizes O_DIRECT. I am not sure if there are other ways to avoid host cache.
qemu use librbd to access directly to ceph, so host don't have any
Eneko,
I'm not kernel/IO expert in any way, but I think this test program has a race
condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on userspace.
Yes, and this is exactly what is happening inside
Alexandre,
qemu use librbd to access directly to ceph, so host don't have any
/dev/rbd.. or filesystem mount.
Ah, I understand, this is not a normal block device but userspace lib.
ceph use O_DIRECT+O_DYNC to write to the journal of osds.
Is this done inside KVM process? If so then KVM keeps
I'm not kernel/IO expert in any way, but I think this test program has a
race condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on userspace.
For this test, we do not care about userspace
On 28/05/15 12:38, dea wrote:
Il Thu, 28 May 2015 12:02:21 +0200 (CEST), Dietmar Maurer scrisse
I've find this...
http://www.psc.edu/index.php/hpn-ssh
What do you all think?
This is great, but unfortunately ssh people rejected those patches
(AFAIK). So default ssh tools from Debian does not
BTW: can anybody test drbd_oos_test.c against Ceph? I guess we will have the
same result.
I think they are no problem with ceph, qemu cache option only enable|disable
rbd_cache.
qemu rbd access is only userland, so host don't have any cache or buffer.
When data is written to ceph, it's
On May 28, 2015 at 1:31 PM Eneko Lacunza elacu...@binovo.es wrote:
I'm not kernel/IO expert in any way, but I think this test program has a
race condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on
I've find this...
http://www.psc.edu/index.php/hpn-ssh
What do you all think?
This is great, but unfortunately ssh people rejected those patches (AFAIK).
So default ssh tools from Debian does not have that features.
___
pve-devel mailing list
Here the patches serie for implemtation of cpuunits through cgroups
If you implement it inside qemu-server you have a race-condition, because
you do it too late and qemu already started the threads? Or maybe only parts
of necessary threads are already created?
So wouldn't it be easier to do it
If you implement it inside qemu-server you have a race-condition, because
you do it too late and qemu already started the threads? Or maybe only parts
of necessary threads are already created?
Yes maybe, I don't have see race when I have tested it.
new threads should go automatically to the
Another way could be use launch qemu with systemd-run, I think we can specify
cgroup (slice in systemd) directly.
- Mail original -
De: aderumier aderum...@odiso.com
À: Stefan Priebe s.pri...@profihost.ag
Cc: pve-devel pve-devel@pve.proxmox.com
Envoyé: Jeudi 28 Mai 2015 13:36:25
Objet:
On May 28, 2015 at 2:07 PM Stanislav German-Evtushenko ginerm...@gmail.com
wrote:
With O_DIRECT we have to trust our user space application because data
is getting by kernel directly from the application memory. We can
think that kernel could copy buffer from user space before writing it
to
On 28/05/15 13:44, Stanislav German-Evtushenko wrote:
Eneko,
I'm not kernel/IO expert in any way, but I think this test program has a race
condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on
Dietmar,
I'm not kernel/IO expert in any way, but I think this test program has a
race condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on userspace.
For this test, we do not care about userspace
On 28/05/15 13:49, Dietmar Maurer wrote:
I'm not kernel/IO expert in any way, but I think this test program has a
race condition, so it is not helping us diagnose the problem.
We're writing to buffer x while it is in use by write syscall. This is
plainly wrong on userspace.
For this test, we
Eneko,
Note that without O_DIRECT you won't get a correct result either; disk
may end not containing the data in the buffer when write was called.
Softmirror data will be identically uncertain :)
You are right. That is why I suppose there is a bug (operations with
buffer are not ensured to be
On 28/05/15 15:01, Stanislav German-Evtushenko wrote:
Note that without O_DIRECT you won't get a correct result either; disk
may end not containing the data in the buffer when write was called.
Softmirror data will be identically uncertain :)
You are right. That is why I suppose there is a bug
Another way could be use launch qemu with systemd-run, I think we can specify
cgroup (slice in systemd) directly.
for example:
systemd-run --remain-after-exit --slice=qemu --unit=100 -p CPUShares=499
/usr/bin/kvm -id 100
Seem better than hacking qemu ?
- Mail original -
De:
better with scope:
systemd-run --scope --slice=qemu --unit 100 -p CPUShares=499 /usr/bin/kvm -id
100 ...
like this, cgroup is autoremoved on process stop.
- Mail original -
De: aderumier aderum...@odiso.com
À: dietmar diet...@proxmox.com
Cc: pve-devel pve-devel@pve.proxmox.com
This first patch should still work with the existing scripts and allow us to
gradually change the startup of our daemon scripts so that they do not try to
perform tasks that need root before handling the autogenerated build-time
utiliity commands like 'printmanpod'.
We can then move mkdirs,
On Thu, May 28, 2015 at 4:17 PM, Eneko Lacunza elacu...@binovo.es wrote:
On 28/05/15 15:01, Stanislav German-Evtushenko wrote:
Note that without O_DIRECT you won't get a correct result either; disk
may end not containing the data in the buffer when write was called.
Softmirror data will be
Signed-off-by: Alexandre Derumier aderum...@odiso.com
---
PVE/QemuServer.pm | 15 +++
1 file changed, 15 insertions(+)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 3cd4475..fe40140 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -67,6 +67,17 @@
On 28/05/15 15:32, Stanislav German-Evtushenko wrote:
What does it mean that operations with buffer are not ensured to be
thread-safe in qemu?
O_DIRECT doesn't guarantee that buffer reading is finished when write
returns if I read man -s 2 open correctly.
The statement seems to be not
Signed-off-by: Alexandre Derumier aderum...@odiso.com
---
PVE/QemuServer.pm | 87 ---
1 file changed, 87 deletions(-)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index fe40140..94b9176 100644
--- a/PVE/QemuServer.pm
+++
A first step towards untangling some of the intermingled data and
functionality setup tasks for the daemons:
Daemon::new now only validates and untaints arguments, but doesn't
perform any actions such as setuid/setgid until the new Daemon::setup
method which is now executed from Daemon::start
On May 28, 2015 at 3:17 PM Eneko Lacunza elacu...@binovo.es wrote:
On 28/05/15 15:01, Stanislav German-Evtushenko wrote:
Note that without O_DIRECT you won't get a correct result either; disk
may end not containing the data in the buffer when write was called.
Softmirror data will be
A first step towards untangling some of the intermingled data and
functionality setup tasks for the daemons:
Daemon::new now only validates and untaints arguments, but doesn't
perform any actions such as setuid/setgid until the new Daemon::setup
method which is now executed from Daemon::start
Okay things are running again.
I noticed the first git --amend didn't make it into the previous patches either
so here's the updated and working patch.
Wolfgang Bumiller (1):
defer some daemon setup routines
src/PVE/CLIHandler.pm | 4 +-
src/PVE/Daemon.pm | 116
Each kvm process have multiple threads and the number of them is
changing in time.
AFAIK all disk IO is done by a single, dedicated thread.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
On Thu, May 28, 2015 at 7:19 PM, Dietmar Maurer diet...@proxmox.com wrote:
Each kvm process have multiple threads and the number of them is
changing in time.
AFAIK all disk IO is done by a single, dedicated thread.
I tried to read qemu-kvm code but it is difficult for me as I have
never
ceph use O_DIRECT+O_DYNC to write to the journal of osds.
Is this done inside KVM process? If so then KVM keeps buffer for this
O_DIRECT writing. Therefore if multiple threads can access (and change)
this buffer at the same time then the similar issue can happen in theory.
It only happens if
applied. But we should implement/use the new functionality asap to see
if everything works as expecteced.
On 05/28/2015 04:54 PM, Wolfgang Bumiller wrote:
Okay things are running again.
I noticed the first git --amend didn't make it into the previous patches either
so here's the updated and
Message: 9
Date: Thu, 28 May 2015 17:47:54 +0200 (CEST)
From: Dietmar Maurer diet...@proxmox.com
To: Stanislav German-Evtushenko ginerm...@gmail.com
Cc: pve-devel pve-devel@pve.proxmox.com
Subject: Re: [pve-devel] Default cache mode for VM hard drives
Message-ID:
On Thu, May 28, 2015 at 6:47 PM, Dietmar Maurer diet...@proxmox.com wrote:
But there is currently only one io-thread in qemu, so this
cannot happen with qemu if above is the only problem?
But there are other threads, right? Buffer can be changed by another
thread where guest OS itself is
Another way could be use launch qemu with systemd-run, I think we can specify
cgroup (slice in systemd) directly.
What would be the advantage? Fr me that just makes things more complex?
___
pve-devel mailing list
pve-devel@pve.proxmox.com
This is not okay and this is what is actually happening:
0. set_buffer
1. start_writing_with_o_direct_from_buffer
2. change_buffer (we can do this only in another thread)
3. finish_writing_with_o_direct_from_buffer
4. change_buffer
5. start_writing_with_o_direct_from_buffer
6.
But there is currently only one io-thread in qemu, so this
cannot happen with qemu if above is the only problem?
But there are other threads, right? Buffer can be changed by another
thread where guest OS itself is running.
No, AFAIK there is only one thread doing all IO (currently). But
If you provide a buffer to the kernel, that you change while it is
working with it, I don't know why you expect a reliable/predictable
result? Specially (but not only) if you tell it not to make a copy!!
Note that without O_DIRECT you won't get a correct result either; disk
may end not
systemd-run --scope --slice=qemu --unit 100 -p CPUShares=499 /usr/bin/kvm -id
100 ...
like this, cgroup is autoremoved on process stop.
Interesting. I need toö take a look at the code in 'systemd-run' to see how
complex that
is. Are there any disadvantages?
On Thu, May 28, 2015 at 6:35 PM, Dietmar Maurer diet...@proxmox.com wrote:
This is not okay and this is what is actually happening:
0. set_buffer
1. start_writing_with_o_direct_from_buffer
2. change_buffer (we can do this only in another thread)
3. finish_writing_with_o_direct_from_buffer
4.
On Thu, May 28, 2015 at 7:19 PM, Dietmar Maurer diet...@proxmox.com wrote:
Each kvm process have multiple threads and the number of them is
changing in time.
AFAIK all disk IO is done by a single, dedicated thread.
I tried to read qemu-kvm code but it is difficult for me as I have
Interesting. I need toö take a look at the code in 'systemd-run' to see how
complex that is. Are there any disadvantages?
I don't see any disadvantages.
from systemd-run:
If a command is run as transient scope unit, it will be started directly by
systemd-run and thus inherit the execution
56 matches
Mail list logo
