Re: privileged entropy sources in QEMU/KVM guests
On Thu, Nov 07, 2019 at 11:10:57AM +0100, Laszlo Ersek wrote: > Hi, > > related TianoCore BZ: > > https://bugzilla.tianocore.org/show_bug.cgi?id=1871 > > (I'm starting this thread separately because at least some of the topics > are specific to QEMU, and I didn't want to litter the BZ with a > discussion that may not be interesting to all participants CC'd on the > BZ. I am keeping people CC'd on this initial posting; please speak up if > you'd like to be dropped from the email thread.) > > QEMU provides guests with the virtio-rng device, and the OVMF and > ArmVirtQemu* edk2 platforms build EFI_RNG_PROTOCOL on top of that > device. But, that doesn't seem enough for all edk2 use cases. > > Also, virtio-rng (hence EFI_RNG_PROTOCOL too) is optional, and its > absence may affect some other use cases. The optional nature of virtio-rng is something that results in the the same problems for Linux. If virtio-rng is absent, then Linux now has a general purpose fallback via the CPU timing jitter entropy source: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bb5530e4082446aac3a3d69780cd4dbfa4520013 Is it practical to provide a jitter entropy source for EDK2 too ? > (1) For UEFI HTTPS boot, TLS would likely benefit from good quality > entropy. If the VM config includes virtio-rng (hence the guest firmware > has EFI_RNG_PROTOCOL), then it should be used as a part of HTTPS boot. > > However, what if virtio-rng (hence EFI_RNG_PROTOCOL) are absent? Should > UEFI HTTPS boot be disabled completely (or prevented / rejected > somehow), blaming lack of good entropy? Or should TLS silently fall back > to "mixing some counters [such as TSC] together and applying a > deterministic cryptographic transformation"? > > IOW, knowing that the TLS setup may not be based on good quality > entropy, should we allow related firmware services to "degrade silently" > (not functionally, but potentially in security), or should we deny the > services altogether? If we can guarantee we always present fallback like jitterentropy then the problem with TLS init goes away IIUC. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH 0/6] migration/postcopy: enable compress during postcopy
* Wei Yang (richardw.y...@linux.intel.com) wrote: > On Thu, Nov 07, 2019 at 09:15:44AM +, Dr. David Alan Gilbert wrote: > >* Wei Yang (richardw.y...@linux.intel.com) wrote: > >> On Wed, Nov 06, 2019 at 08:11:44PM +, Dr. David Alan Gilbert wrote: > >> >* Wei Yang (richardw.y...@linux.intel.com) wrote: > >> >> This patch set tries enable compress during postcopy. > >> >> > >> >> postcopy requires to place a whole host page, while migration thread > >> >> migrate > >> >> memory in target page size. This makes postcopy need to collect all > >> >> target > >> >> pages in one host page before placing via userfaultfd. > >> >> > >> >> To enable compress during postcopy, there are two problems to solve: > >> >> > >> >> 1. Random order for target page arrival > >> >> 2. Target pages in one host page arrives without interrupt by target > >> >>page from other host page > >> >> > >> >> The first one is handled by counting the number of target pages arrived > >> >> instead of the last target page arrived. > >> >> > >> >> The second one is handled by: > >> >> > >> >> 1. Flush compress thread for each host page > >> >> 2. Wait for decompress thread for before placing host page > >> >> > >> >> With the combination of these two changes, compress is enabled during > >> >> postcopy. > >> > > >> >What have you tested this with? 2MB huge pages I guess? > >> > > >> > >> I tried with this qemu option: > >> > >>-object > >> memory-backend-file,id=mem1,mem-path=/dev/hugepages/guest2,size=4G \ > >>-device pc-dimm,id=dimm1,memdev=mem1 > >> > >> /dev/hugepages/guest2 is a file under hugetlbfs > >> > >>hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) > > > >OK, yes that should be fine. > >I suspect on Power/ARM where they have normal memory with 16/64k pages, > >the cost of the flush will mean compression is more expensive in > >postcopy mode; but still makes it possible. > > > > Not get your point clearly about more expensive. You mean more expensive on > ARM/Power? Yes; you're doing a flush at the end of each host page; on x86 without hugepages you don't do anything, on arm/power you'll need to do a flush at the end of each of their normal pages - so that's a bit more expensive. > If the solution looks good to you, I would prepare v2. Yes; I think it is OK. Dave > >Dave > > > >> >Dave > >> > > >> >> Wei Yang (6): > >> >> migration/postcopy: reduce memset when it is zero page and > >> >> matches_target_page_size > >> >> migration/postcopy: wait for decompress thread in precopy > >> >> migration/postcopy: count target page number to decide the > >> >> place_needed > >> >> migration/postcopy: set all_zero to true on the first target page > >> >> migration/postcopy: enable random order target page arrival > >> >> migration/postcopy: enable compress during postcopy > >> >> > >> >> migration/migration.c | 11 > >> >> migration/ram.c | 65 ++- > >> >> 2 files changed, 45 insertions(+), 31 deletions(-) > >> >> > >> >> -- > >> >> 2.17.1 > >> >> > >> >-- > >> >Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK > >> > >> -- > >> Wei Yang > >> Help you, Help me > >-- > >Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK > > -- > Wei Yang > Help you, Help me -- Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
Re: Looking for issues/features for my first contribution
On Thu, Nov 7, 2019 at 11:37 AM Aleksandar Markovic wrote: > > > > On Thursday, November 7, 2019, Rajath Shashidhara > wrote: >> >> Hi all, >> >> I am a Computer Science graduate student at The University of Texas at >> Austin (UT, Austin). I am looking forward to contributing to qemu ! >> >> This semester, I am taking a class in Virtualization >> (https://github.com/vijay03/cs378-f19) and contributing to a virtualization >> related open-source project is a significant part of the course. >> I would be interested in contributing a patchset to qemu - possibly a >> self-contained feature or a reasonably complex bug fix that can be completed >> in under a month's time. I did look at both the bugtracker and the QEMU >> Google Summer of Code 2019 page >> [https://wiki.qemu.org/Google_Summer_of_Code_2019] for ideas. However, I >> would be interested in hearing from the community and I would be delighted >> if somebody can be suggest a suitable project ! >> > > Hello, Rajath! > > Thank you for expressing interest in QEMU open source project. > > There is certainly a place for you and your contributions in QEMU, and you > are very welcomed! > > It looks to me the following project would fit your description: > > 'Implement emulation of DS3231 real time clock in QEMU' > > Datasheet: > > https://datasheets.maximintegrated.com/en/ds/DS3231.pdf > > The steps needed to complete it (in my opinion): > > - collect datasheets of as many as possible RTC chips already emulated in > QEMU (there are around of dozen of them, see folder hw/rtc) > I did a quick Google search on datasheets of existing RTC implemtations, and the result is: DS1338: https://datasheets.maximintegrated.com/en/ds/DS1338-DS1338Z.pdf M41T80: https://www.st.com/resource/en/datasheet/m41t80.pdf M48T59: http://www.elektronikjk.pl/elementy_czynne/IC/M48T59V.pdf MC146818: https://www.nxp.com/docs/en/data-sheet/MC146818.pdf PL031: http://infocenter.arm.com/help/topic/com.arm.doc.ddi0224c/real_time_clock_pl031_r1p3_technical_reference_manual_DDI0224C.pdf TWL92230: https://datasheet.octopart.com/TWL92230C-Texas-Instruments-datasheet-150321.pdf Zynq RTC: https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ultrascale-trm.pdf (chapter 7) Aleksandar > - do a comparative analysis of selected RTC implementations in QEMU > > - get to know general QEMU device model > > - design and implement DS3231 emulation > > I can give you (unfortunately constrained by tight time limits) some help and > guidance. But there are other people in community too (more knowledgable in > the area than me). > > I salute your initiative! > > Yours, > Aleksandar > > > >> >> I am an advanced C programmer with both professional and academic background >> in systems design & implementation - especially OS & Networks. Given my >> background, I feel fairly confident that I can pickup the QEMU codebase >> quickly. >> >> Eagerly looking forward to hearing from the community ! >> >> Thanks, >> Rajath Shashidhara >> >>
Re: privileged entropy sources in QEMU/KVM guests
On 11/07/19 13:47, Paolo Bonzini wrote: > On 07/11/19 12:52, Daniel P. Berrangé wrote: >> >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bb5530e4082446aac3a3d69780cd4dbfa4520013 >> >> Is it practical to provide a jitter entropy source for EDK2 >> too ? > > The hard part is not collecting jitter (though the firmware might be too > deterministic for that), but rather turning it into a random number seed > (mixing data from various sources, crediting entropy, etc.). If there is *any* entropy source that (a) we can trust to be random enough and (b) depends only on the CPU, then we shouldn't struggle with virtio-rng (or similar devices) at all. RDRAND would be a no-brainer, but the "community literature" suggests it should not be trusted in itself. I've read the commit message linked above, and it appears too good to be true. The CPU Jitter RNG provides a source of good entropy by collecting CPU executing time jitter. [...] The RNG does not have any dependencies on any other service in the kernel. The RNG only needs a high-resolution time stamp. [...] http://www.chronox.de/jent.html The CPU Jitter Random Number Generator provides a non-physical true random number generator that works equally in kernel and user land. The only prerequisite is the availability of a high-resolution timer that is available in modern CPUs. http://www.chronox.de/jent/doc/CPU-Jitter-NPTRNG.html Today’s operating systems provide non-physical true random number generators which are based on hardware events. With the advent of virtualization and the ever growing need of more high-quality random numbers, these random number generators reach their limits. Additional sources of entropy must be opened up. This document introduces an entropy source based on CPU execution time jitter. The design and implementation of a non-physical true random number generator, the CPU Jitter random number generator, its statistical properties and the maintenance and behavior of entropy is discussed in this document. If this construct is legit, a core edk2 implementation (available to all platforms, and on all edk2 arches) would be a huge win. On the other hand, we're having this discussion because the premise of TianoCore#1871 is that we shouldn't rely on just the CPU and a high resolution timer... I simply cannot decide if this construct is trustworthy. (With any solution that was based in the host's /dev/random or /dev/urandom, the trustworthiness question would be side-stepped in the firmware.) Laszlo
Re: [PATCH v1 1/3] target/microblaze: Plug temp leaks for loads/stores
On 11/6/19 3:14 PM, Edgar E. Iglesias wrote: > @@ -967,12 +967,14 @@ static void dec_load(DisasContext *dc) > 10 -> 10 > 11 -> 00 */ > TCGv low = tcg_temp_new(); > +TCGv t3 = tcg_const_tl(3); > > tcg_gen_andi_tl(low, addr, 3); > -tcg_gen_sub_tl(low, tcg_const_tl(3), low); > +tcg_gen_sub_tl(low, t3, low); > tcg_gen_andi_tl(addr, addr, ~3); > tcg_gen_or_tl(addr, addr, low); > tcg_temp_free(low); > +tcg_temp_free(t3); > break; While Luc correctly notes that tcg_gen_subfi_tl() may be used here, I will note (1) there's a typo in the comment (not 2->2, but 2->1), and (2) that this whole section can be done with tcg_gen_xori_tl(addr, addr, 3); Similarly in dec_store. The other changes in this patch around gen_helper_memalign are ok. r~
Re: [PATCH v4 20/20] fuzz: add documentation to docs/devel/
On 11/7/19 8:40 AM, Stefan Hajnoczi wrote: On Wed, Oct 30, 2019 at 02:50:04PM +, Oleinik, Alexander wrote: +== Building the fuzzers == + +NOTE: If possible, build a 32-bit binary. When forking, the 32-bit fuzzer is +much faster, since the page-map has a smaller size. This is due to the fact that +AddressSanitizer mmaps ~20TB of memory, as part of its detection. This results +in a large page-map, and a much slower fork(). O + +To build the fuzzers, install a recent version of clang: +Configure with (substitute the clang binaries with the version you installed): + +CC=clang-8 CXX=clang++-8 /path/to/configure --enable-fuzzing + +Fuzz targets are built similarly to system/softmmu: + +make i386-softmmu/fuzz + +This builds ./i386-softmmu/qemu-fuzz-i386 I'm surprised that "make i386-softmmu/fuzz" builds i386-softmmu/qemu-fuzz-i386. Should that be "make i386-softmmu/qemu-fuzz-i386" I tried to make the rule match the names for regular targets. Ie: make i386-softmmu/clean make i386-softmmu/all make i386-softmmu/install Now there is an i386-softmmu/fuzz += Implmentation Details = s/Implmentation/Implementation/ + +== The Fuzzer's Lifecycle == + +The fuzzer has two entrypoints that libfuzzer calls. libfuzzer provides it's +own main(), which performs some setup, and calls the entrypoints: + +LLVMFuzzerInitialize: called prior to fuzzing. Used to initialize all of the +necessary state + +LLVMFuzzerTestOneInput: called for each fuzzing run. Processes the input and +resets the state at the end of each run. + +In more detail: + +LLVMFuzzerInitialize parses the arguments to the fuzzer (must start with two +dashes, so they are ignored by libfuzzer main()). Currently, the arguments +select the fuzz target. Then, the qtest client is initialized. If the target +requires qos, qgraph is set up and the QOM/LIBQOS modules are initailized. s/initailized/initialized/
Re: [RFC PATCH 06/18] qemu-storage-daemon: Add --nbd-server option
On 11/7/19 9:27 AM, Kevin Wolf wrote: Am 07.11.2019 um 14:45 hat Eric Blake geschrieben: On 11/7/19 2:33 AM, Kevin Wolf wrote: As a replacement nbd-server-add, I envisioned adding something like a block-export-add, which would work the way that --export already does. It would also come with query-block-exports and block-export-del, and it wouldn't contain only NBD devices, but also vhost-user, FUSE, etc. exports. Now I'm wondering if the same would make sense for nbd-server-start. Maybe an API change would even allow us to start multiple NBD servers (e.g. listening on different IP addresses or using different tls-creds). We want that (the ability to run multiple parallel NBD servers) anyway, to allow parallel incremental backups from different points in time to different clients. Can't you already have multiple exports on a single NBD server for multiple clients today? Or do you need a different server configuration for each client? With our current code base, you can only run a single NBD server, with multiple exports, but the TLS creds are shared among all exports. It is indeed technically possible to tweak things where the single server changes _which_ exports are exposed based on _which_ creds were used by the client (but only when TLS is used, and note that qemu-nbd currently refuses to mix TLS and Unix sockets, although I need to post a v2 of a patch I proposed a while back to improve that). But it is easier still to run two separate servers on different ports with two different creds, and where there is no magic on which exports to show merely based on which creds were presented (and this includes a plaintext connection over Unix). Either way, it requires code changes, and most likely for 5.0. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v1 4/4] iotests: add test for virtio-scsi and virtio-blk machine type settings
On Wed, Nov 06, 2019 at 04:26:41PM -0300, Eduardo Habkost wrote: > On Wed, Nov 06, 2019 at 11:04:16AM +0100, Max Reitz wrote: > > On 06.11.19 10:24, Stefan Hajnoczi wrote: > > > On Tue, Nov 05, 2019 at 07:11:05PM +0300, Denis Plotnikov wrote: > > >> It tests proper queue size settings for all available machine types. > > >> > > >> Signed-off-by: Denis Plotnikov > > >> --- > > >> tests/qemu-iotests/267 | 154 + > > >> tests/qemu-iotests/267.out | 1 + > > >> tests/qemu-iotests/group | 1 + > > >> 3 files changed, 156 insertions(+) > > >> create mode 100755 tests/qemu-iotests/267 > > >> create mode 100644 tests/qemu-iotests/267.out > > > > > > The qemu-iotests maintainers might prefer for this to be at the > > > top-level in tests/ since it's not really an iotest, but the code itself > > > looks fine to me: > > > > > > Reviewed-by: Stefan Hajnoczi > > > > Good question. I don’t really mind, but it would be weird if started > > adding all kinds of “external” qemu tests (i.e. that use QMP) in the > > iotests directory. > > > > What is the alternative? Just putting it in a different directory > > doesn’t sound that appealing to me either, because it would still depend > > on the iotests infrastructure, right? (i.e., iotests.py and check) > > We do have tests/acceptance for simple test cases written in > Python. What's the reason for this test case to depend on the > iotests infrastructure? > > -- > Eduardo This test does look similar in spirit to "tests/acceptance/virtio_version.py". Denis, If you think this is more of a generic test than an IO test, and would rather want to have it a more agnostic location, I can provide you with tips (or a patch) to do so. Cheers, - Cleber.
[PATCH v3 17/22] iotests: Make 091 work with data_file
The image end offset as reported by qemu-img check is different when
using an external data file; we do not care about its value here, so we
can just filter it. Incidentally, common.rc already has _check_test_img
for us which does exactly that.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/091 | 2 +-
tests/qemu-iotests/091.out | 2 --
2 files changed, 1 insertion(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091
index f4b44659ae..0874fa84c8 100755
--- a/tests/qemu-iotests/091
+++ b/tests/qemu-iotests/091
@@ -101,7 +101,7 @@ echo "Check image pattern"
${QEMU_IO} -c "read -P 0x22 0 4M" "${TEST_IMG}" | _filter_testdir |
_filter_qemu_io
echo "Running 'qemu-img check -r all \$TEST_IMG'"
-"${QEMU_IMG}" check -r all "${TEST_IMG}" 2>&1 | _filter_testdir | _filter_qemu
+_check_test_img -r all
echo "*** done"
rm -f $seq.full
diff --git a/tests/qemu-iotests/091.out b/tests/qemu-iotests/091.out
index 5017f8c2d9..5ec7b00f13 100644
--- a/tests/qemu-iotests/091.out
+++ b/tests/qemu-iotests/091.out
@@ -23,6 +23,4 @@ read 4194304/4194304 bytes at offset 0
4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Running 'qemu-img check -r all $TEST_IMG'
No errors were found on the image.
-80/16384 = 0.49% allocated, 0.00% fragmented, 0.00% compressed clusters
-Image end offset: 5570560
*** done
--
2.23.0
[PATCH v3 11/22] iotests: Replace IMGOPTS= by -o
Tests should not overwrite all user-supplied image options, but only add to it (which will effectively overwrite conflicting values). Accomplish this by passing options to _make_test_img via -o instead of $IMGOPTS. For some tests, there is no functional change because they already only appended options to IMGOPTS. For these, this patch is just a simplification. For others, this is a change, so they now heed user-specified $IMGOPTS. Some of those tests do not work with all image options, though, so we need to disable them accordingly. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/031 | 9 --- tests/qemu-iotests/039 | 24 ++ tests/qemu-iotests/059 | 18 ++--- tests/qemu-iotests/060 | 6 ++--- tests/qemu-iotests/061 | 57 ++ tests/qemu-iotests/079 | 3 +-- tests/qemu-iotests/106 | 2 +- tests/qemu-iotests/108 | 2 +- tests/qemu-iotests/112 | 32 tests/qemu-iotests/115 | 3 +-- tests/qemu-iotests/121 | 6 ++--- tests/qemu-iotests/125 | 2 +- tests/qemu-iotests/137 | 2 +- tests/qemu-iotests/138 | 3 +-- tests/qemu-iotests/175 | 2 +- tests/qemu-iotests/190 | 2 +- tests/qemu-iotests/191 | 3 +-- tests/qemu-iotests/220 | 4 ++- tests/qemu-iotests/243 | 6 +++-- tests/qemu-iotests/244 | 10 +--- tests/qemu-iotests/250 | 3 +-- tests/qemu-iotests/265 | 2 +- 22 files changed, 100 insertions(+), 101 deletions(-) diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031 index a3c25ec237..c44fcf91bb 100755 --- a/tests/qemu-iotests/031 +++ b/tests/qemu-iotests/031 @@ -40,19 +40,22 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file +# We want to test compat=0.10, which does not support refcount widths +# other than 16 +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' CLUSTER_SIZE=65536 # qcow2.py output depends on the exact options used, so override the command # line here as an exception -for IMGOPTS in "compat=0.10" "compat=1.1"; do +for compat in "compat=0.10" "compat=1.1"; do echo -echo = Testing with -o $IMGOPTS = +echo = Testing with -o $compat = echo echo === Create image with unknown header extension === echo -_make_test_img 64M +_make_test_img -o $compat 64M $PYTHON qcow2.py "$TEST_IMG" add-header-ext 0x12345678 "This is a test header extension" $PYTHON qcow2.py "$TEST_IMG" dump-header _check_test_img diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039 index 325da63a4c..99563bf126 100755 --- a/tests/qemu-iotests/039 +++ b/tests/qemu-iotests/039 @@ -50,8 +50,7 @@ size=128M echo echo "== Checking that image is clean on shutdown ==" -IMGOPTS="compat=1.1,lazy_refcounts=on" -_make_test_img $size +_make_test_img -o "compat=1.1,lazy_refcounts=on" $size $QEMU_IO -c "write -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io @@ -62,8 +61,7 @@ _check_test_img echo echo "== Creating a dirty image file ==" -IMGOPTS="compat=1.1,lazy_refcounts=on" -_make_test_img $size +_make_test_img -o "compat=1.1,lazy_refcounts=on" $size _NO_VALGRIND \ $QEMU_IO -c "write -P 0x5a 0 512" \ @@ -98,8 +96,7 @@ $QEMU_IO -c "read -P 0x5a 0 512" "$TEST_IMG" | _filter_qemu_io echo echo "== Opening a dirty image read/write should repair it ==" -IMGOPTS="compat=1.1,lazy_refcounts=on" -_make_test_img $size +_make_test_img -o "compat=1.1,lazy_refcounts=on" $size _NO_VALGRIND \ $QEMU_IO -c "write -P 0x5a 0 512" \ @@ -117,8 +114,7 @@ $PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features echo echo "== Creating an image file with lazy_refcounts=off ==" -IMGOPTS="compat=1.1,lazy_refcounts=off" -_make_test_img $size +_make_test_img -o "compat=1.1,lazy_refcounts=off" $size _NO_VALGRIND \ $QEMU_IO -c "write -P 0x5a 0 512" \ @@ -132,11 +128,9 @@ _check_test_img echo echo "== Committing to a backing file with lazy_refcounts=on ==" -IMGOPTS="compat=1.1,lazy_refcounts=on" -TEST_IMG="$TEST_IMG".base _make_test_img $size +TEST_IMG="$TEST_IMG".base _make_test_img -o "compat=1.1,lazy_refcounts=on" $size -IMGOPTS="compat=1.1,lazy_refcounts=on,backing_file=$TEST_IMG.base" -_make_test_img $size +_make_test_img -o "compat=1.1,lazy_refcounts=on,backing_file=$TEST_IMG.base" $size $QEMU_IO -c "write 0 512" "$TEST_IMG" | _filter_qemu_io $QEMU_IMG commit "$TEST_IMG" @@ -151,8 +145,7 @@ TEST_IMG="$TEST_IMG".base _check_test_img echo echo "== Changing lazy_refcounts setting at runtime ==" -IMGOPTS="compat=1.1,lazy_refcounts=off" -_make_test_img $size +_make_test_img -o "compat=1.1,lazy_refcounts=off" $size _NO_VALGRIND \ $QEMU_IO -c "reopen -o lazy-refcounts=on" \ @@ -164,8 +157,7 @@ $QEMU_IO -c "reopen -o lazy-refcounts=on" \ $PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features _check_test_img -IMGOPTS="compat=1.1,lazy_refcounts=on" -_make_test_img $size
[PATCH v3 21/22] iotests: Disable data_file where it cannot be used
Signed-off-by: Max Reitz --- tests/qemu-iotests/007 | 5 +++-- tests/qemu-iotests/014 | 2 ++ tests/qemu-iotests/015 | 5 +++-- tests/qemu-iotests/026 | 5 - tests/qemu-iotests/029 | 5 +++-- tests/qemu-iotests/031 | 6 +++--- tests/qemu-iotests/036 | 5 +++-- tests/qemu-iotests/039 | 3 +++ tests/qemu-iotests/046 | 2 ++ tests/qemu-iotests/048 | 2 ++ tests/qemu-iotests/051 | 5 +++-- tests/qemu-iotests/058 | 5 +++-- tests/qemu-iotests/060 | 6 -- tests/qemu-iotests/061 | 6 -- tests/qemu-iotests/062 | 2 +- tests/qemu-iotests/066 | 4 +++- tests/qemu-iotests/067 | 6 -- tests/qemu-iotests/068 | 5 +++-- tests/qemu-iotests/071 | 3 +++ tests/qemu-iotests/073 | 4 tests/qemu-iotests/074 | 2 ++ tests/qemu-iotests/080 | 5 +++-- tests/qemu-iotests/090 | 2 ++ tests/qemu-iotests/098 | 6 -- tests/qemu-iotests/099 | 3 ++- tests/qemu-iotests/103 | 5 +++-- tests/qemu-iotests/108 | 6 -- tests/qemu-iotests/112 | 5 +++-- tests/qemu-iotests/114 | 2 ++ tests/qemu-iotests/121 | 3 +++ tests/qemu-iotests/138 | 3 +++ tests/qemu-iotests/156 | 2 ++ tests/qemu-iotests/176 | 7 +-- tests/qemu-iotests/191 | 2 ++ tests/qemu-iotests/201 | 6 +++--- tests/qemu-iotests/214 | 3 ++- tests/qemu-iotests/217 | 3 ++- tests/qemu-iotests/220 | 5 +++-- tests/qemu-iotests/243 | 6 -- tests/qemu-iotests/244 | 5 +++-- tests/qemu-iotests/250 | 2 ++ tests/qemu-iotests/261 | 3 ++- tests/qemu-iotests/267 | 5 +++-- 43 files changed, 124 insertions(+), 53 deletions(-) diff --git a/tests/qemu-iotests/007 b/tests/qemu-iotests/007 index 7d3544b479..160683adf8 100755 --- a/tests/qemu-iotests/007 +++ b/tests/qemu-iotests/007 @@ -41,8 +41,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic # refcount_bits must be at least 4 so we can create ten internal snapshots -# (1 bit supports none, 2 bits support two, 4 bits support 14) -_unsupported_imgopts 'refcount_bits=\(1\|2\)[^0-9]' +# (1 bit supports none, 2 bits support two, 4 bits support 14); +# snapshot are generally impossible with external data files +_unsupported_imgopts 'refcount_bits=\(1\|2\)[^0-9]' data_file echo echo "creating image" diff --git a/tests/qemu-iotests/014 b/tests/qemu-iotests/014 index 2f728a1956..e1221c0fff 100755 --- a/tests/qemu-iotests/014 +++ b/tests/qemu-iotests/014 @@ -43,6 +43,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file _supported_os Linux +# Compression and snapshots do not work with external data files +_unsupported_imgopts data_file TEST_OFFSETS="0 4294967296" TEST_OPS="writev read write readv" diff --git a/tests/qemu-iotests/015 b/tests/qemu-iotests/015 index eec5387f3d..4d8effd0ae 100755 --- a/tests/qemu-iotests/015 +++ b/tests/qemu-iotests/015 @@ -40,8 +40,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # actually any format that supports snapshots _supported_fmt qcow2 _supported_proto generic -# Internal snapshots are (currently) impossible with refcount_bits=1 -_unsupported_imgopts 'refcount_bits=1[^0-9]' +# Internal snapshots are (currently) impossible with refcount_bits=1, +# and generally impossible with external data files +_unsupported_imgopts 'refcount_bits=1[^0-9]' data_file echo echo "creating image" diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026 index 3430029ed6..a4aa74764f 100755 --- a/tests/qemu-iotests/026 +++ b/tests/qemu-iotests/026 @@ -49,7 +49,10 @@ _supported_cache_modes writethrough none # 32 and 64 bits do not work either, however, due to different leaked cluster # count on error. # Thus, the only remaining option is refcount_bits=16. -_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' +# +# As for data_file, none of the refcount tests can work for it. +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' \ +data_file echo "Errors while writing 128 kB" echo diff --git a/tests/qemu-iotests/029 b/tests/qemu-iotests/029 index 9254ede5e5..2161a4b87a 100755 --- a/tests/qemu-iotests/029 +++ b/tests/qemu-iotests/029 @@ -42,8 +42,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _unsupported_proto vxhs -# Internal snapshots are (currently) impossible with refcount_bits=1 -_unsupported_imgopts 'refcount_bits=1[^0-9]' +# Internal snapshots are (currently) impossible with refcount_bits=1, +# and generally impossible with external data files +_unsupported_imgopts 'refcount_bits=1[^0-9]' data_file offset_size=24 offset_l1_size=36 diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031 index c44fcf91bb..646ecd593f 100755 --- a/tests/qemu-iotests/031 +++ b/tests/qemu-iotests/031 @@ -40,9 +40,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file -# We want to test compat=0.10, which does not support refcount widths -# other than 16 -_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' +# We want to test
Re: [PATCH v2 2/4] Memory: Enable writeback for given memory region
On Thu, 7 Nov 2019 at 16:57, Alex Bennée wrote: > > > Beata Michalska writes: > > > On Wed, 6 Nov 2019 at 12:20, Richard Henderson > > wrote: > >> qemu_log_mask w/ GUEST_ERROR? How do we expect the length to overflow? > > > > In theory it shouldn't, at least with current usage. > > I guess the probe_access will make sure of that. > > This was more of a precaution to enable catching potential/future misuses > > aka debugging purpose. I can get rid of that it that's playing too > > safe. > > If the internal code might get it wrong and that would be a bug then the > g_assert(), if the values are ultimately from the guest then log with > GUEST_ERROR as Richard suggests. ...or consider asserting at this level and making the target specific calling code sanitize and do the GUEST_ERROR logging (it can do a better job of it because it knows what the target-specific operation that the guest just got wrong was). thanks -- PMM
Re: [PATCH for 4.2 v1 1/1] riscv/virt: Increase flash size
On Wed, 06 Nov 2019 16:47:20 PST (-0800), Alistair Francis wrote:
Coreboot developers have requested that they have at least 32MB of flash
to load binaries. We currently have 32MB of flash, but it is split in
two to allow loading two flash binaries. Let's increase the flash size
from 32MB to 64MB to ensure we have a single region that is 32MB.
No QEMU release has include flash in the RISC-V virt machine, so this
isn't a breaking change.
Even if we had, I wouldn't consider it a breaking change because it adds to
the memory map so existing programs will continue to run fine.
Signed-off-by: Alistair Francis
---
hw/riscv/virt.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index cc8f311e6b..23f340df19 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -62,7 +62,7 @@ static const struct MemmapEntry {
[VIRT_PLIC] ={ 0xc00, 0x400 },
[VIRT_UART0] = { 0x1000, 0x100 },
[VIRT_VIRTIO] = { 0x10001000,0x1000 },
-[VIRT_FLASH] = { 0x2000, 0x200 },
+[VIRT_FLASH] = { 0x2000, 0x400 },
[VIRT_DRAM] ={ 0x8000, 0x0 },
[VIRT_PCIE_MMIO] = { 0x4000,0x4000 },
[VIRT_PCIE_PIO] ={ 0x0300,0x0001 },
Reviewed-by: Palmer Dabbelt
I'll include this in my next PR, which should be soon -- I was about to send
it, but figure I should look at my email first :)
Re: [PATCH 4/4] Added tests for close and change of logfile.
Alex Bennée writes:
> Robert Foley writes:
>
>> One test ensures that the logfile handle is still valid even if
>> the logfile is changed during logging.
>> The other test validates that the logfile handle remains valid under
>> the logfile lock even if the logfile is closed.
Also this doesn't see to work:
17:24:31 [alex@zen:~/l/q/b/all] review/rcu-logfile|… 2 + ./tests/test-logging
/logging/parse_range: OK
/logging/parse_path: OK
/logging/logfile_write_path: free(): double free detected in tcache 2
fish: “./tests/test-logging” terminated by signal SIGABRT (Abort)
in gdb
Starting program: /home/alex/lsrc/qemu.git/builds/all/tests/test-logging
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x76f38700 (LWP 28960)]
/logging/parse_range: OK
/logging/parse_path: OK
/logging/logfile_write_path: free(): double free detected in tcache 2
Thread 1 "test-logging" received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x77587535 in __GI_abort () at abort.c:79
#2 0x775de508 in __libc_message (action=action@entry=do_abort,
fmt=fmt@entry=0x776e928d "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3 0x775e4c1a in malloc_printerr (str=str@entry=0x776eaf58
"free(): double free detected in tcache 2") at malloc.c:5341
#4 0x775e66fd in _int_free (av=0x77720c40 ,
p=0x555cac40, have_lock=) at malloc.c:4193
#5 0x555614a8 in qemu_set_log_filename (filename=0x555cb110
"/tmp/qemu-test-logging.RO35A0/qemu_test_log_write0.log", errp=0x7fffdef0)
at /home/alex/lsrc/qemu.git/util/log.c:148
#6 0xd8be in test_logfile_write (data=0x555c7370) at
/home/alex/lsrc/qemu.git/tests/test-logging.c:127
#7 0x77cdc15a in test_case_run (tc=0x555c9c60) at
../../../glib/gtestutils.c:2318
#8 g_test_run_suite_internal (suite=suite@entry=0x555c8a40,
path=path@entry=0x0) at ../../../glib/gtestutils.c:2403
#9 0x77cdc014 in g_test_run_suite_internal
(suite=suite@entry=0x555c8a20, path=path@entry=0x0) at
../../../glib/gtestutils.c:2415
#10 0x77cdc412 in g_test_run_suite (suite=0x555c8a20) at
../../../glib/gtestutils.c:2490
#11 0x77cdc431 in g_test_run () at ../../../glib/gtestutils.c:1755
#12 0xce07 in main (argc=, argv=) at
/home/alex/lsrc/qemu.git/tests/test-logging.c:212
>>
>> Signed-off-by: Robert Foley
>> ---
>> tests/test-logging.c | 74
>> 1 file changed, 74 insertions(+)
>>
>> diff --git a/tests/test-logging.c b/tests/test-logging.c
>> index a12585f70a..a3190ff92c 100644
>> --- a/tests/test-logging.c
>> +++ b/tests/test-logging.c
>> @@ -108,6 +108,76 @@ static void test_parse_path(gconstpointer data)
>> error_free_or_abort();
>> }
>>
>> +static void test_logfile_write(gconstpointer data)
>> +{
>> +QemuLogFile *logfile;
>> +gchar const *dir = data;
>> +Error *err = NULL;
>> +gchar *file_path;
>> +gchar *file_path1;
>
> with g_autofree char *file_path you can avoid the free down bellow.
>
>> +FILE *orig_fd;
>> +
>> +file_path = g_build_filename(dir, "qemu_test_log_write0.log", NULL);
>> +file_path1 = g_build_filename(dir, "qemu_test_log_write1.log", NULL);
>> +
>> +/*
>> + * Test that even if an open file handle is changed,
>> + * our handle remains valid due to RCU.
>> + */
>> +qemu_set_log_filename(file_path, );
>> +g_assert(!err);
>> +rcu_read_lock();
>> +logfile = atomic_rcu_read(_logfile);
>> +orig_fd = logfile->fd;
>> +g_assert(logfile && logfile->fd);
>> +fprintf(logfile->fd, "%s 1st write to file\n", __func__);
>> +fflush(logfile->fd);
>> +
>> +/* Change the logfile and ensure that the handle is still valid. */
>> +qemu_set_log_filename(file_path1, );
>> +g_assert(!err);
>
> Maybe better would be:
>
> logfile2 = atomic_rcu_read(_logfile);
> g_assert(logfile->fd == orig_fd);
> g_assert(logfile2->fd != logfile->fd);
> fprintf(logfile2->fd, "%s 2nd write to file\n", __func__);
> fflush(logfile2->fd);
>
>
>> +g_assert(logfile->fd == orig_fd);
>> +fprintf(logfile->fd, "%s 2nd write to file\n", __func__);
>> +fflush(logfile->fd);
>> +rcu_read_unlock();
>> +
>> +g_free(file_path);
>> +g_free(file_path1);
>> +}
>> +
>> +static void test_logfile_lock(gconstpointer data)
>> +{
>> +FILE *logfile;
>> +gchar const *dir = data;
>> +Error *err = NULL;
>> +gchar *file_path;
>
> g_autofree
>
>> +
>> +file_path = g_build_filename(dir, "qemu_test_logfile_lock0.log", NULL);
>> +
>> +/*
>> + * Test the use of the logfile lock, such
>> + * that even if an open file handle is closed,
>> + * our
Re: [PATCH v8 0/3] RTC support for QEMU RISC-V virt machine
On 11/7/19 5:52 PM, Palmer Dabbelt wrote: On Wed, 06 Nov 2019 03:56:29 PST (-0800), Anup Patel wrote: This series adds RTC device to QEMU RISC-V virt machine. We have selected Goldfish RTC device model for this. It's a pretty simple synthetic device with few MMIO registers and no dependency external clock. The driver for Goldfish RTC is already available in Linux so we just need to enable it in Kconfig for RISCV and also update Linux defconfigs. We have tested this series with Linux-5.4-rc4 plus defconfig changes available in 'goldfish_rtc_v2' branch of: https://github.com/avpatel/linux.git Changes since v7: - Fix broken "stdout-path" in "/chosen" DT node of virt machine Changes since v6: - Rebased on latest QEMU master - Addressed all nit comments from Philippe Mathieu-Daude Changes since v5: - Rebased on latest QEMU master - Added maintainer entry for Goldfish RTC Changes since v4: - Fixed typo in trace event usage - Moved goldfish_rtc.h to correct location Changes since v3: - Address all nit comments from Alistair Changes since v2: - Rebased on RTC code refactoring Changes since v1: - Implemented VMState save/restore callbacks Anup Patel (3): hw: rtc: Add Goldfish RTC device riscv: virt: Use Goldfish RTC device MAINTAINERS: Add maintainer entry for Goldfish RTC MAINTAINERS | 8 + hw/riscv/Kconfig | 1 + hw/riscv/virt.c | 16 ++ hw/rtc/Kconfig | 3 + hw/rtc/Makefile.objs | 1 + hw/rtc/goldfish_rtc.c | 285 ++ hw/rtc/trace-events | 4 + include/hw/riscv/virt.h | 2 + include/hw/rtc/goldfish_rtc.h | 46 ++ 9 files changed, 366 insertions(+) create mode 100644 hw/rtc/goldfish_rtc.c create mode 100644 include/hw/rtc/goldfish_rtc.h Thanks. I've updated the patches on my queue, LMK if there are any more changes! I'm happy with the series, thanks Anup for addressing all the comments. Regards, Phil.
Re: [PATCH for 4.2 v1 1/1] riscv/virt: Increase flash size
On Thu, Nov 7, 2019 at 10:01 AM Peter Maydell wrote: > > On Thu, 7 Nov 2019 at 17:09, Palmer Dabbelt wrote: > > > > On Wed, 06 Nov 2019 16:47:20 PST (-0800), Alistair Francis wrote: > > > Coreboot developers have requested that they have at least 32MB of flash > > > to load binaries. We currently have 32MB of flash, but it is split in > > > two to allow loading two flash binaries. Let's increase the flash size > > > from 32MB to 64MB to ensure we have a single region that is 32MB. > > > > > > No QEMU release has include flash in the RISC-V virt machine, so this > > > isn't a breaking change. > > > > Even if we had, I wouldn't consider it a breaking change because it adds to > > the memory map so existing programs will continue to run fine. > > I have a feeling you may find that some old command lines won't > work any more because they specified a flash contents binary > that was the old 32MB and now it needs to be padded out to 64MB. Yes, that is correct. Everyone using -pflash will need to change the size of their binaries. This was only just merged into QEMU master though and hasn't been in a release so I don't think many people are using it. I only know of two users, one is me and someone from Coreboot who requested the larger size. It doesn't seem like a problem users will see. Alistair > But I haven't tested whether this theory is correct (it will > depend on how the flash contents are specified -- --bios will > be ok, as will loading contents directly as an ELF file or > similar, specifying contents by a -drive option intended to be > consumed by the pflash is the case which likely needs extra padding.) > > thanks > -- PMM
[PATCH v2 1/2] i386: Add missing cpu feature bits in EPYC model
Adds the following missing CPUID bits:
perfctr-core : core performance counter extensions support. Enables the VM
to use extended performance counter support. It enables six
programmable counters instead of 4 counters.
clzero : instruction zeroes out the 64 byte cache line specified in RAX.
xsaveerptr : XSAVE, XSAVE, FXSAVEOPT, XSAVEC, XSAVES always save error
pointers and FXRSTOR, XRSTOR, XRSTORS always restore error
pointers.
ibpb : Indirect Branch Prediction Barrie.
xsaves : XSAVES, XRSTORS and IA32_XSS supported.
Depends on following kernel commits:
40bc47b08b6e ("kvm: x86: Enumerate support for CLZERO instruction")
504ce1954fba ("KVM: x86: Expose XSAVEERPTR to the guest")
52297436199d ("kvm: svm: Update svm_xsaves_supported")
These new features will be added in EPYC-v3. The -cpu help output after the
change.
x86 EPYC-v1 AMD EPYC Processor
x86 EPYC-v2 AMD EPYC Processor (with IBPB)
x86 EPYC-v3 AMD EPYC Processor
Signed-off-by: Babu Moger
---
target/i386/cpu.c | 17 +
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 07cf562d89..6b7b0f8a4b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3116,10 +3116,6 @@ static X86CPUDefinition builtin_x86_defs[] = {
CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_BMI2 | CPUID_7_0_EBX_RDSEED |
CPUID_7_0_EBX_ADX | CPUID_7_0_EBX_SMAP | CPUID_7_0_EBX_CLFLUSHOPT |
CPUID_7_0_EBX_SHA_NI,
-/* Missing: XSAVES (not supported by some Linux versions,
- * including v4.1 to v4.12).
- * KVM doesn't yet expose any XSAVES state save component.
- */
.features[FEAT_XSAVE] =
CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
CPUID_XSAVE_XGETBV1,
@@ -3142,6 +3138,19 @@ static X86CPUDefinition builtin_x86_defs[] = {
{ /* end of list */ }
}
},
+{
+.version = 3,
+.props = (PropValue[]) {
+{ "ibpb", "on" },
+{ "perfctr-core", "on" },
+{ "clzero", "on" },
+{ "xsaveerptr", "on" },
+{ "xsaves", "on" },
+{ "model-id",
+ "AMD EPYC Processor" },
+{ /* end of list */ }
+}
+},
{ /* end of list */ }
}
},
[PATCH v2 2/2] i386: Add 2nd Generation AMD EPYC processors
Adds the support for 2nd Gen AMD EPYC Processors. The model display
name will be EPYC-Rome.
Adds the following new feature bits on top of the feature bits from the
first generation EPYC models.
perfctr-core : core performance counter extensions support. Enables the VM to
use extended performance counter support. It enables six
programmable counters instead of four counters.
clzero : instruction zeroes out the 64 byte cache line specified in RAX.
xsaveerptr : XSAVE, XSAVE, FXSAVEOPT, XSAVEC, XSAVES always save error
pointers and FXRSTOR, XRSTOR, XRSTORS always restore error
pointers.
wbnoinvd : Write back and do not invalidate cache
ibpb : Indirect Branch Prediction Barrier
amd-stibp: Single Thread Indirect Branch Predictor
clwb : Cache Line Write Back and Retain
xsaves : XSAVES, XRSTORS and IA32_XSS support
rdpid: Read Processor ID instruction support
umip : User-Mode Instruction Prevention support
The Reference documents are available at
https://developer.amd.com/wp-content/resources/55803_0.54-PUB.pdf
https://www.amd.com/system/files/TechDocs/24594.pdf
Depends on following kernel commits:
40bc47b08b6e ("kvm: x86: Enumerate support for CLZERO instruction")
504ce1954fba ("KVM: x86: Expose XSAVEERPTR to the guest")
6d61e3c32248 ("kvm: x86: Expose RDPID in KVM_GET_SUPPORTED_CPUID")
52297436199d ("kvm: svm: Update svm_xsaves_supported")
Signed-off-by: Babu Moger
---
target/i386/cpu.c | 102 -
target/i386/cpu.h |2 +
2 files changed, 103 insertions(+), 1 deletion(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6b7b0f8a4b..70afc3fb30 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1133,7 +1133,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] =
{
"clzero", NULL, "xsaveerptr", NULL,
NULL, NULL, NULL, NULL,
NULL, "wbnoinvd", NULL, NULL,
-"ibpb", NULL, NULL, NULL,
+"ibpb", NULL, NULL, "amd-stibp",
NULL, NULL, NULL, NULL,
NULL, NULL, NULL, NULL,
"amd-ssbd", "virt-ssbd", "amd-no-ssb", NULL,
@@ -1796,6 +1796,56 @@ static CPUCaches epyc_cache_info = {
},
};
+static CPUCaches epyc_rome_cache_info = {
+.l1d_cache = &(CPUCacheInfo) {
+.type = DATA_CACHE,
+.level = 1,
+.size = 32 * KiB,
+.line_size = 64,
+.associativity = 8,
+.partitions = 1,
+.sets = 64,
+.lines_per_tag = 1,
+.self_init = 1,
+.no_invd_sharing = true,
+},
+.l1i_cache = &(CPUCacheInfo) {
+.type = INSTRUCTION_CACHE,
+.level = 1,
+.size = 32 * KiB,
+.line_size = 64,
+.associativity = 8,
+.partitions = 1,
+.sets = 64,
+.lines_per_tag = 1,
+.self_init = 1,
+.no_invd_sharing = true,
+},
+.l2_cache = &(CPUCacheInfo) {
+.type = UNIFIED_CACHE,
+.level = 2,
+.size = 512 * KiB,
+.line_size = 64,
+.associativity = 8,
+.partitions = 1,
+.sets = 1024,
+.lines_per_tag = 1,
+},
+.l3_cache = &(CPUCacheInfo) {
+.type = UNIFIED_CACHE,
+.level = 3,
+.size = 16 * MiB,
+.line_size = 64,
+.associativity = 16,
+.partitions = 1,
+.sets = 16384,
+.lines_per_tag = 1,
+.self_init = true,
+.inclusive = true,
+.complex_indexing = true,
+},
+};
+
static X86CPUDefinition builtin_x86_defs[] = {
{
.name = "qemu64",
@@ -3204,6 +3254,56 @@ static X86CPUDefinition builtin_x86_defs[] = {
.model_id = "Hygon Dhyana Processor",
.cache_info = _cache_info,
},
+{
+.name = "EPYC-Rome",
+.level = 0xd,
+.vendor = CPUID_VENDOR_AMD,
+.family = 23,
+.model = 49,
+.stepping = 0,
+.features[FEAT_1_EDX] =
+CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | CPUID_MMX | CPUID_CLFLUSH |
+CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | CPUID_MCA | CPUID_PGE |
+CPUID_MTRR | CPUID_SEP | CPUID_APIC | CPUID_CX8 | CPUID_MCE |
+CPUID_PAE | CPUID_MSR | CPUID_TSC | CPUID_PSE | CPUID_DE |
+CPUID_VME | CPUID_FP87,
+.features[FEAT_1_ECX] =
+CPUID_EXT_RDRAND | CPUID_EXT_F16C | CPUID_EXT_AVX |
+CPUID_EXT_XSAVE | CPUID_EXT_AES | CPUID_EXT_POPCNT |
+CPUID_EXT_MOVBE | CPUID_EXT_SSE42 | CPUID_EXT_SSE41 |
+CPUID_EXT_CX16 | CPUID_EXT_FMA | CPUID_EXT_SSSE3 |
+CPUID_EXT_MONITOR | CPUID_EXT_PCLMULQDQ | CPUID_EXT_SSE3,
+.features[FEAT_8000_0001_EDX] =
+CPUID_EXT2_LM | CPUID_EXT2_RDTSCP | CPUID_EXT2_PDPE1GB |
+CPUID_EXT2_FFXSR | CPUID_EXT2_MMXEXT | CPUID_EXT2_NX |
+CPUID_EXT2_SYSCALL,
+.features[FEAT_8000_0001_ECX] =
+
Re: [PULL 0/3] Block patches for 4.2.0-rc0/4.1.1
On Thu, 7 Nov 2019 at 14:34, Max Reitz wrote: > > The following changes since commit d0f90e1423b4f412adc620eee93e8bfef8af4117: > > Merge remote-tracking branch > 'remotes/kraxel/tags/audio-20191106-pull-request' into staging (2019-11-07 > 09:21:52 +) > > are available in the Git repository at: > > https://github.com/XanClic/qemu.git tags/pull-block-2019-11-07 > > for you to fetch changes up to b7cd2c11f76d27930f53d3cf26d7b695c78d613b: > > iotests: Add test for 4G+ compressed qcow2 write (2019-11-07 14:37:46 +0100) > > > Block patches for 4.2.0-rc0/4.1.1: > - Fix writing to compressed qcow2 images > 4 GB > - Fix size sanity check for qcow2 bitmaps > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2 for any user-visible changes. -- PMM
Re: [PULL 0/1] Usb 20191107 patches
On Thu, 7 Nov 2019 at 08:58, Gerd Hoffmann wrote: > > The following changes since commit 412fbef3d076c43e56451bacb28c4544858c66a3: > > Merge remote-tracking branch > 'remotes/philmd-gitlab/tags/fw_cfg-next-pull-request' into staging > (2019-11-05 20:17:11 +) > > are available in the Git repository at: > > git://git.kraxel.org/qemu tags/usb-20191107-pull-request > > for you to fetch changes up to 1dfe2b91dcb1633d0ba450a8139d53006e700a9b: > > usb-host: add option to allow all resets. (2019-11-06 13:26:04 +0100) > > > usb: fix for usb-host > > > > Gerd Hoffmann (1): > usb-host: add option to allow all resets. > > hw/usb/host-libusb.c | 13 + > 1 file changed, 9 insertions(+), 4 deletions(-) This didn't quite make rc0 but it'll go in for rc1. thanks -- PMM
[PATCH v3 01/22] iotests: s/qocw2/qcow2/
Probably due to blind copy-pasting, we have several instances of "qocw2" in our iotests. Fix them. Reported-by: Maxim Levitsky Signed-off-by: Max Reitz --- tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/061 | 2 +- tests/qemu-iotests/062 | 2 +- tests/qemu-iotests/066 | 2 +- tests/qemu-iotests/068 | 2 +- tests/qemu-iotests/108 | 2 +- tests/qemu-iotests/138 | 2 +- tests/qemu-iotests/261 | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index b91d8321bb..725e58a5a5 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -44,7 +44,7 @@ _filter_io_error() . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file _supported_os Linux diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index 4eac5b83bd..e1b8044630 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file _supported_os Linux diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062 index d5f818fcce..79738b1c26 100755 --- a/tests/qemu-iotests/062 +++ b/tests/qemu-iotests/062 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto generic diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066 index 28f8c98412..cacbdb6ae0 100755 --- a/tests/qemu-iotests/066 +++ b/tests/qemu-iotests/066 @@ -36,7 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto generic diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068 index 22f5ca3ba6..c164ccc64a 100755 --- a/tests/qemu-iotests/068 +++ b/tests/qemu-iotests/068 @@ -36,7 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto generic diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108 index 9c08172237..872a9afec9 100755 --- a/tests/qemu-iotests/108 +++ b/tests/qemu-iotests/108 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file _supported_os Linux diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138 index 6a731370db..8b2f587af0 100755 --- a/tests/qemu-iotests/138 +++ b/tests/qemu-iotests/138 @@ -36,7 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file _supported_os Linux diff --git a/tests/qemu-iotests/261 b/tests/qemu-iotests/261 index fb96bcfbe2..9f2817251f 100755 --- a/tests/qemu-iotests/261 +++ b/tests/qemu-iotests/261 @@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.rc . ./common.filter -# This tests qocw2-specific low-level functionality +# This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file _supported_os Linux -- 2.23.0
[PATCH v3 00/22] iotests: Allow ./check -o data_file
Hi, The cover letter from v1 (explaining the motivation behind this series and the general structure) is here: https://lists.nongnu.org/archive/html/qemu-block/2019-09/msg01323.html For v2, I’ve addressed more of Maxim’s comments: - Patch 1: Added; Maxim noted this problem on patch 5, but that patch doesn’t touch all files that have this mistake, so I decided to make it an extra patch - Patch 20 (now patch 21): - Added TODO comments where it would make sense to at some point split off some cases into an own test file (so they can run with an external data file, where the whole test now has to be skipped) - Fixed the reason why we have to skip 138 with external data files - Disable 261, too (which was added in the meantime) - Some contextual differences in some patches due to the $SOCK_DIR series git-backport-diff against v2: Key: [] : patches are identical [] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/22:[down] 'iotests: s/qocw2/qcow2/' 002/22:[] [--] 'iotests/qcow2.py: Add dump-header-exts' 003/22:[] [--] 'iotests/qcow2.py: Split feature fields into bits' 004/22:[] [--] 'iotests: Add _filter_json_filename' 005/22:[] [--] 'iotests: Filter refcount_order in 036' 006/22:[] [-C] 'iotests: Replace IMGOPTS by _unsupported_imgopts' 007/22:[] [--] 'iotests: Drop compat=1.1 in 050' 008/22:[] [--] 'iotests: Let _make_test_img parse its parameters' 009/22:[] [--] 'iotests: Add -o and --no-opts to _make_test_img' 010/22:[] [--] 'iotests: Inject space into -ocompat=0.10 in 051' 011/22:[] [--] 'iotests: Replace IMGOPTS= by -o' 012/22:[] [--] 'iotests: Replace IMGOPTS='' by --no-opts' 013/22:[] [-C] 'iotests: Drop IMGOPTS use in 267' 014/22:[] [--] 'iotests: Avoid qemu-img create' 015/22:[] [-C] 'iotests: Use _rm_test_img for deleting test images' 016/22:[] [--] 'iotests: Avoid cp/mv of test images' 017/22:[] [--] 'iotests: Make 091 work with data_file' 018/22:[] [--] 'iotests: Make 110 work with data_file' 019/22:[] [--] 'iotests: Make 137 work with data_file' 020/22:[] [--] 'iotests: Make 198 work with data_file' 021/22:[0010] [FC] 'iotests: Disable data_file where it cannot be used' 022/22:[] [-C] 'iotests: Allow check -o data_file' Max Reitz (22): iotests: s/qocw2/qcow2/ iotests/qcow2.py: Add dump-header-exts iotests/qcow2.py: Split feature fields into bits iotests: Add _filter_json_filename iotests: Filter refcount_order in 036 iotests: Replace IMGOPTS by _unsupported_imgopts iotests: Drop compat=1.1 in 050 iotests: Let _make_test_img parse its parameters iotests: Add -o and --no-opts to _make_test_img iotests: Inject space into -ocompat=0.10 in 051 iotests: Replace IMGOPTS= by -o iotests: Replace IMGOPTS='' by --no-opts iotests: Drop IMGOPTS use in 267 iotests: Avoid qemu-img create iotests: Use _rm_test_img for deleting test images iotests: Avoid cp/mv of test images iotests: Make 091 work with data_file iotests: Make 110 work with data_file iotests: Make 137 work with data_file iotests: Make 198 work with data_file iotests: Disable data_file where it cannot be used iotests: Allow check -o data_file tests/qemu-iotests/005 | 2 +- tests/qemu-iotests/007 | 5 ++- tests/qemu-iotests/014 | 2 + tests/qemu-iotests/015 | 5 ++- tests/qemu-iotests/019 | 6 +-- tests/qemu-iotests/020 | 6 +-- tests/qemu-iotests/024 | 10 ++--- tests/qemu-iotests/026 | 5 ++- tests/qemu-iotests/028 | 2 +- tests/qemu-iotests/029 | 7 ++-- tests/qemu-iotests/031 | 9 ++-- tests/qemu-iotests/031.out | 36 tests/qemu-iotests/036 | 15 --- tests/qemu-iotests/036.out | 66 - tests/qemu-iotests/039 | 27 +--- tests/qemu-iotests/039.out | 22 +- tests/qemu-iotests/043 | 4 +- tests/qemu-iotests/046 | 2 + tests/qemu-iotests/048 | 4 +- tests/qemu-iotests/050 | 8 +--- tests/qemu-iotests/051 | 7 ++-- tests/qemu-iotests/053 | 4 +- tests/qemu-iotests/058 | 7 ++-- tests/qemu-iotests/059 | 20 - tests/qemu-iotests/060 | 14 --- tests/qemu-iotests/060.out | 20 - tests/qemu-iotests/061 | 63 +++- tests/qemu-iotests/061.out | 72 tests/qemu-iotests/062 | 5 ++- tests/qemu-iotests/063 | 18 tests/qemu-iotests/063.out | 3 +- tests/qemu-iotests/066 | 7 +++- tests/qemu-iotests/067 | 6 ++- tests/qemu-iotests/068 | 6 ++- tests/qemu-iotests/069
[PATCH v3 04/22] iotests: Add _filter_json_filename
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/common.filter | 24
1 file changed, 24 insertions(+)
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index f870e00e44..8a0169f19a 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -230,5 +230,29 @@ _filter_qmp_empty_return()
grep -v '{"return": {}}'
}
+_filter_json_filename()
+{
+$PYTHON -c 'import sys
+result, *fnames = sys.stdin.read().split("json:{")
+depth = 0
+for fname in fnames:
+depth += 1 # For the opening brace in the split separator
+for chr_i, chr in enumerate(fname):
+if chr == "{":
+depth += 1
+elif chr == "}":
+depth -= 1
+if depth == 0:
+break
+
+# json:{} filenames may be nested; filter out everything from
+# inside the outermost one
+if depth == 0:
+chr_i += 1 # First character past the filename
+result += "json:{ /* filtered */ }" + fname[chr_i:]
+
+sys.stdout.write(result)'
+}
+
# make sure this script returns success
true
--
2.23.0
[PATCH v3 02/22] iotests/qcow2.py: Add dump-header-exts
This is useful for tests that want to whitelist fields from dump-header (with grep) but still print all header extensions. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/qcow2.py | 5 + 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py index b392972d1b..d813b4fc81 100755 --- a/tests/qemu-iotests/qcow2.py +++ b/tests/qemu-iotests/qcow2.py @@ -154,6 +154,10 @@ def cmd_dump_header(fd): h.dump() h.dump_extensions() +def cmd_dump_header_exts(fd): +h = QcowHeader(fd) +h.dump_extensions() + def cmd_set_header(fd, name, value): try: value = int(value, 0) @@ -230,6 +234,7 @@ def cmd_set_feature_bit(fd, group, bit): cmds = [ [ 'dump-header', cmd_dump_header, 0, 'Dump image header and header extensions' ], +[ 'dump-header-exts', cmd_dump_header_exts, 0, 'Dump image header extensions' ], [ 'set-header', cmd_set_header, 2, 'Set a field in the header'], [ 'add-header-ext', cmd_add_header_ext, 2, 'Add a header extension' ], [ 'add-header-ext-stdio', cmd_add_header_ext_stdio, 1, 'Add a header extension, data from stdin' ], -- 2.23.0
[PATCH v3 07/22] iotests: Drop compat=1.1 in 050
IMGOPTS can never be empty for qcow2, because the check scripts adds compat=1.1 unless the user specified any compat option themselves. Thus, this block does not do anything and can be dropped. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/050 | 4 1 file changed, 4 deletions(-) diff --git a/tests/qemu-iotests/050 b/tests/qemu-iotests/050 index 211fc00797..272ecab195 100755 --- a/tests/qemu-iotests/050 +++ b/tests/qemu-iotests/050 @@ -41,10 +41,6 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 qed _supported_proto file -if test "$IMGFMT" = qcow2 && test $IMGOPTS = ""; then - IMGOPTS=compat=1.1 -fi - echo echo "== Creating images ==" -- 2.23.0
[PATCH v3 09/22] iotests: Add -o and --no-opts to _make_test_img
Blindly overriding IMGOPTS is suboptimal as this discards user-specified
options. Whatever options the test needs should simply be appended.
Some tests do this (with IMGOPTS=$(_optstr_add "$IMGOPTS" "...")), but
that is cumbersome. It’s simpler to just give _make_test_img an -o
parameter with which tests can add options.
Some tests actually must override the user-specified options, though,
for example when creating an image in a different format than the test
$IMGFMT. For such cases, --no-opts allows clearing the current option
list.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/common.rc | 13 +
1 file changed, 13 insertions(+)
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index e12216b5f2..a623485f96 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -307,6 +307,7 @@ _make_test_img()
local use_backing=0
local backing_file=""
local object_options=""
+local opts_param=false
local misc_params=()
if [ -n "$TEST_IMG_FILE" ]; then
@@ -327,6 +328,10 @@ _make_test_img()
if [ "$use_backing" = "1" -a -z "$backing_file" ]; then
backing_file=$param
continue
+elif $opts_param; then
+optstr=$(_optstr_add "$optstr" "$param")
+opts_param=false
+continue
fi
case "$param" in
@@ -334,6 +339,14 @@ _make_test_img()
use_backing=1
;;
+-o)
+opts_param=true
+;;
+
+--no-opts)
+optstr=""
+;;
+
*)
misc_params=("${misc_params[@]}" "$param")
;;
--
2.23.0
[PATCH v3 03/22] iotests/qcow2.py: Split feature fields into bits
Print the feature fields as a set of bits so that filtering is easier. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/031.out | 36 +-- tests/qemu-iotests/036.out | 18 +- tests/qemu-iotests/039.out | 22 ++-- tests/qemu-iotests/060.out | 20 +-- tests/qemu-iotests/061.out | 72 ++--- tests/qemu-iotests/137.out | 2 +- tests/qemu-iotests/qcow2.py | 18 +++--- 7 files changed, 99 insertions(+), 89 deletions(-) diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out index 68a74d03b9..d535e407bc 100644 --- a/tests/qemu-iotests/031.out +++ b/tests/qemu-iotests/031.out @@ -18,9 +18,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 72 @@ -46,9 +46,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 72 @@ -74,9 +74,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 72 @@ -109,9 +109,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 104 @@ -142,9 +142,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 104 @@ -175,9 +175,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 104 diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index e489b44386..15229a9604 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -16,9 +16,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x8000 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [63] +compatible_features [] +autoclear_features[] refcount_order4 header_length 104 @@ -50,9 +50,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x8000 +incompatible_features [] +compatible_features [] +autoclear_features[63] refcount_order4 header_length 104 @@ -78,9 +78,9 @@ refcount_table_offset 0x1 refcount_table_clusters 1 nb_snapshots 0 snapshot_offset 0x0 -incompatible_features 0x0 -compatible_features 0x0 -autoclear_features0x0 +incompatible_features [] +compatible_features [] +autoclear_features[] refcount_order4 header_length 104 diff --git a/tests/qemu-iotests/039.out b/tests/qemu-iotests/039.out index 2e356d51b6..bdafa3ace3 100644 --- a/tests/qemu-iotests/039.out +++ b/tests/qemu-iotests/039.out @@ -4,7 +4,7 @@ QA output created by 039 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 wrote 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -incompatible_features 0x0 +incompatible_features [] No errors were found on the image. == Creating a dirty image file == @@ -12,7 +12,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 wrote 512/512 bytes at offset 0 512
[PATCH v3 05/22] iotests: Filter refcount_order in 036
This test can run just fine with other values for refcount_bits, so we should filter the value from qcow2.py's dump-header. In fact, we can filter everything but the feature bits and header extensions, because that is what the test is about. (036 currently ignores user-specified image options, but that will be fixed in the next patch.) Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/036 | 9 --- tests/qemu-iotests/036.out | 48 -- 2 files changed, 6 insertions(+), 51 deletions(-) diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036 index f06ff67408..5f929ad3be 100755 --- a/tests/qemu-iotests/036 +++ b/tests/qemu-iotests/036 @@ -55,7 +55,8 @@ $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 63 # Without feature table $PYTHON qcow2.py "$TEST_IMG" del-header-ext 0x6803f857 -$PYTHON qcow2.py "$TEST_IMG" dump-header +$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features +$PYTHON qcow2.py "$TEST_IMG" dump-header-exts _img_info # With feature table containing bit 63 @@ -103,14 +104,16 @@ echo === Create image with unknown autoclear feature bit === echo _make_test_img 64M $PYTHON qcow2.py "$TEST_IMG" set-feature-bit autoclear 63 -$PYTHON qcow2.py "$TEST_IMG" dump-header +$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features +$PYTHON qcow2.py "$TEST_IMG" dump-header-exts echo echo === Repair image === echo _check_test_img -r all -$PYTHON qcow2.py "$TEST_IMG" dump-header +$PYTHON qcow2.py "$TEST_IMG" dump-header | grep features +$PYTHON qcow2.py "$TEST_IMG" dump-header-exts # success, all done echo "*** done" diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index 15229a9604..0b52b934e1 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -3,25 +3,9 @@ QA output created by 036 === Image with unknown incompatible feature bit === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 -magic 0x514649fb -version 3 -backing_file_offset 0x0 -backing_file_size 0x0 -cluster_bits 16 -size 67108864 -crypt_method 0 -l1_size 1 -l1_table_offset 0x3 -refcount_table_offset 0x1 -refcount_table_clusters 1 -nb_snapshots 0 -snapshot_offset 0x0 incompatible_features [63] compatible_features [] autoclear_features[] -refcount_order4 -header_length 104 - qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Unsupported IMGFMT feature(s): Unknown incompatible feature: 8000 qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Unsupported IMGFMT feature(s): Test feature @@ -37,25 +21,9 @@ qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Unsupported IMGFMT feature(s): tes === Create image with unknown autoclear feature bit === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 -magic 0x514649fb -version 3 -backing_file_offset 0x0 -backing_file_size 0x0 -cluster_bits 16 -size 67108864 -crypt_method 0 -l1_size 1 -l1_table_offset 0x3 -refcount_table_offset 0x1 -refcount_table_clusters 1 -nb_snapshots 0 -snapshot_offset 0x0 incompatible_features [] compatible_features [] autoclear_features[63] -refcount_order4 -header_length 104 - Header extension: magic 0x6803f857 length192 @@ -65,25 +33,9 @@ data === Repair image === No errors were found on the image. -magic 0x514649fb -version 3 -backing_file_offset 0x0 -backing_file_size 0x0 -cluster_bits 16 -size 67108864 -crypt_method 0 -l1_size 1 -l1_table_offset 0x3 -refcount_table_offset 0x1 -refcount_table_clusters 1 -nb_snapshots 0 -snapshot_offset 0x0 incompatible_features [] compatible_features [] autoclear_features[] -refcount_order4 -header_length 104 - Header extension: magic 0x6803f857 length192 -- 2.23.0
[PATCH v3 22/22] iotests: Allow check -o data_file
The problem with allowing the data_file option is that you want to use a
different data file per image used in the test. Therefore, we need to
allow patterns like -o data_file='$TEST_IMG.data_file'.
Then, we need to filter it out from qemu-img map, qemu-img create, and
remove the data file in _rm_test_img.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/common.filter | 23 +--
tests/qemu-iotests/common.rc | 22 +-
2 files changed, 42 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 8a0169f19a..6902f9d94b 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -122,7 +122,13 @@ _filter_actual_image_size()
# replace driver-specific options in the "Formatting..." line
_filter_img_create()
{
-$SED -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \
+data_file_filter=()
+if data_file=$(_get_data_file "$TEST_IMG"); then
+data_file_filter=(-e "s# data_file=$data_file##")
+fi
+
+$SED "${data_file_filter[@]}" \
+-e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \
-e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
-e "s#$TEST_DIR#TEST_DIR#g" \
-e "s#$SOCK_DIR#SOCK_DIR#g" \
@@ -207,9 +213,22 @@ _filter_img_info()
# human and json output
_filter_qemu_img_map()
{
+# Assuming the data_file value in $IMGOPTS contains a '$TEST_IMG',
+# create a filter that replaces the data file name by $TEST_IMG.
+# Example:
+# In $IMGOPTS: 'data_file=$TEST_IMG.data_file'
+# Then data_file_pattern == '\(.*\).data_file'
+# And data_file_filter == -e 's#\(.*\).data_file#\1#
+data_file_filter=()
+if data_file_pattern=$(_get_data_file '\\(.*\\)'); then
+data_file_filter=(-e "s#$data_file_pattern#\\1#")
+fi
+
$SED -e 's/\([0-9a-fx]* *[0-9a-fx]* *\)[0-9a-fx]* */\1/g' \
-e 's/"offset": [0-9]\+/"offset": OFFSET/g' \
--e 's/Mapped to *//' | _filter_testdir | _filter_imgfmt
+-e 's/Mapped to *//' \
+"${data_file_filter[@]}" \
+| _filter_testdir | _filter_imgfmt
}
_filter_nbd()
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index a623485f96..a07a10a305 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -297,6 +297,20 @@ _stop_nbd_server()
fi
}
+# Gets the data_file value from IMGOPTS and replaces the '$TEST_IMG'
+# pattern by '$1'
+# Caution: The replacement is done with sed, so $1 must be escaped
+# properly. (The delimiter is '#'.)
+_get_data_file()
+{
+if ! echo "$IMGOPTS" | grep -q 'data_file='; then
+return 1
+fi
+
+echo "$IMGOPTS" | sed -e 's/.*data_file=\([^,]*\).*/\1/' \
+| sed -e "s#\\\$TEST_IMG#$1#"
+}
+
_make_test_img()
{
# extra qemu-img options can be added by tests
@@ -317,7 +331,8 @@ _make_test_img()
fi
if [ -n "$IMGOPTS" ]; then
-optstr=$(_optstr_add "$optstr" "$IMGOPTS")
+imgopts_expanded=$(echo "$IMGOPTS" | sed -e
"s#\\\$TEST_IMG#$img_name#")
+optstr=$(_optstr_add "$optstr" "$imgopts_expanded")
fi
if [ -n "$IMGKEYSECRET" ]; then
object_options="--object secret,id=keysec0,data=$IMGKEYSECRET"
@@ -396,6 +411,11 @@ _rm_test_img()
# Remove all the extents for vmdk
"$QEMU_IMG" info "$img" 2>/dev/null | grep 'filename:' | cut -f 2 -d: \
| xargs -I {} rm -f "{}"
+elif [ "$IMGFMT" = "qcow2" ]; then
+# Remove external data file
+if data_file=$(_get_data_file "$img"); then
+rm -f "$data_file"
+fi
fi
rm -f "$img"
}
--
2.23.0
Re: [PATCH 2/5] ipmi: Add support to customize OEM functions
On Sun, Oct 27, 2019 at 01:33:47PM -0500, Corey Minyard wrote:
> On Sun, Oct 27, 2019 at 06:47:39PM +0100, David Gibson wrote:
> > On Mon, Oct 21, 2019 at 09:30:17AM -0500, Corey Minyard wrote:
> > > On Mon, Oct 21, 2019 at 03:12:12PM +0200, Cédric Le Goater wrote:
> > > > The routine ipmi_register_oem_netfn() lets external modules register
> > > > command handlers for OEM functions. Required for the PowerNV machine.
> > >
> > > Comments inline.
> > >
> > > >
> > > > Cc: Corey Minyard
> > > > Signed-off-by: Cédric Le Goater
> > > > ---
> > > > include/hw/ipmi/ipmi.h | 36
> > > > hw/ipmi/ipmi_bmc_sim.c | 41 ++---
> > > > 2 files changed, 42 insertions(+), 35 deletions(-)
> > > >
> > > > diff --git a/include/hw/ipmi/ipmi.h b/include/hw/ipmi/ipmi.h
> > > > index 6f2413b39b4a..cb7203b06767 100644
> > > > --- a/include/hw/ipmi/ipmi.h
> > > > +++ b/include/hw/ipmi/ipmi.h
> > > > @@ -265,4 +265,40 @@ int ipmi_bmc_sdr_find(IPMIBmc *b, uint16_t recid,
> > > >const struct ipmi_sdr_compact **sdr, uint16_t
> > > > *nextrec);
> > > > void ipmi_bmc_gen_event(IPMIBmc *b, uint8_t *evt, bool log);
> > > >
> > > > +typedef struct IPMIBmcSim IPMIBmcSim;
> > >
> > > This type isn't very useful outside of the simulator, but changes for
> > > that can come as they are needed. I don't see an easy way to avoid
> > > putting it here.
> > >
> > > > +
> > > > +typedef struct RspBuffer {
> > > > +uint8_t buffer[MAX_IPMI_MSG_SIZE];
> > > > +unsigned int len;
> > > > +} RspBuffer;
> > > > +
> > > > +static inline void rsp_buffer_set_error(RspBuffer *rsp, uint8_t byte)
> > > > +{
> > > > +rsp->buffer[2] = byte;
> > > > +}
> > > > +
> > > > +/* Add a byte to the response. */
> > > > +static inline void rsp_buffer_push(RspBuffer *rsp, uint8_t byte)
> > > > +{
> > > > +if (rsp->len >= sizeof(rsp->buffer)) {
> > > > +rsp_buffer_set_error(rsp, IPMI_CC_REQUEST_DATA_TRUNCATED);
> > > > +return;
> > > > +}
> > > > +rsp->buffer[rsp->len++] = byte;
> > > > +}
> > > > +
> > > > +typedef struct IPMICmdHandler {
> > > > +void (*cmd_handler)(IPMIBmcSim *s,
> > > > +uint8_t *cmd, unsigned int cmd_len,
> > > > +RspBuffer *rsp);
> > > > +unsigned int cmd_len_min;
> > > > +} IPMICmdHandler;
> > > > +
> > > > +typedef struct IPMINetfn {
> > > > +unsigned int cmd_nums;
> > > > +const IPMICmdHandler *cmd_handlers;
> > > > +} IPMINetfn;
> > > > +
> > > > +int ipmi_register_oem_netfn(IPMIBmc *b, const IPMINetfn *netfnd);
> > > > +
> > > > #endif
> > > > diff --git a/hw/ipmi/ipmi_bmc_sim.c b/hw/ipmi/ipmi_bmc_sim.c
> > > > index 71e56f3b13d1..770aace55b08 100644
> > > > --- a/hw/ipmi/ipmi_bmc_sim.c
> > > > +++ b/hw/ipmi/ipmi_bmc_sim.c
> > > > @@ -98,6 +98,7 @@
> > > > #define IPMI_CMD_GET_SEL_TIME 0x48
> > > > #define IPMI_CMD_SET_SEL_TIME 0x49
> > > >
> > > > +#define IPMI_NETFN_OEM0x3a
> > > >
> > > > /* Same as a timespec struct. */
> > > > struct ipmi_time {
> > > > @@ -167,23 +168,8 @@ typedef struct IPMISensor {
> > > > #define MAX_SENSORS 20
> > > > #define IPMI_WATCHDOG_SENSOR 0
> > > >
> > > > -typedef struct IPMIBmcSim IPMIBmcSim;
> > > > -typedef struct RspBuffer RspBuffer;
> > > > -
> > > > #define MAX_NETFNS 64
> > > >
> > > > -typedef struct IPMICmdHandler {
> > > > -void (*cmd_handler)(IPMIBmcSim *s,
> > > > -uint8_t *cmd, unsigned int cmd_len,
> > > > -RspBuffer *rsp);
> > > > -unsigned int cmd_len_min;
> > > > -} IPMICmdHandler;
> > > > -
> > > > -typedef struct IPMINetfn {
> > > > -unsigned int cmd_nums;
> > > > -const IPMICmdHandler *cmd_handlers;
> > > > -} IPMINetfn;
> > > > -
> > > > typedef struct IPMIRcvBufEntry {
> > > > QTAILQ_ENTRY(IPMIRcvBufEntry) entry;
> > > > uint8_t len;
> > > > @@ -279,28 +265,8 @@ struct IPMIBmcSim {
> > > > #define IPMI_BMC_WATCHDOG_ACTION_POWER_DOWN 2
> > > > #define IPMI_BMC_WATCHDOG_ACTION_POWER_CYCLE 3
> > > >
> > > > -struct RspBuffer {
> > > > -uint8_t buffer[MAX_IPMI_MSG_SIZE];
> > > > -unsigned int len;
> > > > -};
> > > > -
> > > > #define RSP_BUFFER_INITIALIZER { }
> > > >
> > > > -static inline void rsp_buffer_set_error(RspBuffer *rsp, uint8_t byte)
> > > > -{
> > > > -rsp->buffer[2] = byte;
> > > > -}
> > > > -
> > > > -/* Add a byte to the response. */
> > > > -static inline void rsp_buffer_push(RspBuffer *rsp, uint8_t byte)
> > > > -{
> > > > -if (rsp->len >= sizeof(rsp->buffer)) {
> > > > -rsp_buffer_set_error(rsp, IPMI_CC_REQUEST_DATA_TRUNCATED);
> > > > -return;
> > > > -}
> > > > -rsp->buffer[rsp->len++] = byte;
> > > > -}
> > > > -
> > > > static inline void rsp_buffer_pushmore(RspBuffer *rsp, uint8_t *bytes,
> > > > unsigned int n)
> > > > {
> > >
Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test
Le 07/11/2019 à 17:38, Cleber Rosa a écrit : > > > - Original Message - >> From: "Eric Blake" >> To: "Cleber Rosa" , qemu-devel@nongnu.org >> Cc: "Peter Maydell" , "Eduardo Habkost" >> , "Philippe Mathieu-Daudé" >> , "Wainer dos Santos Moschetta" , >> "Laurent Vivier" , >> "Willian Rampazzo" , "Philippe Mathieu-Daudé" >> >> Sent: Thursday, November 7, 2019 10:43:08 AM >> Subject: Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 >> test >> >> On 10/29/19 6:23 PM, Cleber Rosa wrote: >>> The boot_linux_console.py:BootLinuxConsole.test_m68k_q800 was very >>> recently merged, but between its last review and now, the Kernel >>> package used went missing. >>> >> >> meta-question: Why was this series posted in-reply-to the pull request, >> rather than as a new top-level thread? I nearly missed it because I >> don't expect to see unreviewed patches buried in threading like that. >> My workflow would have been to post the series in isolation, then >> manually reply to the pull request to mention the message-id of the >> related series proposed as a followup. >> > > Hi Eric, > > That was my attempt to signal that it was a fix to something which had *just* > being merged as part of that pull request (though now caused by it). > > I basically did not know how to act properly, so I thank you for the workflow > suggestion. I'll certainly follow it next time. IMHO, you should send your series and then replies to the pull request to tell you have sent your series that fixes the patch in the pull request, or vice-versa. But your series has been queued by Alex, so there is no problem... Thanks, Laurent
[Bug 1851664] Re: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine
Presumably w-bits (aw-bits?) implies using intel-iommu, there's a opportunity for the vfio iommu backend to return -ENOSPC (-28) if we exceed the default number of in-flight DMA mappings per container. The default limit is 65535. You can try increasing this by changing the dma_entry_limit module option on the vfio_iommu_type1 module. Note that in a typical vIOMMU config there's a container per device, so the number of VFs attached is possibly not a factor. It is however a lot of DMA mappings for a single device if this is the issue and you'd generally want to boot the guest with iommu=pt in order to have reasonable assigned device performance with a vIOMMU, which would also greatly reduce the number of mappings. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1851664 Title: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine Status in QEMU: Incomplete Bug description: We are trying to attach 6 VF's to the guest machine on 4.1.1 qemu emulator. We are observing "VFIO_MAP_DMA : -28" error. We are using w-bits=48 bits while lunching VM. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1851664/+subscriptions
[PATCH v2 0/2] Add support for 2nd generation AMD EPYC processors
The following series adds the support for 2nd generation AMD EPYC Processors on qemu guests. The model display name for 2nd generation will be EPYC-Rome. Also fixes few missed cpu feature bits in 1st generation EPYC models. The Reference documents are available at https://developer.amd.com/wp-content/resources/55803_0.54-PUB.pdf https://www.amd.com/system/files/TechDocs/24594.pdf --- v2: Used the versioned CPU models instead of machine-type-based CPU compatibility (commented by Eduardo). Babu Moger (2): i386: Add missing cpu feature bits in EPYC model i386: Add 2nd Generation AMD EPYC processors target/i386/cpu.c | 119 +++-- target/i386/cpu.h |2 + 2 files changed, 116 insertions(+), 5 deletions(-) --
Re: [PATCH v7 0/8] Acceptance test: Add "boot_linux" acceptance test
Hi Cleber, On 11/4/19 1:13 PM, Cleber Rosa wrote: This acceptance test, validates that a full blown Linux guest can successfully boot in QEMU. In this specific case, the guest chosen is Fedora version 31. * x86_64, pc and q35 machine types, with and without kvm as an accellerator * aarch64 and virt machine type, with and without kvm as an accellerator * ppc64 and pseries machine type * s390x and s390-ccw-virtio machine type This has been tested on x86_64 and ppc64le hosts and has been running reliably (in my experience) on Travis CI. Some hopefully useful pointers for reviewers: = Git Info: - URI: https://github.com/clebergnu/qemu/tree/test_boot_linux_v7 - Remote: https://github.com/clebergnu/qemu - Branch: test_boot_linux_v7 Travis CI Info: - Build: https://travis-ci.org/clebergnu/qemu/builds/606191094 - Job: https://travis-ci.org/clebergnu/qemu/jobs/606191120 Previous version: - v6: https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg01202.html - v5: https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04652.html - v4: https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg02032.html - v3: https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01677.html - v2: https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg04318.html - v1: http://lists.nongnu.org/archive/html/qemu-devel/2018-09/msg02530.html Changes from v6: * Bumped Fedora to most recently released version (31). * Included new architectures (ppc64 and s390x), consolidating all tests into the same commit. * New commit: "Acceptance tests: use avocado tags for machine type" * New commit: "Acceptance tests: introduce utility method for tags unique vals" It seems 02 and 03 are re-sending of patches originally sent in the series with Message-Id: 20190924194501.9303-1-cr...@redhat.com. On the original thread you can find my reviews. - Wainer * New commit: "Acceptance test x86_cpu_model_versions: use default vm", needed to normalize the use of the machine type tags * Added a lot of leniency to the test setup (and reliability to the test/job), canceling the test if there are any failures while downloading/preparing the boot images. * Made use of Avocado's data drainer a regular feature (dropped the commit with RFC) and squashed it. * Bumped pycdlib version to 1.8.0 * Dropped explicit "--enable-slirp=git" (added on v5) to Travis CI configure line, as the default configuration on Travis CI now results in user networking capabilities. Changes from v5: * Added explicit "--enable-slirp=git" to Travis CI configure line, as these tests depend on "-netdev user" like networking. * Bumped Fedora to most recently released version (30). * Changed "checksum" parameter to 'sha256' and use the same hashes as provided by the Fedora project (instead of using Avocado's default sha1 and compute and use a different hash value). * New commit: Add "boot_linux" test for aarch64 and virt machine type * New commit: [RFC]: use Avocado data drainer for console logging Changes from v4: * New commit "Acceptance tests: use relative location for tests" * New commit "Acceptance tests: keep a stable reference to the QEMU build dir" * Pinned the Fedora 29 image by adding a checksum. The goal is to never allow more than one component to change at a time (the one allowed to change is QEMU itself). Updates to the image should be manual. (Based on comments from Cornelia) * Moved the downloading of the Fedora 29 cloud image to the test setUp() method, canceling the test if the image can not be downloaded. * Removed the ":avocado: enable" tag, given that Avocado versions 68.0 and later operate on a "recursive by default" manner, that is able to correctly identify this as an Avocado test. Changes from v3: * New patch "Acceptance tests: depend on qemu-img" Known Issues on v3 (no longer applicable): == * A recent TCG performance regression[1] affects this test in a number of ways: - The test execution may timeout by itself - The generation of SSH host keys in the guest's first boot is also affected (possibly also a timeout) - The cloud-init "phone home" feature attempts to read the host keys and fails, causing the test to timeout and fail These are not observed anymore once the fix[2] is applied. [1] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg00338.html [2] - https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg01129.html Changes from v2: * Updated the tag to include the "arch:" key, in a similar fashion as to the tests in the "Acceptance Tests: target architecture support": -
Re: [PATCH for 4.2 v1 1/1] riscv/virt: Increase flash size
Alistair Francis writes: > On Thu, Nov 7, 2019 at 10:01 AM Peter Maydell > wrote: >> >> On Thu, 7 Nov 2019 at 17:09, Palmer Dabbelt wrote: >> > >> > On Wed, 06 Nov 2019 16:47:20 PST (-0800), Alistair Francis wrote: >> > > Coreboot developers have requested that they have at least 32MB of flash >> > > to load binaries. We currently have 32MB of flash, but it is split in >> > > two to allow loading two flash binaries. Let's increase the flash size >> > > from 32MB to 64MB to ensure we have a single region that is 32MB. >> > > >> > > No QEMU release has include flash in the RISC-V virt machine, so this >> > > isn't a breaking change. >> > >> > Even if we had, I wouldn't consider it a breaking change because it adds to >> > the memory map so existing programs will continue to run fine. >> >> I have a feeling you may find that some old command lines won't >> work any more because they specified a flash contents binary >> that was the old 32MB and now it needs to be padded out to 64MB. > > Yes, that is correct. Everyone using -pflash will need to change the > size of their binaries. This was only just merged into QEMU master > though and hasn't been in a release so I don't think many people are > using it. > > I only know of two users, one is me and someone from Coreboot who > requested the larger size. It doesn't seem like a problem users will > see. At least the error message they get will be more informative now ;-) -- Alex Bennée
Re: [PULL 0/1] Usb 20191107 patches
On Thu, 7 Nov 2019 at 18:57, Philippe Mathieu-Daudé wrote: > > Hi Peter, > > On 11/7/19 7:26 PM, Peter Maydell wrote: > > On Thu, 7 Nov 2019 at 08:58, Gerd Hoffmann wrote: > >> > >> The following changes since commit > >> 412fbef3d076c43e56451bacb28c4544858c66a3: > >> > >>Merge remote-tracking branch > >> 'remotes/philmd-gitlab/tags/fw_cfg-next-pull-request' into staging > >> (2019-11-05 20:17:11 +) > >> > >> are available in the Git repository at: > >> > >>git://git.kraxel.org/qemu tags/usb-20191107-pull-request > >> > >> for you to fetch changes up to 1dfe2b91dcb1633d0ba450a8139d53006e700a9b: > >> > >>usb-host: add option to allow all resets. (2019-11-06 13:26:04 +0100) > >> > >> > >> usb: fix for usb-host > >> > >> > >> > >> Gerd Hoffmann (1): > >>usb-host: add option to allow all resets. > >> > >> hw/usb/host-libusb.c | 13 + > >> 1 file changed, 9 insertions(+), 4 deletions(-) > > > > This didn't quite make rc0 but it'll go in for rc1. > > Won't this make bisection confusing? No, why should it? thanks -- PMM
Re: QEMU HTML documentation now on qemu.org
On Thu, 7 Nov 2019 at 09:29, Stefan Hajnoczi wrote: > > On Wed, Nov 6, 2019 at 5:21 PM Stefan Hajnoczi wrote: > > Hi, > > You can now access the latest QEMU HTML documentation built from > > qemu.git/master nightly at: > > > > https://wiki.qemu.org/docs/qemu-doc.html > > https://wiki.qemu.org/docs/qemu-qmp-ref.html > > https://wiki.qemu.org/docs/qemu-ga-ref.html > > ...as well as interop/ and specs/ > > > > Feel free to link to the documentation from the QEMU website and/or > > wiki! > > > > The container image that builds the docs is here: > > > > https://github.com/stefanha/qemu-docs > > > > It is hosted on QEMU's Rackspace cloud account. > > I forgot to add Markus. > > I hope this helps the QEMU documentation effort. I currently do not > have plans to work on this further. You are welcome to send pull > requests to the qemu-docs container image repo or just ask me and I'll > make changes. Yep, it's definitely helpful. One simple thing we could perhaps add is a hand-written "top level" page which just has links to the various documentation URLs you list above. (Possibly this should be done in-tree so the in-tree docs also have a top level landing page.) thanks -- PMM
[Bug 1851664] Re: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine
Please provide information how you started QEMU, and some information about your PCI device (e.g. the output of lspci). ** Information type changed from Private Security to Public ** Changed in: qemu Status: New => Incomplete -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1851664 Title: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine Status in QEMU: Incomplete Bug description: We are trying to attach 6 VF's to the guest machine on 4.1.1 qemu emulator. We are observing "VFIO_MAP_DMA : -28" error. We are using w-bits=48 bits while lunching VM. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1851664/+subscriptions
Re: [PATCH 2/5] ipmi: Add support to customize OEM functions
On 07/11/2019 18:25, Corey Minyard wrote: > On Thu, Nov 07, 2019 at 06:14:58PM +0100, Cédric Le Goater wrote: > What's the plan for merging this, once it's ready? Is there an IPMI > tree for it to be staged in? If not I could take it through the ppc > tree, but I'd need some Acked-bys in that case. I have an IPMI tree for this. I was assuming it was going in to the PPC tree, but it's not big deal. >>> >>> I'd be more comfortable if the generic ipmi changes went through the >>> ipmi tree. >> >> Here is the patch : >> >> http://patchwork.ozlabs.org/patch/1185187/ > > Ok, I have this in my tree. > > I assume there is nothing like the linux-next tree for qemu, right? no. These IPMI OEM commands are handled by the OPAL firmware only. C.
[Bug 1810105] Re: Hint showing volume never disappears, blocking buttons to minimize, maximize and close
I finally found a real computer on which this bug is present. The computer in question is a netbook manufactured by Positivo, the Mobo 5900, which is common on schools. It has a touchscreen with a pen which is compatible with evdev but that doesn't work with libinput. Its touchscreen is: Bus 004 Device 006: ID 22b9:0005 eTurboTouch Technology, Inc. The attached image not only shows the hint from the pulseaudio-plugin, but another one caused by pavucontrol. As a virtual keyboard is normally used because the netbook can be used as a tablet, the hints can make some keys impossible to use. ** Attachment added: "Captura de tela_2019-11-07_14-12-19.png" https://bugs.launchpad.net/ubuntu/+source/xfce4-pulseaudio-plugin/+bug/1810105/+attachment/5303552/+files/Captura%20de%20tela_2019-11-07_14-12-19.png -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1810105 Title: Hint showing volume never disappears, blocking buttons to minimize, maximize and close Status in QEMU: New Status in xfce4-pulseaudio-plugin package in Ubuntu: New Bug description: When hovering the mouse over the volume indicator or changing its volume using the mouse wheel it shows the current volume set as a hint. For example: Volume 100% The problem is that the hint never disappears, not even clicking on it. On some occasions the hint can cover the minimize, maximize and close buttons, causing significant problems on using the desktop environment, as these three buttons won't be usable anymore with the hint over it. Where the hint appears it's no longer possible to interact with the screen. ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: xfce4-pulseaudio-plugin 0.4.1-0ubuntu1 ProcVersionSignature: Ubuntu 4.15.0-43.46-generic 4.15.18 Uname: Linux 4.15.0-43-generic x86_64 ApportVersion: 2.20.9-0ubuntu7.5 Architecture: amd64 CurrentDesktop: XFCE Date: Sun Dec 30 17:09:23 2018 InstallationDate: Installed on 2018-12-30 (0 days ago) InstallationMedia: Xubuntu 18.04.2 LTS "Bionic Beaver" - Beta amd64 (20181230) SourcePackage: xfce4-pulseaudio-plugin UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1810105/+subscriptions
Re: [PATCH v1 0/6] testing/next (netbsd stuff)
On Thu, 7 Nov 2019 at 17:54, Kamil Rytarowski wrote: > > On 07.11.2019 18:46, Peter Maydell wrote: > > On Mon, 4 Nov 2019 at 17:39, Alex Bennée wrote: > >> > >> Hi, > >> > >> As we approach hard-freeze I'm trying to temper what comes in through > >> the testing/next tree. However it would be nice to get the NetBSD upto > >> speed with the other NetBSDs. Although the serial install is working > >> well for me this has had a rocky road so if others could also give it > >> a good testing that would be great. I've also disabled one of the > >> regular failing tests for non-Linux targets. There are other tests > >> that still fail however including the tests/test-aio-multithread which > >> asserts in the async utils around about 20% of the time: > >> > >> assertion "QSLIST_EMPTY(>scheduled_coroutines)" failed: file > >> "/home/qemu/qemu-test.nS1czd/src/util/async.c", line 279, function > >> "aio_ctx_finalize" > > > > This is unrelated to your NetBSD update in this series -- it's > > one of the persistent intermittents I see on the BSDs: > > https://lore.kernel.org/qemu-devel/20190916153312.GD25552@stefanha-x1.localdomain/t/ > > > > (though the failure rate I see is I think <20%, but I haven't > > really carefully measured it.) > Does this patch rely on AIO API in the kernel? If so than this is > unreliable as of today on NetBSD. We plan to fix it, but there is no > expected time of accomplishment. No, we use our own AIO implementation which puts fds into non-blocking mode and uses a thread which polls them to identify when they're ready to actually perform IO (plus a lot of coroutine magic). thanks -- PMM
Re: Looking for issues/features for my first contribution
Thank you Aleksandar ! This is really helpful. Rajath Shashidhara On 07-11-2019 07:33, Aleksandar Markovic wrote: On Thu, Nov 7, 2019 at 11:37 AM Aleksandar Markovic wrote: On Thursday, November 7, 2019, Rajath Shashidhara wrote: Hi all, I am a Computer Science graduate student at The University of Texas at Austin (UT, Austin). I am looking forward to contributing to qemu ! This semester, I am taking a class in Virtualization (https://github.com/vijay03/cs378-f19) and contributing to a virtualization related open-source project is a significant part of the course. I would be interested in contributing a patchset to qemu - possibly a self-contained feature or a reasonably complex bug fix that can be completed in under a month's time. I did look at both the bugtracker and the QEMU Google Summer of Code 2019 page [https://wiki.qemu.org/Google_Summer_of_Code_2019] for ideas. However, I would be interested in hearing from the community and I would be delighted if somebody can be suggest a suitable project ! Hello, Rajath! Thank you for expressing interest in QEMU open source project. There is certainly a place for you and your contributions in QEMU, and you are very welcomed! It looks to me the following project would fit your description: 'Implement emulation of DS3231 real time clock in QEMU' Datasheet: https://datasheets.maximintegrated.com/en/ds/DS3231.pdf The steps needed to complete it (in my opinion): - collect datasheets of as many as possible RTC chips already emulated in QEMU (there are around of dozen of them, see folder hw/rtc) I did a quick Google search on datasheets of existing RTC implemtations, and the result is: DS1338: https://datasheets.maximintegrated.com/en/ds/DS1338-DS1338Z.pdf M41T80: https://www.st.com/resource/en/datasheet/m41t80.pdf M48T59: http://www.elektronikjk.pl/elementy_czynne/IC/M48T59V.pdf MC146818: https://www.nxp.com/docs/en/data-sheet/MC146818.pdf PL031: http://infocenter.arm.com/help/topic/com.arm.doc.ddi0224c/real_time_clock_pl031_r1p3_technical_reference_manual_DDI0224C.pdf TWL92230: https://datasheet.octopart.com/TWL92230C-Texas-Instruments-datasheet-150321.pdf Zynq RTC: https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ultrascale-trm.pdf (chapter 7) Aleksandar - do a comparative analysis of selected RTC implementations in QEMU - get to know general QEMU device model - design and implement DS3231 emulation I can give you (unfortunately constrained by tight time limits) some help and guidance. But there are other people in community too (more knowledgable in the area than me). I salute your initiative! Yours, Aleksandar I am an advanced C programmer with both professional and academic background in systems design & implementation - especially OS & Networks. Given my background, I feel fairly confident that I can pickup the QEMU codebase quickly. Eagerly looking forward to hearing from the community ! Thanks, Rajath Shashidhara
Re: [Qemu-devel] Exposing feature deprecation to machine clients
Hi Markus,
On 8/15/19 7:40 PM, John Snow wrote:
On 8/15/19 10:16 AM, Markus Armbruster wrote:
John Snow writes:
[...]
I asked Markus this not too long ago; do we want to amend the QAPI
schema specification to allow commands to return with "Warning" strings,
or "Deprecated" stings to allow in-band deprecation notices for cases
like these?
example:
{ "return": {},
"deprecated": True,
"warning": "Omitting filter-node-name parameter is deprecated, it will
be required in the future"
}
There's no "error" key, so this should be recognized as success by
compatible clients, but they'll definitely see the extra information.
This is a compatible evolution of the QMP protocol.
Part of my motivation is to facilitate a more aggressive deprecation of
legacy features by ensuring that we are able to rigorously notify users
through any means that they need to adjust their scripts.
Yes, we should help libvirt etc. with detecting use of deprecated
features. We discussed this at the KVM Forum 2018 BoF on deprecating
stuff. Minutes:
Message-ID: <87mur0ls8o@dusky.pond.sub.org>
https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg05828.html
Last item is relevant here.
Adding deprecation information to QMP's success response belongs to "We
can also pass the buck to the next layer up", next to "emit a QMP
event".
Let's compare the two, i.e. "deprecation info in success response"
vs. "deprecation event".
1. Possible triggers
Anything we put in the success response should only ever apply to the
(successful) command. So this one's limited to QMP commands.
A QMP event is not limited to QMP commands. For instance, it could be
emitted for deprecated CLI features (long after the fact, in addition to
human-readable warnings on stderr), or when we detect use of a
deprecated feature only after we sent the success response, say in a
job. Neither use case is particularly convincing. Reporting use of
deprecated CLI in QMP feels like a work-around for the CLI's
machine-unfriendliness. Job-like commands should really check their
arguments upfront.
2. Connection to trigger
Connecting responses to commands is the QMP protocol's responsibility.
Transmitting deprecation information in the response trivially ties it
to the offending command.
The QMP protocol doesn't tie events to anything. Thus, a deprecation
event needs an event-specific tie to its trigger.
The obvious way to tie it to a command mirrors how the QMP protocol ties
responses to commands: by command ID. The event either has to be sent
just to the offending monitor (currently, all events are broadcast to
all monitors), or include a suitable monitor ID.
For non-command triggers, we'd have to invent some other tie.
3. Interface complexity
Tying the event to some arbitrary trigger adds complexity.
Do we need non-command triggers, and badly enough to justify the
additional complexity?
4. Implementation complexity
Emitting an event could be as simple as
qapi_event_send_deprecated(qmp_command_id(),
"Omitting 'filter-node-name'");
where qmp_command_id() returns the ID of the currently executing
command. Making qmp_command_id() work is up to the QMP core. Simple
enough as long as each QMP command runs to completion before its monitor
starts the next one.
The event is "fire and forget". There is no warning object propagated
up the call chain into the QMP core like errors objects are.
"Fire and forget" is ideal for letting arbitrary code decide "this is
deprecated".
Note the QAPI schema remains untouched.
Unlike an event, which can be emitted anywhere, the success response
gets built in the QMP core. To have the core add deprecation info to
it, we need to get the info to the core.
If deprecation info originates in command code, like errors do, we need
to propagate it up the call chain into the QMP core like errors.
Propagating errors is painful. It has caused massive churn all over the
place.
I don't think we can hitch deprecation info to the existing error
propagation, since we need to take the success path back to the QMP
core, not an error path.
Propagating a second object for warnings... thanks, but no thanks.
Probably the best argument against it. Fire-and-forget avoids the
problem. Events might work just fine, but the "tie" bit seems like a yak
in need of a shave.
The QMP core could provide a function for recording deprecation info for
the currently executing QMP command. This is how we used to record
errors in QMP commands, until Anthony rammed through what we have now.
The commit messages (e.g. d5ec4f27c38) provide no justification. I
dimly recall adamant (oral?) claims that recording errors in the Monitor
object cannot work for us.
I smell a swamp.
Can we avoid plumbing deprecation info from command code to QMP core?
Only if the QMP core itself can recognize deprecated interfaces. I
believe it can for the cases we can expose in introspecion. Let me
explain.
Re: Looking for issues/features for my first contribution
Hi, Thanks Stefan ! I spoke to Dinah and this issue is still up for grabs. I would be working on both SeaBIOS MMConfig task and the DS3231 RTC emulation feature. Thanks for your help! Regards, Rajath Shashidhara On 07-11-2019 07:54, Stefan Hajnoczi wrote: On Wed, Nov 06, 2019 at 05:50:44PM -0600, Rajath Shashidhara wrote: Hi all, I am a Computer Science graduate student at The University of Texas at Austin (UT, Austin). I am looking forward to contributing to qemu ! This semester, I am taking a class in Virtualization (https://github.com/vijay03/cs378-f19) and contributing to a virtualization related open-source project is a significant part of the course. I would be interested in contributing a patchset to qemu - possibly a self-contained feature or a reasonably complex bug fix that can be completed in under a month's time. I did look at both the bugtracker and the QEMU Google Summer of Code 2019 page [https://wiki.qemu.org/Google_Summer_of_Code_2019] for ideas. However, I would be interested in hearing from the community and I would be delighted if somebody can be suggest a suitable project ! I am an advanced C programmer with both professional and academic background in systems design & implementation - especially OS & Networks. Given my background, I feel fairly confident that I can pickup the QEMU codebase quickly. Please check with Dinah Baum whether the SeaBIOS MMConfig task is already taken, maybe you'd like to work on it if the task is still available: https://lore.kernel.org/qemu-devel/20191105163952.GI166646@stefanha-x1.localdomain/ Stefan
Re: [PATCH v7 4/8] Acceptance tests: use relative location for tests
On 11/4/19 1:13 PM, Cleber Rosa wrote: An Avocado Test ID[1] is composed by a number of components, but it starts with the Test Name, usually a file system location that was given to the loader. Because the source directory is being given as a prefix to the "tests/acceptance" directory containing the acceptance tests, the test names will needlessly include the directory the user is using to host the QEMU sources (and/or build tree). Let's remove the source dir (or a build dir) from the path given to the test loader. This should give more constant names, and when using result servers and databases, it should give the same test names across executions from different people or from different directories. Completely agree. [1] - https://avocado-framework.readthedocs.io/en/69.0/ReferenceGuide.html#test-id Signed-off-by: Cleber Rosa --- tests/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Wainer dos Santos Moschetta - Wainer diff --git a/tests/Makefile.include b/tests/Makefile.include index 56f73b46e2..65e85f5275 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -1180,7 +1180,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ --filter-by-tags-include-empty --filter-by-tags-include-empty-key \ $(AVOCADO_TAGS) \ ---failfast=on $(SRC_PATH)/tests/acceptance, \ +--failfast=on tests/acceptance, \ "AVOCADO", "tests/acceptance") # Consolidated targets
Re: QEMU HTML documentation now on qemu.org
On Thu, Nov 07, 2019 at 04:44:34PM +0100, Stefan Hajnoczi wrote: > On Thu, Nov 7, 2019 at 11:07 AM Daniel P. Berrangé > wrote: > > > > On Wed, Nov 06, 2019 at 05:19:28PM +0100, Stefan Hajnoczi wrote: > > > Hi, > > > You can now access the latest QEMU HTML documentation built from > > > qemu.git/master nightly at: > > > > > > https://wiki.qemu.org/docs/qemu-doc.html > > > https://wiki.qemu.org/docs/qemu-qmp-ref.html > > > https://wiki.qemu.org/docs/qemu-ga-ref.html > > > ...as well as interop/ and specs/ > > > > > > Feel free to link to the documentation from the QEMU website and/or > > > wiki! > > > > What's the reason for putting on wiki.qemu.org URL ? It feels like > > having it under www.qemu.org would be a more natural home, especially > > if we can then make it pick up the jekyll theme around the pages. > > > > Ideally we should publish the docs under versioned URL when we > > make a release. eg /docs/latest/ for current GIT master > > which I presume the above is tracking, and then a /docs/$VERSION/... > > for each major release we cut. > > > > That way users can get an accurate view of features in the QEMU > > they are actually using. > > Versioned release docs should be generated during the release process. > I have CCed Mike Roth. That way the docs are available as soon as the > release drops. This container image only runs once a day and would > leave a window when users cannot access the docs yet. > > Moving from wiki.qemu.org should be possible. How does the jekyll > theme you mentioned work? IIUC, when there's a push to the qemu-web.git repo, some git hook (?) runs on the server which invokes jekyll to build the content, and then publish it to the webroot. To integrate these docs into that we need something along the lines of: 1. Generate the HTML files as you do now 2. Copy them into the qemu-web.git in a /docs/ subdir 3. Prepend a magic header to make jeykll process the file --- permalink: /docs/qemu-doc --- 4. Trigger the jekyll builder to refresh the generated docs 5. Publish the docs to the webroot You can see what I did here as an example where I simply committed the generated docs to qemu-web.git: https://www.mail-archive.com/qemu-devel@nongnu.org/msg578110.html If we're not storing the generated docs in git, then when pushing to qemu-web.git we need to ensure we preserve the extra /docs dir content in some manner. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PATCH 2/4] Add use of RCU for qemu_logfile.
Robert Foley writes:
> This now allows changing the logfile while logging is active,
> and also solves the issue of a seg fault while changing the logfile.
>
> Any read access to the qemu_logfile handle will use
> the rcu_read_lock()/unlock() around the use of the handle.
> To fetch the handle we will use atomic_rcu_read().
> We also in many cases do a check for validity of the
> logfile handle before using it to deal with the case where the
> file is closed and set to NULL.
>
> The cases where we write to the qemu_logfile will use atomic_rcu_set().
> Writers will also use call_rcu() with a newly added qemu_logfile_free
> function for freeing/closing when readers have finished.
>
> Signed-off-by: Robert Foley
> ---
> include/qemu/log.h | 47
> util/log.c | 78 ++
> include/exec/log.h | 33 +---
> tcg/tcg.c | 12 +--
> 4 files changed, 138 insertions(+), 32 deletions(-)
>
> diff --git a/include/qemu/log.h b/include/qemu/log.h
> index a91105b2ad..975de18e23 100644
> --- a/include/qemu/log.h
> +++ b/include/qemu/log.h
> @@ -3,9 +3,17 @@
>
> /* A small part of this API is split into its own header */
> #include "qemu/log-for-trace.h"
> +#include "qemu/rcu.h"
> +
> +struct QemuLogFile {
> +struct rcu_head rcu;
> +FILE *fd;
> +};
> +typedef struct QemuLogFile QemuLogFile;
If you are declaring a typedef then do it in one:
typedef struct QemuLogFile { ...
We only really use the second form for opaque structs where the handle
is passed around but the contents hidden from the rest of QEMU.
>
> /* Private global variable, don't use */
> -extern FILE *qemu_logfile;
> +extern QemuLogFile *qemu_logfile;
> +
Do we need multiple
>
> /*
> * The new API:
> @@ -25,7 +33,17 @@ static inline bool qemu_log_enabled(void)
> */
> static inline bool qemu_log_separate(void)
> {
> -return qemu_logfile != NULL && qemu_logfile != stderr;
> +QemuLogFile *logfile;
> +
> +if (qemu_log_enabled()) {
> +rcu_read_lock();
> +logfile = atomic_rcu_read(_logfile);
> +if (logfile && logfile->fd != stderr) {
> +return true;
> +}
> +rcu_read_unlock();
> +}
> +return false;
This is unbalanced as you'll have incremented the reader count. Also
qemu_log_enabled() is also synonymous with logfile existing so you could
fold that into:
bool res = false;
rcu_read_lock();
*logfile = atomic_rcu_read(_logfile);
if (logfile && logfile->fd != stderr) {
res = true;
}
rcu_read_unlock();
return res;
There is technically a race there as the answer may become invalid after
qemu_log_separate() returns. However given the users I don't see what
could fail afterwards.
> }
>
> #define CPU_LOG_TB_OUT_ASM (1 << 0)
> @@ -55,12 +73,23 @@ static inline bool qemu_log_separate(void)
>
> static inline void qemu_log_lock(void)
> {
> -qemu_flockfile(qemu_logfile);
> +QemuLogFile *logfile;
> +rcu_read_lock();
> +logfile = atomic_rcu_read(_logfile);
> +if (logfile) {
> +qemu_flockfile(logfile->fd);
> +}
> +rcu_read_unlock();
> }
static inline FILE *fd qemu_log_lock(void)
{
FILE *fd;
rcu_read_lock();
fd = atomic_rcu_read(_logfile);
if (fd) {
qemu_flockfile(fd);
return fd;
} else {
rcu_read_unlock();
return NULL;
}
}
static inline qemu_log_unlock(FILE *fd)
{
if (fd) {
qemu_funlockfile(fd);
rcu_read_unlock();
}
}
While the rcu_read_lock() is in progress then we won't ever finish with
the fd - which we don't want to do until the file locking is finished.
>
> /* Logging functions: */
> @@ -70,9 +99,14 @@ static inline void qemu_log_unlock(void)
> static inline void GCC_FMT_ATTR(1, 0)
> qemu_log_vprintf(const char *fmt, va_list va)
> {
> -if (qemu_logfile) {
> -vfprintf(qemu_logfile, fmt, va);
> +QemuLogFile *logfile;
> +
> +rcu_read_lock();
> +logfile = atomic_rcu_read(_logfile);
> +if (logfile) {
> +vfprintf(logfile->fd, fmt, va);
> }
> +rcu_read_unlock();
> }
>
> /* log only if a bit is set on the current loglevel mask:
> @@ -129,5 +163,6 @@ void qemu_print_log_usage(FILE *f);
> void qemu_log_flush(void);
> /* Close the log file */
> void qemu_log_close(void);
> +void qemu_logfile_free(QemuLogFile *logfile);
>
> #endif
> diff --git a/util/log.c b/util/log.c
> index dff2f98c8c..d409532b8f 100644
> --- a/util/log.c
> +++ b/util/log.c
> @@ -29,7 +29,7 @@
> static char *logfilename;
> static bool qemu_logfile_initialized;
> static QemuMutex qemu_logfile_mutex;
> -FILE *qemu_logfile;
> +QemuLogFile *qemu_logfile;
> int qemu_loglevel;
> static int log_append = 0;
> static GArray *debug_regions;
> @@ -38,10 +38,14 @@ static GArray *debug_regions;
> int qemu_log(const char *fmt, ...)
> {
> int ret = 0;
> -if (qemu_logfile) {
> +QemuLogFile *logfile;
> +
> +
[PATCH v3 10/22] iotests: Inject space into -ocompat=0.10 in 051
It did not matter before, but now that _make_test_img understands -o, we should use it properly here. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/051 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 index 53bcdbc911..9cd1d60d45 100755 --- a/tests/qemu-iotests/051 +++ b/tests/qemu-iotests/051 @@ -157,7 +157,7 @@ echo echo === With version 2 images enabling lazy refcounts must fail === echo -_make_test_img -ocompat=0.10 $size +_make_test_img -o compat=0.10 $size run_qemu -drive file="$TEST_IMG",format=qcow2,lazy-refcounts=on run_qemu -drive file="$TEST_IMG",format=qcow2,lazy-refcounts=off -- 2.23.0
[PATCH v3 16/22] iotests: Avoid cp/mv of test images
This will not work with external data files, so try to get tests working
without it as far as possible.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/063 | 12
tests/qemu-iotests/063.out | 3 ++-
tests/qemu-iotests/085 | 9 +++--
tests/qemu-iotests/085.out | 8
4 files changed, 13 insertions(+), 19 deletions(-)
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index eef2b8a534..c750b3806e 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -51,15 +51,13 @@ _unsupported_imgopts "subformat=monolithicFlat" \
_make_test_img 4M
echo "== Testing conversion with -n fails with no target file =="
-# check .orig file does not exist
-rm -f "$TEST_IMG.orig"
if $QEMU_IMG convert -f $IMGFMT -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG.orig"
>/dev/null 2>&1; then
exit 1
fi
echo "== Testing conversion with -n succeeds with a target file =="
-rm -f "$TEST_IMG.orig"
-cp "$TEST_IMG" "$TEST_IMG.orig"
+_rm_test_img "$TEST_IMG.orig"
+TEST_IMG="$TEST_IMG.orig" _make_test_img 4M
if ! $QEMU_IMG convert -f $IMGFMT -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG.orig" ;
then
exit 1
fi
@@ -85,10 +83,8 @@ fi
_check_test_img
echo "== Testing conversion to a smaller file fails =="
-rm -f "$TEST_IMG.orig"
-mv "$TEST_IMG" "$TEST_IMG.orig"
-_make_test_img 2M
-if $QEMU_IMG convert -f $IMGFMT -O $IMGFMT -n "$TEST_IMG.orig" "$TEST_IMG"
>/dev/null 2>&1; then
+TEST_IMG="$TEST_IMG.target" _make_test_img 2M
+if $QEMU_IMG convert -f $IMGFMT -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG.target"
>/dev/null 2>&1; then
exit 1
fi
diff --git a/tests/qemu-iotests/063.out b/tests/qemu-iotests/063.out
index 7b691b2c9e..890b719bf0 100644
--- a/tests/qemu-iotests/063.out
+++ b/tests/qemu-iotests/063.out
@@ -2,11 +2,12 @@ QA output created by 063
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304
== Testing conversion with -n fails with no target file ==
== Testing conversion with -n succeeds with a target file ==
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=4194304
== Testing conversion to raw is the same after conversion with -n ==
== Testing conversion back to original format ==
No errors were found on the image.
== Testing conversion to a smaller file fails ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152
+Formatting 'TEST_DIR/t.IMGFMT.target', fmt=IMGFMT size=2097152
== Regression testing for copy offloading bug ==
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
Formatting 'TEST_DIR/t.IMGFMT.target', fmt=IMGFMT size=1048576
diff --git a/tests/qemu-iotests/085 b/tests/qemu-iotests/085
index bbea1252d2..46981dbb64 100755
--- a/tests/qemu-iotests/085
+++ b/tests/qemu-iotests/085
@@ -105,8 +105,7 @@ add_snapshot_image()
{
base_image="${TEST_DIR}/$((${1}-1))-${snapshot_virt0}"
snapshot_file="${TEST_DIR}/${1}-${snapshot_virt0}"
-_make_test_img -u -b "${base_image}" "$size"
-mv "${TEST_IMG}" "${snapshot_file}"
+TEST_IMG=$snapshot_file _make_test_img -u -b "${base_image}" "$size"
do_blockdev_add "$1" "'backing': null, " "${snapshot_file}"
}
@@ -122,10 +121,8 @@ blockdev_snapshot()
size=128M
-_make_test_img $size
-mv "${TEST_IMG}" "${TEST_IMG}.1"
-_make_test_img $size
-mv "${TEST_IMG}" "${TEST_IMG}.2"
+TEST_IMG="$TEST_IMG.1" _make_test_img $size
+TEST_IMG="$TEST_IMG.2" _make_test_img $size
echo
echo === Running QEMU ===
diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out
index 2a5f256cd3..313198f182 100644
--- a/tests/qemu-iotests/085.out
+++ b/tests/qemu-iotests/085.out
@@ -1,6 +1,6 @@
QA output created by 085
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT.1', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728
=== Running QEMU ===
@@ -55,10 +55,10 @@ Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2
size=134217728 backing_fil
=== Create a couple of snapshots using blockdev-snapshot ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
backing_file=TEST_DIR/10-snapshot-v0.IMGFMT
+Formatting 'TEST_DIR/11-snapshot-v0.IMGFMT', fmt=IMGFMT size=134217728
backing_file=TEST_DIR/10-snapshot-v0.IMGFMT
{"return": {}}
{"return": {}}
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
backing_file=TEST_DIR/11-snapshot-v0.IMGFMT
+Formatting 'TEST_DIR/12-snapshot-v0.IMGFMT', fmt=IMGFMT size=134217728
backing_file=TEST_DIR/11-snapshot-v0.IMGFMT
{"return": {}}
{"return": {}}
--
2.23.0
[PATCH v3 13/22] iotests: Drop IMGOPTS use in 267
Overwriting IMGOPTS means ignoring all user-supplied options, which is
not what we want. Replace the current IMGOPTS use by a new BACKING_FILE
variable.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/267 | 12
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/tests/qemu-iotests/267 b/tests/qemu-iotests/267
index 170e173c0a..f54262a4ad 100755
--- a/tests/qemu-iotests/267
+++ b/tests/qemu-iotests/267
@@ -68,7 +68,11 @@ size=128M
run_test()
{
-_make_test_img $size
+if [ -n "$BACKING_FILE" ]; then
+_make_test_img -b "$BACKING_FILE" $size
+else
+_make_test_img $size
+fi
printf "savevm snap0\ninfo snapshots\nloadvm snap0\n" | run_qemu "$@" |
_filter_date
}
@@ -119,12 +123,12 @@ echo
TEST_IMG="$TEST_IMG.base" _make_test_img $size
-IMGOPTS="backing_file=$TEST_IMG.base" \
+BACKING_FILE="$TEST_IMG.base" \
run_test -blockdev
driver=file,filename="$TEST_IMG.base",node-name=backing-file \
-blockdev driver=file,filename="$TEST_IMG",node-name=file \
-blockdev driver=$IMGFMT,file=file,backing=backing-file,node-name=fmt
-IMGOPTS="backing_file=$TEST_IMG.base" \
+BACKING_FILE="$TEST_IMG.base" \
run_test -blockdev
driver=file,filename="$TEST_IMG.base",node-name=backing-file \
-blockdev driver=$IMGFMT,file=backing-file,node-name=backing-fmt \
-blockdev driver=file,filename="$TEST_IMG",node-name=file \
@@ -141,7 +145,7 @@ echo
echo "=== -blockdev with NBD server on the backing file ==="
echo
-IMGOPTS="backing_file=$TEST_IMG.base" _make_test_img $size
+_make_test_img -b "$TEST_IMG.base" $size
cat <
[PATCH v3 19/22] iotests: Make 137 work with data_file
When using an external data file, there are no refcounts for data
clusters. We thus have to adjust the corruption test in this patch to
not be based around a data cluster allocation, but the L2 table
allocation (L2 tables are still refcounted with external data files).
Furthermore, we should not print qcow2.py's list of incompatible
features because it differs depending on whether there is an external
data file or not.
With those two changes, the test will work both with and without
external data files (once that options works with the iotests at all).
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/137 | 15 +++
tests/qemu-iotests/137.out | 6 ++
2 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137
index 6cf2997577..7ae86892f7 100755
--- a/tests/qemu-iotests/137
+++ b/tests/qemu-iotests/137
@@ -138,14 +138,21 @@ $QEMU_IO \
"$TEST_IMG" 2>&1 | _filter_qemu_io
# The dirty bit must not be set
-$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+# (Filter the external data file bit)
+if $PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features \
+| grep -q '\<0\>'
+then
+echo 'ERROR: Dirty bit set'
+else
+echo 'OK: Dirty bit not set'
+fi
# Similarly we can test whether corruption detection has been enabled:
-# Create L1/L2, overwrite first entry in refcount block, allocate something.
+# Create L1, overwrite refcounts, force allocation of L2 by writing
+# data.
# Disabling the checks should fail, so the corruption must be detected.
_make_test_img 64M
-$QEMU_IO -c "write 0 64k" "$TEST_IMG" | _filter_qemu_io
-poke_file "$TEST_IMG" "$((0x2))" "\x00\x00"
+poke_file "$TEST_IMG" "$((0x2))" "\x00\x00\x00\x00\x00\x00\x00\x00"
$QEMU_IO \
-c "reopen -o overlap-check=none,lazy-refcounts=42" \
-c "write 64k 64k" \
diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out
index bd4523a853..86377c80cd 100644
--- a/tests/qemu-iotests/137.out
+++ b/tests/qemu-iotests/137.out
@@ -36,11 +36,9 @@ qemu-io: Unsupported value 'blubb' for qcow2 option
'overlap-check'. Allowed are
wrote 512/512 bytes at offset 0
512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
./common.rc: Killed ( VALGRIND_QEMU="${VALGRIND_QEMU_IO}"
_qemu_proc_exec "${VALGRIND_LOGFILE}" "$QEMU_IO_PROG" $QEMU_IO_ARGS "$@" )
-incompatible_features []
+OK: Dirty bit not set
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-wrote 65536/65536 bytes at offset 0
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
qemu-io: Parameter 'lazy-refcounts' expects 'on' or 'off'
-qcow2: Marking image as corrupt: Preventing invalid write on metadata
(overlaps with qcow2_header); further corruption events will be suppressed
+qcow2: Marking image as corrupt: Preventing invalid allocation of L2 table at
offset 0; further corruption events will be suppressed
write failed: Input/output error
*** done
--
2.23.0
Re: [PATCH v8 0/3] RTC support for QEMU RISC-V virt machine
On Wed, 06 Nov 2019 03:56:29 PST (-0800), Anup Patel wrote: This series adds RTC device to QEMU RISC-V virt machine. We have selected Goldfish RTC device model for this. It's a pretty simple synthetic device with few MMIO registers and no dependency external clock. The driver for Goldfish RTC is already available in Linux so we just need to enable it in Kconfig for RISCV and also update Linux defconfigs. We have tested this series with Linux-5.4-rc4 plus defconfig changes available in 'goldfish_rtc_v2' branch of: https://github.com/avpatel/linux.git Changes since v7: - Fix broken "stdout-path" in "/chosen" DT node of virt machine Changes since v6: - Rebased on latest QEMU master - Addressed all nit comments from Philippe Mathieu-Daude Changes since v5: - Rebased on latest QEMU master - Added maintainer entry for Goldfish RTC Changes since v4: - Fixed typo in trace event usage - Moved goldfish_rtc.h to correct location Changes since v3: - Address all nit comments from Alistair Changes since v2: - Rebased on RTC code refactoring Changes since v1: - Implemented VMState save/restore callbacks Anup Patel (3): hw: rtc: Add Goldfish RTC device riscv: virt: Use Goldfish RTC device MAINTAINERS: Add maintainer entry for Goldfish RTC MAINTAINERS | 8 + hw/riscv/Kconfig | 1 + hw/riscv/virt.c | 16 ++ hw/rtc/Kconfig| 3 + hw/rtc/Makefile.objs | 1 + hw/rtc/goldfish_rtc.c | 285 ++ hw/rtc/trace-events | 4 + include/hw/riscv/virt.h | 2 + include/hw/rtc/goldfish_rtc.h | 46 ++ 9 files changed, 366 insertions(+) create mode 100644 hw/rtc/goldfish_rtc.c create mode 100644 include/hw/rtc/goldfish_rtc.h Thanks. I've updated the patches on my queue, LMK if there are any more changes!
[PATCH v3 14/22] iotests: Avoid qemu-img create
Use _make_test_img whenever possible. This way, we will not ignore
user-specified image options.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/094 | 2 +-
tests/qemu-iotests/111 | 3 +--
tests/qemu-iotests/123 | 2 +-
tests/qemu-iotests/153 | 2 +-
tests/qemu-iotests/200 | 4 ++--
5 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/tests/qemu-iotests/094 b/tests/qemu-iotests/094
index 9343e09492..d645952d54 100755
--- a/tests/qemu-iotests/094
+++ b/tests/qemu-iotests/094
@@ -45,7 +45,7 @@ _supported_proto nbd
_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
_make_test_img 64M
-$QEMU_IMG create -f $IMGFMT "$TEST_DIR/source.$IMGFMT" 64M | _filter_img_create
+TEST_IMG_FILE="$TEST_DIR/source.$IMGFMT" IMGPROTO=file _make_test_img 64M
_launch_qemu -drive if=none,id=src,file="$TEST_DIR/source.$IMGFMT",format=raw \
-nodefaults
diff --git a/tests/qemu-iotests/111 b/tests/qemu-iotests/111
index 490a5bbcb5..3b43d1bd83 100755
--- a/tests/qemu-iotests/111
+++ b/tests/qemu-iotests/111
@@ -41,8 +41,7 @@ _supported_fmt qed qcow qcow2 vmdk
_supported_proto file
_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
-$QEMU_IMG create -f $IMGFMT -b "$TEST_IMG.inexistent" "$TEST_IMG" 2>&1 \
-| _filter_testdir | _filter_imgfmt
+_make_test_img -b "$TEST_IMG.inexistent"
# success, all done
echo '*** done'
diff --git a/tests/qemu-iotests/123 b/tests/qemu-iotests/123
index d33950eb54..74d40d0478 100755
--- a/tests/qemu-iotests/123
+++ b/tests/qemu-iotests/123
@@ -44,7 +44,7 @@ _supported_os Linux
SRC_IMG="$TEST_DIR/source.$IMGFMT"
_make_test_img 1M
-$QEMU_IMG create -f $IMGFMT "$SRC_IMG" 1M | _filter_img_create
+TEST_IMG_FILE=$SRC_IMG IMGPROTO=file _make_test_img 1M
$QEMU_IO -c 'write -P 42 0 1M' "$SRC_IMG" | _filter_qemu_io
diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153
index c969a1a16f..e59090259c 100755
--- a/tests/qemu-iotests/153
+++ b/tests/qemu-iotests/153
@@ -98,7 +98,7 @@ for opts1 in "" "read-only=on" "read-only=on,force-share=on";
do
echo
echo "== Creating test image =="
-$QEMU_IMG create -f $IMGFMT "${TEST_IMG}" -b ${TEST_IMG}.base |
_filter_img_create
+_make_test_img -b "${TEST_IMG}.base"
echo
echo "== Launching QEMU, opts: '$opts1' =="
diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200
index 72d431f251..d904885136 100755
--- a/tests/qemu-iotests/200
+++ b/tests/qemu-iotests/200
@@ -46,8 +46,8 @@ _supported_proto file
BACKING_IMG="${TEST_DIR}/backing.img"
TEST_IMG="${TEST_DIR}/test.img"
-${QEMU_IMG} create -f $IMGFMT "${BACKING_IMG}" 512M | _filter_img_create
-${QEMU_IMG} create -f $IMGFMT -F $IMGFMT "${TEST_IMG}" -b "${BACKING_IMG}"
512M | _filter_img_create
+TEST_IMG="$BACKING_IMG" _make_test_img 512M
+_make_test_img -F $IMGFMT -b "$BACKING_IMG" 512M
${QEMU_IO} -c "write -P 0xa5 512 300M" "${BACKING_IMG}" | _filter_qemu_io
--
2.23.0
Re: [PATCH 1/4] Add a mutex to guarantee single writer to qemu_logfile handle.
Robert Foley writes:
> This is being added in preparation for using RCU with the logfile handle.
> Also added qemu_logfile_init() for initializing the logfile mutex.
>
> Signed-off-by: Robert Foley
> ---
> util/log.c | 23 +++
> 1 file changed, 23 insertions(+)
>
> diff --git a/util/log.c b/util/log.c
> index 1ca13059ee..dff2f98c8c 100644
> --- a/util/log.c
> +++ b/util/log.c
> @@ -24,8 +24,11 @@
> #include "qapi/error.h"
> #include "qemu/cutils.h"
> #include "trace/control.h"
> +#include "qemu/thread.h"
>
> static char *logfilename;
> +static bool qemu_logfile_initialized;
> +static QemuMutex qemu_logfile_mutex;
> FILE *qemu_logfile;
> int qemu_loglevel;
> static int log_append = 0;
> @@ -49,6 +52,14 @@ int qemu_log(const char *fmt, ...)
> return ret;
> }
>
> +static void qemu_logfile_init(void)
> +{
> +if (!qemu_logfile_initialized) {
> +qemu_mutex_init(_logfile_mutex);
> +qemu_logfile_initialized = true;
> +}
> +}
> +
> static bool log_uses_own_buffers;
>
> /* enable or disable low levels log */
> @@ -58,6 +69,12 @@ void qemu_set_log(int log_flags)
> #ifdef CONFIG_TRACE_LOG
> qemu_loglevel |= LOG_TRACE;
> #endif
> +
> +/* Is there a better place to call this to init the logfile subsystem? */
> +if (!qemu_logfile_initialized) {
> +qemu_logfile_init();
> +}
It wouldn't be the worst thing in the world to expose:
qemu_logfile_init()
and make vl.c and main.c call it before the setup. Then you can drop the
flag or even just g_assert(qemu_log_mutex_initialised) in qemu_set_log
and qemu_set_logfile.
In fact you could just use:
static void __attribute__((__constructor__)) qemu_logfile_init(void)
and make the compiler do it for you.
> +qemu_mutex_lock(_logfile_mutex);
> if (!qemu_logfile &&
> (is_daemonized() ? logfilename != NULL : qemu_loglevel)) {
> if (logfilename) {
> @@ -93,6 +110,7 @@ void qemu_set_log(int log_flags)
> log_append = 1;
> }
> }
> +qemu_mutex_unlock(_logfile_mutex);
> if (qemu_logfile &&
> (is_daemonized() ? logfilename == NULL : !qemu_loglevel)) {
> qemu_log_close();
> @@ -114,6 +132,11 @@ void qemu_set_log_filename(const char *filename, Error
> **errp)
> char *pidstr;
> g_free(logfilename);
>
> +/* Is there a better place to call this to init the logfile subsystem? */
> +if (!qemu_logfile_initialized) {
> +qemu_logfile_init();
> +}
> +
> pidstr = strstr(filename, "%");
> if (pidstr) {
> /* We only accept one %d, no other format strings */
--
Alex Bennée
Re: [PULL v3 0/3] Trivial branch patches
On Wed, 6 Nov 2019 at 16:26, Laurent Vivier wrote: > > The following changes since commit 36609b4fa36f0ac934874371874416f7533a5408: > > Merge remote-tracking branch > 'remotes/palmer/tags/palmer-for-master-4.2-sf1' into staging (2019-11-02 > 17:59:03 +) > > are available in the Git repository at: > > git://github.com/vivier/qemu.git tags/trivial-branch-pull-request > > for you to fetch changes up to df59feb197cda31a8b807c13bf509259db9e018f: > > global: Squash 'the the' (2019-11-06 17:19:40 +0100) > > > Trivial fixes (20191105-v3) > > v3: remove disas/libvixl/vixl/invalset.h changes > v2: remove patch from Greg that has lines with more than 80 columns > > > > Dr. David Alan Gilbert (1): > global: Squash 'the the' > > Philippe Mathieu-Daudé (2): > hw/misc/grlib_ahb_apb_pnp: Avoid crash when writing to PnP registers > hw/misc/grlib_ahb_apb_pnp: Fix 8-bit accesses > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2 for any user-visible changes. -- PMM
Re: [PATCH v1 0/6] testing/next (netbsd stuff)
On 07.11.2019 18:46, Peter Maydell wrote: > On Mon, 4 Nov 2019 at 17:39, Alex Bennée wrote: >> >> Hi, >> >> As we approach hard-freeze I'm trying to temper what comes in through >> the testing/next tree. However it would be nice to get the NetBSD upto >> speed with the other NetBSDs. Although the serial install is working >> well for me this has had a rocky road so if others could also give it >> a good testing that would be great. I've also disabled one of the >> regular failing tests for non-Linux targets. There are other tests >> that still fail however including the tests/test-aio-multithread which >> asserts in the async utils around about 20% of the time: >> >> assertion "QSLIST_EMPTY(>scheduled_coroutines)" failed: file >> "/home/qemu/qemu-test.nS1czd/src/util/async.c", line 279, function >> "aio_ctx_finalize" > > This is unrelated to your NetBSD update in this series -- it's > one of the persistent intermittents I see on the BSDs: > https://lore.kernel.org/qemu-devel/20190916153312.GD25552@stefanha-x1.localdomain/t/ > > (though the failure rate I see is I think <20%, but I haven't > really carefully measured it.) > > thanks > -- PMM > Does this patch rely on AIO API in the kernel? If so than this is unreliable as of today on NetBSD. We plan to fix it, but there is no expected time of accomplishment. signature.asc Description: OpenPGP digital signature
Re: [PATCH v15 02/12] util/cutils: Add qemu_strtotime_ns()
On Thu, Nov 07, 2019 at 03:45:01PM +0800, Tao Xu wrote: > To convert strings with time suffixes to numbers, support time unit are > "ns" for nanosecond, "us" for microsecond, "ms" for millisecond or "s" > for second. Add test for qemu_strtotime_ns, test the input of basic, > time suffixes, float, invaild, trailing and overflow. > > Signed-off-by: Tao Xu Reviewed-by: Eduardo Habkost -- Eduardo
[Bug 1851664] Re: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine
qemu-system-x86_64 -name guest=fedora24 -machine q35,accel=kvm,kernel-irqchip=split \ -enable-kvm \ -m 4G \ -smp 8,sockets=1,cores=8,threads=1 \ -device intel-iommu,intremap=on,caching-mode=on,aw-bits=48 \ -drive file=,format=raw \ -device ioh3420,id=pcie.1,chassis=1 \ -device virtio-net-pci,bus=pcie.1,disable-legacy=on,disable-modern=off,iommu_platform=on,ats=on,netdev=net0 \ -netdev user,id=net0,hostfwd=tcp::-:22\ -device vfio-pci,host=3f:02.1 \ -device vfio-pci,host=3f:02.2 \ -device vfio-pci,host=3f:02.3 \ -device vfio-pci,host=3f:02.4 \ -device vfio-pci,host=3d:02.4 \ -device vfio-pci,host=3d:02.5 \ -device vfio-pci,host=3d:02.6 \ -nographic -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1851664 Title: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine Status in QEMU: Incomplete Bug description: We are trying to attach 6 VF's to the guest machine on 4.1.1 qemu emulator. We are observing "VFIO_MAP_DMA : -28" error. We are using w-bits=48 bits while lunching VM. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1851664/+subscriptions
Re: [PATCH v3 01/22] iotests: s/qocw2/qcow2/
On 11/7/19 10:36 AM, Max Reitz wrote: Probably due to blind copy-pasting, we have several instances of "qocw2" in our iotests. Fix them. Reported-by: Maxim Levitsky Signed-off-by: Max Reitz --- tests/qemu-iotests/060 | 2 +- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PULL 0/1] Usb 20191107 patches
Hi Peter, On 11/7/19 7:26 PM, Peter Maydell wrote: On Thu, 7 Nov 2019 at 08:58, Gerd Hoffmann wrote: The following changes since commit 412fbef3d076c43e56451bacb28c4544858c66a3: Merge remote-tracking branch 'remotes/philmd-gitlab/tags/fw_cfg-next-pull-request' into staging (2019-11-05 20:17:11 +) are available in the Git repository at: git://git.kraxel.org/qemu tags/usb-20191107-pull-request for you to fetch changes up to 1dfe2b91dcb1633d0ba450a8139d53006e700a9b: usb-host: add option to allow all resets. (2019-11-06 13:26:04 +0100) usb: fix for usb-host Gerd Hoffmann (1): usb-host: add option to allow all resets. hw/usb/host-libusb.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) This didn't quite make rc0 but it'll go in for rc1. Won't this make bisection confusing?
Re: [PATCH] qmp: Reset mon->commands on CHR_EVENT_CLOSED
On Wed, Nov 6, 2019 at 8:08 PM wrote:
>
> Patchew URL:
> https://patchew.org/QEMU/20191106130309.6737-1-jandr...@gmail.com/
>
>
>
> Hi,
>
> This series failed the docker-quick@centos7 build test. Please find the
> testing commands and
> their output below. If you have Docker installed, you can probably reproduce
> it
> locally.
>
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> make docker-image-centos7 V=1 NETWORK=1
> time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
> === TEST SCRIPT END ===
>
> TESTiotest-qcow2: 268
> Failures: 060 071 176 184
> Failed 4 of 108 iotests
> make: *** [check-tests/check-block.sh] Error 1
> Traceback (most recent call last):
> File "./tests/docker/docker.py", line 662, in
> sys.exit(main())
> ---
> raise CalledProcessError(retcode, cmd)
> subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run',
> '--label', 'com.qemu.instance.uuid=cb707bce0c3c456d8ecec70aeb08fddc', '-u',
> '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=',
> '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=',
> '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v',
> '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v',
> '/var/tmp/patchew-tester-tmp-mxl5_jec/src/docker-src.2019-11-06-19.55.47.20736:/var/tmp/qemu:z,ro',
> 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit
> status 2.
> filter=--filter=label=com.qemu.instance.uuid=cb707bce0c3c456d8ecec70aeb08fddc
> make[1]: *** [docker-run] Error 1
> make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-mxl5_jec/src'
> make: *** [docker-run-test-quick@centos7] Error 2
>
> real13m1.810s
> user0m8.371s
>
>
> The full log is available at
> http://patchew.org/logs/20191106130309.6737-1-jandr...@gmail.com/testing.docker-quick@centos7/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-de...@redhat.com
The full logs shows iotest failures:
Failures: 060 071 176 184
Failed 4 of 108 iotests
The failures are the lack of SHUTDOWN events:
-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP},
"event": "SHUTDOWN", "data": {"guest": false, "reason":
"host-qmp-quit"}}
The results are inconsistent though. On my workstation, for one run I
had test 071 fail, but the others pass. On another couple of runs,
they all passed. An a final one, 176 failed.
Looking at 071, they all send a qmp command to exit, but only some of
the tests have an expected shutdown event. Looking more broadly, it
seems like tests expect shutdown events.
I don't know the code flow, but is it possible on shutdown for the
chardev to be marked closed before the QMP event is generated? After
this patch, those would not be sent. If "quit" is expected to always
generate a QMP event, it seems like some ordering needs to be
enforced.
For the tests, the QMP input comes from a shell "here document" ('<<
EOF'), so I suppose stdin could read EOF and mark the chardev closed
before the QMP event is generated. Before this change, QMP events
would still be generated and stdout would still be connected. Indeed,
chardev/char-fd.c:fd_chr_read() closes the chardev on stdio EOF
regardless of stdout state.
Regards,
Jason
Re: [RFC PATCH 01/18] qemu-storage-daemon: Add barebone tool
In addition to Eric's review:
Kevin Wolf writes:
> This adds a new binary qemu-storage-daemon that doesn't yet do more than
> some typical initialisation for tools and parsing the basic command
> options --version, --help and --trace.
>
> Signed-off-by: Kevin Wolf
> ---
> configure | 2 +-
> qemu-storage-daemon.c | 141 ++
> Makefile | 1 +
> 3 files changed, 143 insertions(+), 1 deletion(-)
> create mode 100644 qemu-storage-daemon.c
>
> diff --git a/configure b/configure
> index 08ca4bcb46..bb3d55fb25 100755
> --- a/configure
> +++ b/configure
> @@ -6034,7 +6034,7 @@ tools=""
> if test "$want_tools" = "yes" ; then
>tools="qemu-img\$(EXESUF) qemu-io\$(EXESUF) qemu-edid\$(EXESUF) $tools"
>if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
> -tools="qemu-nbd\$(EXESUF) $tools"
> +tools="qemu-nbd\$(EXESUF) qemu-storage-daemon\$(EXESUF) $tools"
>fi
>if [ "$ivshmem" = "yes" ]; then
> tools="ivshmem-client\$(EXESUF) ivshmem-server\$(EXESUF) $tools"
> diff --git a/qemu-storage-daemon.c b/qemu-storage-daemon.c
> new file mode 100644
> index 00..a251dc255c
> --- /dev/null
> +++ b/qemu-storage-daemon.c
> @@ -0,0 +1,141 @@
> +/*
> + * QEMU storage daemon
> + *
> + * Copyright (c) 2019 Kevin Wolf
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> copy
> + * of this software and associated documentation files (the "Software"), to
> deal
> + * in the Software without restriction, including without limitation the
> rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
Standard request for new code: please make this GPLv2+, or tell us why
it has to be something else.
> +
> +#include "qemu/osdep.h"
> +
> +#include "block/block.h"
> +#include "crypto/init.h"
> +
> +#include "qapi/error.h"
> +#include "qemu-common.h"
> +#include "qemu-version.h"
> +#include "qemu/config-file.h"
> +#include "qemu/error-report.h"
> +#include "qemu/log.h"
> +#include "qemu/main-loop.h"
> +#include "qemu/module.h"
> +
> +#include "trace/control.h"
> +
> +#include
> +
> +static void help(void)
> +{
> +printf(
> +"Usage: %s [options]\n"
> +"QEMU storage daemon\n"
> +"\n"
> +" -h, --help display this help and exit\n"
> +" -T, --trace [[enable=]][,events=][,file=]\n"
> +" specify tracing options\n"
> +" -V, --version output version information and exit\n"
> +"\n"
> +QEMU_HELP_BOTTOM "\n",
> +error_get_progname());
> +}
> +
> +static int process_options(int argc, char *argv[], Error **errp)
> +{
> +int c;
> +char *trace_file = NULL;
> +int ret = -EINVAL;
> +
> +static const struct option long_options[] = {
> +{"help", no_argument, 0, 'h'},
You initialize member int *flag with 0 here,
> +{"version", no_argument, 0, 'V'},
> +{"trace", required_argument, NULL, 'T'},
... and with NULL here. Recommend to pick one and stick to it.
> +{0, 0, 0, 0}
{0} or {} would do.
> +};
> +
> +while ((c = getopt_long(argc, argv, ":hT:V", long_options, NULL)) != -1)
> {
> +switch (c) {
> +case '?':
> +error_setg(errp, "Unknown option '%s'", argv[optind - 1]);
> +goto out;
> +case ':':
> +error_setg(errp, "Missing option argument for '%s'",
> + argv[optind - 1]);
> +goto out;
> +case 'h':
> +help();
> +exit(EXIT_SUCCESS);
> +case 'V':
> +printf("qemu-storage-daemon version "
> + QEMU_FULL_VERSION "\n" QEMU_COPYRIGHT "\n");
> +exit(EXIT_SUCCESS);
> +case 'T':
> +g_free(trace_file);
> +trace_file = trace_opt_parse(optarg);
This is QemuOpts below the hood. Fact, not criticism :)
> +break;
> +}
Suggest (your preferred variation of) default: assert(0) to catch
omissions.
> +}
> +if (optind != argc) {
> +error_setg(errp, "Unexpected argument: %s", argv[optind]);
> +goto out;
> +}
> +
> +if
Re: [PATCH 0/4] Make the qemu_logfile handle thread safe.
Robert Foley writes: > This patch adds thread safety to the qemu_logfile handle. This now > allows changing the logfile while logging is active, and also solves > the issue of a seg fault while changing the logfile. > > This patch adds use of RCU for handling the swap out of the > old qemu_logfile file descriptor. I've finished my pass. Looks pretty good - a few minor comments around the persistence of the read lock and some minor stylistic nits. > > Robert Foley (4): > Add a mutex to guarantee single writer to qemu_logfile handle. > Add use of RCU for qemu_logfile. > qemu_log_lock/unlock now preserves the qemu_logfile handle. > Added tests for close and change of logfile. > > accel/tcg/cpu-exec.c | 4 +- > accel/tcg/translate-all.c | 4 +- > accel/tcg/translator.c| 4 +- > exec.c| 4 +- > hw/net/can/can_sja1000.c | 4 +- > include/exec/log.h| 33 ++-- > include/qemu/log.h| 51 +++--- > net/can/can_socketcan.c | 5 +- > target/cris/translate.c | 4 +- > target/i386/translate.c | 5 +- > target/lm32/translate.c | 4 +- > target/microblaze/translate.c | 4 +- > target/nios2/translate.c | 4 +- > target/tilegx/translate.c | 7 +-- > target/unicore32/translate.c | 4 +- > tcg/tcg.c | 28 ++ > tests/test-logging.c | 74 ++ > util/log.c| 99 --- > 18 files changed, 273 insertions(+), 69 deletions(-) -- Alex Bennée
[PATCH v3 20/22] iotests: Make 198 work with data_file
We do not care about the json:{} filenames here, so we can just filter
them out and thus make the test work both with and without external data
files.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/198 | 6 --
tests/qemu-iotests/198.out | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/tests/qemu-iotests/198 b/tests/qemu-iotests/198
index c8f824cfae..fb0d5a29d3 100755
--- a/tests/qemu-iotests/198
+++ b/tests/qemu-iotests/198
@@ -92,13 +92,15 @@ echo
echo "== checking image base =="
$QEMU_IMG info --image-opts $IMGSPECBASE | _filter_img_info --format-specific \
| sed -e "/^disk size:/ D" -e '/refcount bits:/ D' -e '/compat:/ D' \
- -e '/lazy refcounts:/ D' -e '/corrupt:/ D'
+ -e '/lazy refcounts:/ D' -e '/corrupt:/ D' -e '/^\s*data file/ D' \
+| _filter_json_filename
echo
echo "== checking image layer =="
$QEMU_IMG info --image-opts $IMGSPECLAYER | _filter_img_info --format-specific
\
| sed -e "/^disk size:/ D" -e '/refcount bits:/ D' -e '/compat:/ D' \
- -e '/lazy refcounts:/ D' -e '/corrupt:/ D'
+ -e '/lazy refcounts:/ D' -e '/corrupt:/ D' -e '/^\s*data file/ D' \
+| _filter_json_filename
# success, all done
diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out
index e86b175e39..831ce3a289 100644
--- a/tests/qemu-iotests/198.out
+++ b/tests/qemu-iotests/198.out
@@ -32,7 +32,7 @@ read 16777216/16777216 bytes at offset 0
16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
== checking image base ==
-image: json:{"encrypt.key-secret": "sec0", "driver": "IMGFMT", "file":
{"driver": "file", "filename": "TEST_DIR/t.IMGFMT.base"}}
+image: json:{ /* filtered */ }
file format: IMGFMT
virtual size: 16 MiB (16777216 bytes)
Format specific information:
@@ -74,7 +74,7 @@ Format specific information:
master key iters: 1024
== checking image layer ==
-image: json:{"encrypt.key-secret": "sec1", "driver": "IMGFMT", "file":
{"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}}
+image: json:{ /* filtered */ }
file format: IMGFMT
virtual size: 16 MiB (16777216 bytes)
backing file: TEST_DIR/t.IMGFMT.base
--
2.23.0
Re: [PATCH v2 2/4] Memory: Enable writeback for given memory region
Beata Michalska writes:
> On Wed, 6 Nov 2019 at 12:20, Richard Henderson
> wrote:
>>
>> On 11/6/19 12:40 AM, Beata Michalska wrote:
>> > +void qemu_ram_writeback(RAMBlock *block, ram_addr_t start, ram_addr_t
>> > length)
>> > +{
>> > +void *addr = ramblock_ptr(block, start);
>> > +
>> > +/*
>> > + * The requested range might spread up to the very end of the block
>> > + */
>> > +if ((start + length) > block->used_length) {
>> > +qemu_log("%s: sync range outside the block boundaries: "
>> > + "start: " RAM_ADDR_FMT " length: " RAM_ADDR_FMT
>> > + " block length: " RAM_ADDR_FMT " Narrowing down ..."
>> > ,
>> > + __func__, start, length, block->used_length);
>> > +length = block->used_length - start;
>> > +}
>>
>> qemu_log_mask w/ GUEST_ERROR? How do we expect the length to overflow?
>
> In theory it shouldn't, at least with current usage.
> I guess the probe_access will make sure of that.
> This was more of a precaution to enable catching potential/future misuses
> aka debugging purpose. I can get rid of that it that's playing too
> safe.
If the internal code might get it wrong and that would be a bug then the
g_assert(), if the values are ultimately from the guest then log with
GUEST_ERROR as Richard suggests.
--
Alex Bennée
Re: [PATCH 2/5] ipmi: Add support to customize OEM functions
On Thu, Nov 07, 2019 at 06:14:58PM +0100, Cédric Le Goater wrote: > >>> What's the plan for merging this, once it's ready? Is there an IPMI > >>> tree for it to be staged in? If not I could take it through the ppc > >>> tree, but I'd need some Acked-bys in that case. > >> > >> I have an IPMI tree for this. I was assuming it was going in to the PPC > >> tree, but it's not big deal. > > > > I'd be more comfortable if the generic ipmi changes went through the > > ipmi tree. > > Here is the patch : > > http://patchwork.ozlabs.org/patch/1185187/ Ok, I have this in my tree. I assume there is nothing like the linux-next tree for qemu, right? -corey > > > > Note that I've moved the initial ppc specific patch from > > my ppc-for-4.2 tree to my ppc-for-4.3 tree, since it missed my > > previous pull request and it's not really post-freeze material. > > OK. I was wondering where it had gone. > > Thanks, > > C.
Re: [PATCH v1 0/6] testing/next (netbsd stuff)
On Mon, 4 Nov 2019 at 17:39, Alex Bennée wrote: > > Hi, > > As we approach hard-freeze I'm trying to temper what comes in through > the testing/next tree. However it would be nice to get the NetBSD upto > speed with the other NetBSDs. Although the serial install is working > well for me this has had a rocky road so if others could also give it > a good testing that would be great. I've also disabled one of the > regular failing tests for non-Linux targets. There are other tests > that still fail however including the tests/test-aio-multithread which > asserts in the async utils around about 20% of the time: > > assertion "QSLIST_EMPTY(>scheduled_coroutines)" failed: file > "/home/qemu/qemu-test.nS1czd/src/util/async.c", line 279, function > "aio_ctx_finalize" This is unrelated to your NetBSD update in this series -- it's one of the persistent intermittents I see on the BSDs: https://lore.kernel.org/qemu-devel/20190916153312.GD25552@stefanha-x1.localdomain/t/ (though the failure rate I see is I think <20%, but I haven't really carefully measured it.) thanks -- PMM
Re: [PATCH v15 01/12] util/cutils: refactor do_strtosz() to support suffixes list
On Thu, Nov 07, 2019 at 03:45:00PM +0800, Tao Xu wrote: > Add do_strtomul() to convert string according to different suffixes. > > Signed-off-by: Tao Xu Reviewed-by: Eduardo Habkost -- Eduardo
Re: [PATCH for 4.2 v1 1/1] riscv/virt: Increase flash size
On Thu, 7 Nov 2019 at 17:09, Palmer Dabbelt wrote: > > On Wed, 06 Nov 2019 16:47:20 PST (-0800), Alistair Francis wrote: > > Coreboot developers have requested that they have at least 32MB of flash > > to load binaries. We currently have 32MB of flash, but it is split in > > two to allow loading two flash binaries. Let's increase the flash size > > from 32MB to 64MB to ensure we have a single region that is 32MB. > > > > No QEMU release has include flash in the RISC-V virt machine, so this > > isn't a breaking change. > > Even if we had, I wouldn't consider it a breaking change because it adds to > the memory map so existing programs will continue to run fine. I have a feeling you may find that some old command lines won't work any more because they specified a flash contents binary that was the old 32MB and now it needs to be padded out to 64MB. But I haven't tested whether this theory is correct (it will depend on how the flash contents are specified -- --bios will be ok, as will loading contents directly as an ELF file or similar, specifying contents by a -drive option intended to be consumed by the pflash is the case which likely needs extra padding.) thanks -- PMM
Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test
Le 07/11/2019 à 19:00, Philippe Mathieu-Daudé a écrit : > On 11/7/19 6:18 PM, Laurent Vivier wrote: >> Le 07/11/2019 à 17:38, Cleber Rosa a écrit : >>> - Original Message - From: "Eric Blake" To: "Cleber Rosa" , qemu-devel@nongnu.org Cc: "Peter Maydell" , "Eduardo Habkost" , "Philippe Mathieu-Daudé" , "Wainer dos Santos Moschetta" , "Laurent Vivier" , "Willian Rampazzo" , "Philippe Mathieu-Daudé" Sent: Thursday, November 7, 2019 10:43:08 AM Subject: Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test On 10/29/19 6:23 PM, Cleber Rosa wrote: > The boot_linux_console.py:BootLinuxConsole.test_m68k_q800 was very > recently merged, but between its last review and now, the Kernel > package used went missing. > meta-question: Why was this series posted in-reply-to the pull request, rather than as a new top-level thread? I nearly missed it because I don't expect to see unreviewed patches buried in threading like that. My workflow would have been to post the series in isolation, then manually reply to the pull request to mention the message-id of the related series proposed as a followup. >>> >>> Hi Eric, >>> >>> That was my attempt to signal that it was a fix to something which >>> had *just* >>> being merged as part of that pull request (though now caused by it). >>> >>> I basically did not know how to act properly, so I thank you for the >>> workflow >>> suggestion. I'll certainly follow it next time. >> >> IMHO, you should send your series and then replies to the pull request >> to tell you have sent your series that fixes the patch in the pull >> request, or vice-versa. >> >> But your series has been queued by Alex, so there is no problem... > > I prepared a different fix around the same time, but closed my laptop > before the patch was sent and noticed the next day: > https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg08120.html > > Laurent, are you OK with the new kernel being tested? > I'm fine. We could have problems with 5.4 because the address mapping has been changed for SWIM (aee6bff1c325 "m68k: mac: Revisit floppy disc controller base addresses), but this has been fixed by my patch that has been merged today in QEMU (653901ca2b "q800: fix I/O memory map"). Thanks, Laurent
Re: [Qemu-devel] [PATCH PULL 0/2] RDMA queue
On Wed, 6 Nov 2019 at 14:51, Marcel Apfelbaum wrote: > > The following changes since commit 412fbef3d076c43e56451bacb28c4544858c66a3: > > Merge remote-tracking branch > 'remotes/philmd-gitlab/tags/fw_cfg-next-pull-request' into staging > (2019-11-05 20:17:11 +) > > are available in the Git repository at: > > https://github.com/marcel-apf/qemu tags/rdma-pull-request > > for you to fetch changes up to 68b89aee710ab48b3dcaaa721bbc5d8aa5ea24d1: > > hw/rdma: Utilize ibv_reg_mr_iova for memory registration (2019-11-06 > 12:49:04 +0200) > > > RDMA queue > > * better memory registration performance > > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2 for any user-visible changes. -- PMM
Re: [PATCH 3/4] qemu_log_lock/unlock now preserves the qemu_logfile handle.
Robert Foley writes:
> qemu_log_lock() now returns a handle and qemu_log_unlock() receives a
> handle to unlock. This allows for changing the handle during logging
> and ensures the lock() and unlock() are for the same file.
Ahh there it is!
We probably want to put the API change through before we add the RCU
support - so swap the patch order around.
>
> Signed-off-by: Robert Foley
> ---
> include/qemu/log.h| 14 +++---
> accel/tcg/cpu-exec.c | 4 ++--
> accel/tcg/translate-all.c | 4 ++--
> accel/tcg/translator.c| 4 ++--
> exec.c| 4 ++--
> hw/net/can/can_sja1000.c | 4 ++--
> net/can/can_socketcan.c | 5 ++---
> target/cris/translate.c | 4 ++--
> target/i386/translate.c | 5 +++--
> target/lm32/translate.c | 4 ++--
> target/microblaze/translate.c | 4 ++--
> target/nios2/translate.c | 4 ++--
> target/tilegx/translate.c | 7 ---
> target/unicore32/translate.c | 4 ++--
> tcg/tcg.c | 16
> 15 files changed, 44 insertions(+), 43 deletions(-)
>
> diff --git a/include/qemu/log.h b/include/qemu/log.h
> index 975de18e23..3d0f47a479 100644
> --- a/include/qemu/log.h
> +++ b/include/qemu/log.h
> @@ -71,25 +71,25 @@ static inline bool qemu_log_separate(void)
> * qemu_loglevel is never set when qemu_logfile is unset.
> */
>
> -static inline void qemu_log_lock(void)
> +static inline FILE *qemu_log_lock(void)
> {
> QemuLogFile *logfile;
> rcu_read_lock();
> logfile = atomic_rcu_read(_logfile);
> if (logfile) {
> qemu_flockfile(logfile->fd);
> +return logfile->fd;
> }
> rcu_read_unlock();
> +return NULL;
> }
>
> -static inline void qemu_log_unlock(void)
> +static inline void qemu_log_unlock(FILE *fd)
> {
> -QemuLogFile *logfile;
> -logfile = atomic_rcu_read(_logfile);
> -if (logfile) {
> -qemu_funlockfile(logfile->fd);
> +if (fd) {
> +qemu_funlockfile(fd);
> +rcu_read_unlock();
> }
> -rcu_read_unlock();
> }
>
> /* Logging functions: */
> diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
> index c01f59c743..62068d10c3 100644
> --- a/accel/tcg/cpu-exec.c
> +++ b/accel/tcg/cpu-exec.c
> @@ -156,7 +156,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState *cpu,
> TranslationBlock *itb)
> #if defined(DEBUG_DISAS)
> if (qemu_loglevel_mask(CPU_LOG_TB_CPU)
> && qemu_log_in_addr_range(itb->pc)) {
> -qemu_log_lock();
> +FILE *logfile = qemu_log_lock();
> int flags = 0;
> if (qemu_loglevel_mask(CPU_LOG_TB_FPU)) {
> flags |= CPU_DUMP_FPU;
> @@ -165,7 +165,7 @@ static inline tcg_target_ulong cpu_tb_exec(CPUState *cpu,
> TranslationBlock *itb)
> flags |= CPU_DUMP_CCOP;
> #endif
> log_cpu_state(cpu, flags);
> -qemu_log_unlock();
> +qemu_log_unlock(logfile);
> }
> #endif /* DEBUG_DISAS */
>
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index 9f48da9472..bb325a2bc4 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -1804,7 +1804,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
> #ifdef DEBUG_DISAS
> if (qemu_loglevel_mask(CPU_LOG_TB_OUT_ASM) &&
> qemu_log_in_addr_range(tb->pc)) {
> -qemu_log_lock();
> +FILE *logfile = qemu_log_lock();
> qemu_log("OUT: [size=%d]\n", gen_code_size);
> if (tcg_ctx->data_gen_ptr) {
> size_t code_size = tcg_ctx->data_gen_ptr - tb->tc.ptr;
> @@ -1829,7 +1829,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
> }
> qemu_log("\n");
> qemu_log_flush();
> -qemu_log_unlock();
> +qemu_log_unlock(logfile);
> }
> #endif
>
> diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
> index f977682be7..603d17ff83 100644
> --- a/accel/tcg/translator.c
> +++ b/accel/tcg/translator.c
> @@ -138,11 +138,11 @@ void translator_loop(const TranslatorOps *ops,
> DisasContextBase *db,
> #ifdef DEBUG_DISAS
> if (qemu_loglevel_mask(CPU_LOG_TB_IN_ASM)
> && qemu_log_in_addr_range(db->pc_first)) {
> -qemu_log_lock();
> +FILE *logfile = qemu_log_lock();
> qemu_log("\n");
> ops->disas_log(db, cpu);
> qemu_log("\n");
> -qemu_log_unlock();
> +qemu_log_unlock(logfile);
> }
> #endif
> }
> diff --git a/exec.c b/exec.c
> index ffdb518535..c994a00f10 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -1223,13 +1223,13 @@ void cpu_abort(CPUState *cpu, const char *fmt, ...)
> fprintf(stderr, "\n");
> cpu_dump_state(cpu, stderr, CPU_DUMP_FPU | CPU_DUMP_CCOP);
> if (qemu_log_separate()) {
> -qemu_log_lock();
> +FILE *logfile = qemu_log_lock();
> qemu_log("qemu: fatal: ");
> qemu_log_vprintf(fmt, ap2);
> qemu_log("\n");
> log_cpu_state(cpu,
Re: [PATCH 4/4] Added tests for close and change of logfile.
Robert Foley writes:
> One test ensures that the logfile handle is still valid even if
> the logfile is changed during logging.
> The other test validates that the logfile handle remains valid under
> the logfile lock even if the logfile is closed.
>
> Signed-off-by: Robert Foley
> ---
> tests/test-logging.c | 74
> 1 file changed, 74 insertions(+)
>
> diff --git a/tests/test-logging.c b/tests/test-logging.c
> index a12585f70a..a3190ff92c 100644
> --- a/tests/test-logging.c
> +++ b/tests/test-logging.c
> @@ -108,6 +108,76 @@ static void test_parse_path(gconstpointer data)
> error_free_or_abort();
> }
>
> +static void test_logfile_write(gconstpointer data)
> +{
> +QemuLogFile *logfile;
> +gchar const *dir = data;
> +Error *err = NULL;
> +gchar *file_path;
> +gchar *file_path1;
with g_autofree char *file_path you can avoid the free down bellow.
> +FILE *orig_fd;
> +
> +file_path = g_build_filename(dir, "qemu_test_log_write0.log", NULL);
> +file_path1 = g_build_filename(dir, "qemu_test_log_write1.log", NULL);
> +
> +/*
> + * Test that even if an open file handle is changed,
> + * our handle remains valid due to RCU.
> + */
> +qemu_set_log_filename(file_path, );
> +g_assert(!err);
> +rcu_read_lock();
> +logfile = atomic_rcu_read(_logfile);
> +orig_fd = logfile->fd;
> +g_assert(logfile && logfile->fd);
> +fprintf(logfile->fd, "%s 1st write to file\n", __func__);
> +fflush(logfile->fd);
> +
> +/* Change the logfile and ensure that the handle is still valid. */
> +qemu_set_log_filename(file_path1, );
> +g_assert(!err);
Maybe better would be:
logfile2 = atomic_rcu_read(_logfile);
g_assert(logfile->fd == orig_fd);
g_assert(logfile2->fd != logfile->fd);
fprintf(logfile2->fd, "%s 2nd write to file\n", __func__);
fflush(logfile2->fd);
> +g_assert(logfile->fd == orig_fd);
> +fprintf(logfile->fd, "%s 2nd write to file\n", __func__);
> +fflush(logfile->fd);
> +rcu_read_unlock();
> +
> +g_free(file_path);
> +g_free(file_path1);
> +}
> +
> +static void test_logfile_lock(gconstpointer data)
> +{
> +FILE *logfile;
> +gchar const *dir = data;
> +Error *err = NULL;
> +gchar *file_path;
g_autofree
> +
> +file_path = g_build_filename(dir, "qemu_test_logfile_lock0.log", NULL);
> +
> +/*
> + * Test the use of the logfile lock, such
> + * that even if an open file handle is closed,
> + * our handle remains valid for use due to RCU.
> + */
> +qemu_set_log_filename(file_path, );
> +logfile = qemu_log_lock();
> +g_assert(logfile);
> +fprintf(logfile, "%s 1st write to file\n", __func__);
> +fflush(logfile);
> +
> +/*
> + * Initiate a close file and make sure our handle remains
> + * valid since we still have the logfile lock.
> + */
> +qemu_log_close();
> +fprintf(logfile, "%s 2nd write to file\n", __func__);
> +fflush(logfile);
> +qemu_log_unlock(logfile);
> +
> +g_assert(!err);
> +g_free(file_path);
> +}
> +
> /* Remove a directory and all its entries (non-recursive). */
> static void rmdir_full(gchar const *root)
> {
> @@ -134,6 +204,10 @@ int main(int argc, char **argv)
>
> g_test_add_func("/logging/parse_range", test_parse_range);
> g_test_add_data_func("/logging/parse_path", tmp_path, test_parse_path);
> +g_test_add_data_func("/logging/logfile_write_path",
> + tmp_path, test_logfile_write);
> +g_test_add_data_func("/logging/logfile_lock_path",
> + tmp_path, test_logfile_lock);
>
> rc = g_test_run();
--
Alex Bennée
[PATCH v3 15/22] iotests: Use _rm_test_img for deleting test images
Just rm will not delete external data files. Use _rm_test_img every
time we delete a test image.
(In the process, clean up the indentation of every _cleanup() this patch
touches.)
((Also, use quotes consistently. I am happy to see unquoted instances
like "rm -rf $TEST_DIR/..." go.))
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/005 | 2 +-
tests/qemu-iotests/019 | 6 +++---
tests/qemu-iotests/020 | 6 +++---
tests/qemu-iotests/024 | 10 +-
tests/qemu-iotests/028 | 2 +-
tests/qemu-iotests/029 | 2 +-
tests/qemu-iotests/043 | 4 +++-
tests/qemu-iotests/048 | 2 +-
tests/qemu-iotests/050 | 4 ++--
tests/qemu-iotests/053 | 4 ++--
tests/qemu-iotests/058 | 2 +-
tests/qemu-iotests/059 | 2 +-
tests/qemu-iotests/061 | 2 +-
tests/qemu-iotests/063 | 6 --
tests/qemu-iotests/069 | 2 +-
tests/qemu-iotests/074 | 2 +-
tests/qemu-iotests/080 | 2 +-
tests/qemu-iotests/081 | 6 +++---
tests/qemu-iotests/085 | 9 ++---
tests/qemu-iotests/088 | 2 +-
tests/qemu-iotests/092 | 2 +-
tests/qemu-iotests/094 | 2 +-
tests/qemu-iotests/095 | 5 +++--
tests/qemu-iotests/099 | 7 ---
tests/qemu-iotests/109 | 4 ++--
tests/qemu-iotests/110 | 4 ++--
tests/qemu-iotests/122 | 6 --
tests/qemu-iotests/123 | 2 +-
tests/qemu-iotests/141 | 4 +++-
tests/qemu-iotests/142 | 2 +-
tests/qemu-iotests/144 | 4 +++-
tests/qemu-iotests/153 | 10 +++---
tests/qemu-iotests/156 | 8 ++--
tests/qemu-iotests/159 | 2 +-
tests/qemu-iotests/160 | 3 ++-
tests/qemu-iotests/161 | 4 ++--
tests/qemu-iotests/170 | 2 +-
tests/qemu-iotests/172 | 6 +++---
tests/qemu-iotests/173 | 3 ++-
tests/qemu-iotests/178 | 2 +-
tests/qemu-iotests/182 | 2 +-
tests/qemu-iotests/183 | 2 +-
tests/qemu-iotests/185 | 4 ++--
tests/qemu-iotests/187 | 6 +++---
tests/qemu-iotests/190 | 2 +-
tests/qemu-iotests/191 | 6 +++---
tests/qemu-iotests/195 | 2 +-
tests/qemu-iotests/197 | 2 +-
tests/qemu-iotests/200 | 3 ++-
tests/qemu-iotests/215 | 2 +-
tests/qemu-iotests/225 | 2 +-
tests/qemu-iotests/229 | 3 ++-
tests/qemu-iotests/232 | 4 +++-
tests/qemu-iotests/243 | 2 +-
tests/qemu-iotests/244 | 4 ++--
tests/qemu-iotests/247 | 4 +++-
tests/qemu-iotests/249 | 4 ++--
tests/qemu-iotests/252 | 2 +-
58 files changed, 119 insertions(+), 96 deletions(-)
diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index 58442762fe..2b651f2c37 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -62,7 +62,7 @@ if [ "$IMGFMT" = "raw" ]; then
if ! truncate --size=5T "$TEST_IMG"; then
_notrun "file system on $TEST_DIR does not support large enough files"
fi
-rm "$TEST_IMG"
+_rm_test_img "$TEST_IMG"
fi
echo
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index b4f5234609..813a84acac 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -30,9 +30,9 @@ status=1 # failure is the default!
_cleanup()
{
- _cleanup_test_img
-rm -f "$TEST_IMG.base"
-rm -f "$TEST_IMG.orig"
+_cleanup_test_img
+_rm_test_img "$TEST_IMG.base"
+_rm_test_img "$TEST_IMG.orig"
}
trap "_cleanup; exit \$status" 0 1 2 3 15
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index f41b92f35f..20f8f185d0 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -28,9 +28,9 @@ status=1 # failure is the default!
_cleanup()
{
- _cleanup_test_img
-rm -f "$TEST_IMG.base"
-rm -f "$TEST_IMG.orig"
+_cleanup_test_img
+_rm_test_img "$TEST_IMG.base"
+_rm_test_img "$TEST_IMG.orig"
}
trap "_cleanup; exit \$status" 0 1 2 3 15
diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024
index 23298c6f59..e2e766241e 100755
--- a/tests/qemu-iotests/024
+++ b/tests/qemu-iotests/024
@@ -29,12 +29,12 @@ status=1# failure is the default!
_cleanup()
{
_cleanup_test_img
-rm -f "$TEST_DIR/t.$IMGFMT.base_old"
-rm -f "$TEST_DIR/t.$IMGFMT.base_new"
+_rm_test_img "$TEST_DIR/t.$IMGFMT.base_old"
+_rm_test_img "$TEST_DIR/t.$IMGFMT.base_new"
-rm -f "$TEST_DIR/subdir/t.$IMGFMT"
-rm -f "$TEST_DIR/subdir/t.$IMGFMT.base_old"
-rm -f "$TEST_DIR/subdir/t.$IMGFMT.base_new"
+_rm_test_img "$TEST_DIR/subdir/t.$IMGFMT"
+_rm_test_img "$TEST_DIR/subdir/t.$IMGFMT.base_old"
+_rm_test_img "$TEST_DIR/subdir/t.$IMGFMT.base_new"
rmdir "$TEST_DIR/subdir" 2> /dev/null
}
trap "_cleanup; exit \$status" 0 1 2 3 15
diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028
index bba1ee59ae..e2556d8e57 100755
--- a/tests/qemu-iotests/028
+++ b/tests/qemu-iotests/028
@@ -32,7 +32,7 @@ status=1 # failure is the default!
_cleanup()
{
_cleanup_qemu
-rm -f "${TEST_IMG}.copy"
+_rm_test_img "${TEST_IMG}.copy"
_cleanup_test_img
}
trap "_cleanup; exit \$status" 0 1 2 3 15
diff --git a/tests/qemu-iotests/029 b/tests/qemu-iotests/029
index 94c2713132..9254ede5e5 100755
--- a/tests/qemu-iotests/029
+++
[PATCH v3 12/22] iotests: Replace IMGOPTS='' by --no-opts
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/071 | 4 ++--
tests/qemu-iotests/174 | 2 +-
tests/qemu-iotests/178 | 4 ++--
tests/qemu-iotests/197 | 4 ++--
tests/qemu-iotests/215 | 4 ++--
5 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index fab52b..4e31943244 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -58,7 +58,7 @@ echo
echo "=== Testing blkverify through filename ==="
echo
-TEST_IMG="$TEST_IMG.base" IMGOPTS="" IMGFMT="raw" _make_test_img $IMG_SIZE |\
+TEST_IMG="$TEST_IMG.base" IMGFMT="raw" _make_test_img --no-opts $IMG_SIZE |\
_filter_imgfmt
_make_test_img $IMG_SIZE
$QEMU_IO -c "open -o
driver=raw,file.driver=blkverify,file.raw.filename=$TEST_IMG.base $TEST_IMG" \
@@ -73,7 +73,7 @@ echo
echo "=== Testing blkverify through file blockref ==="
echo
-TEST_IMG="$TEST_IMG.base" IMGOPTS="" IMGFMT="raw" _make_test_img $IMG_SIZE |\
+TEST_IMG="$TEST_IMG.base" IMGFMT="raw" _make_test_img --no-opts $IMG_SIZE |\
_filter_imgfmt
_make_test_img $IMG_SIZE
$QEMU_IO -c "open -o
driver=raw,file.driver=blkverify,file.raw.filename=$TEST_IMG.base,file.test.driver=$IMGFMT,file.test.file.filename=$TEST_IMG"
\
diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174
index 0a952a73fd..e2f14a38c6 100755
--- a/tests/qemu-iotests/174
+++ b/tests/qemu-iotests/174
@@ -40,7 +40,7 @@ _unsupported_fmt raw
size=256K
-IMGFMT=raw IMGKEYSECRET= IMGOPTS= _make_test_img $size | _filter_imgfmt
+IMGFMT=raw IMGKEYSECRET= _make_test_img --no-opts $size | _filter_imgfmt
echo
echo "== reading wrong format should fail =="
diff --git a/tests/qemu-iotests/178 b/tests/qemu-iotests/178
index 21231cadd3..75b5e8f314 100755
--- a/tests/qemu-iotests/178
+++ b/tests/qemu-iotests/178
@@ -62,8 +62,8 @@ $QEMU_IMG measure -O foo "$TEST_IMG" # unknown image file
format
make_test_img_with_fmt() {
# Shadow global variables within this function
-local IMGFMT="$1" IMGOPTS=""
-_make_test_img "$2"
+local IMGFMT="$1"
+_make_test_img --no-opts "$2"
}
qemu_io_with_fmt() {
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index 1d4f6786db..4d3d08ad6f 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -66,8 +66,8 @@ if [ "$IMGFMT" = "vpc" ]; then
fi
_make_test_img 4G
$QEMU_IO -c "write -P 55 3G 1k" "$TEST_IMG" | _filter_qemu_io
-IMGPROTO=file IMGFMT=qcow2 IMGOPTS= TEST_IMG_FILE="$TEST_WRAP" \
-_make_test_img -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create
+IMGPROTO=file IMGFMT=qcow2 TEST_IMG_FILE="$TEST_WRAP" \
+_make_test_img --no-opts -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create
$QEMU_IO -f qcow2 -c "write -z -u 1M 64k" "$TEST_WRAP" | _filter_qemu_io
# Ensure that a read of two clusters, but where one is already allocated,
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index 2eb377d682..55a1874dcd 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -63,8 +63,8 @@ if [ "$IMGFMT" = "vpc" ]; then
fi
_make_test_img 4G
$QEMU_IO -c "write -P 55 3G 1k" "$TEST_IMG" | _filter_qemu_io
-IMGPROTO=file IMGFMT=qcow2 IMGOPTS= TEST_IMG_FILE="$TEST_WRAP" \
-_make_test_img -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create
+IMGPROTO=file IMGFMT=qcow2 TEST_IMG_FILE="$TEST_WRAP" \
+_make_test_img --no-opts -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create
$QEMU_IO -f qcow2 -c "write -z -u 1M 64k" "$TEST_WRAP" | _filter_qemu_io
# Ensure that a read of two clusters, but where one is already allocated,
--
2.23.0
[PATCH v3 06/22] iotests: Replace IMGOPTS by _unsupported_imgopts
Some tests require compat=1.1 and thus set IMGOPTS='compat=1.1' globally. That is not how it should be done; instead, they should simply set _unsupported_imgopts to compat=0.10 (compat=1.1 is the default anyway). This makes the tests heed user-specified $IMGOPTS. Some do not work with all image options, though, so we need to disable them accordingly. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/036 | 3 +-- tests/qemu-iotests/060 | 4 ++-- tests/qemu-iotests/062 | 3 ++- tests/qemu-iotests/066 | 3 ++- tests/qemu-iotests/068 | 3 ++- tests/qemu-iotests/098 | 4 ++-- 6 files changed, 11 insertions(+), 9 deletions(-) diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036 index 5f929ad3be..bbaf0ef45b 100755 --- a/tests/qemu-iotests/036 +++ b/tests/qemu-iotests/036 @@ -43,9 +43,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file - # Only qcow2v3 and later supports feature bits -IMGOPTS="compat=1.1" +_unsupported_imgopts 'compat=0.10' echo echo === Image with unknown incompatible feature bit === diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 725e58a5a5..e0117aa4ae 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -48,6 +48,8 @@ _filter_io_error() _supported_fmt qcow2 _supported_proto file _supported_os Linux +# These tests only work for compat=1.1 images with refcount_bits=16 +_unsupported_imgopts 'compat=0.10' 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' rt_offset=65536 # 0x1 (XXX: just an assumption) rb_offset=131072 # 0x2 (XXX: just an assumption) @@ -55,8 +57,6 @@ l1_offset=196608 # 0x3 (XXX: just an assumption) l2_offset=262144 # 0x4 (XXX: just an assumption) l2_offset_after_snapshot=524288 # 0x8 (XXX: just an assumption) -IMGOPTS="compat=1.1" - OPEN_RW="open -o overlap-check=all $TEST_IMG" # Overlap checks are done before write operations only, therefore opening an # image read-only makes the overlap-check option irrelevant diff --git a/tests/qemu-iotests/062 b/tests/qemu-iotests/062 index 79738b1c26..0df8667e5a 100755 --- a/tests/qemu-iotests/062 +++ b/tests/qemu-iotests/062 @@ -40,8 +40,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto generic +# We need zero clusters and snapshots +_unsupported_imgopts 'compat=0.10' 'refcount_bits=1[^0-9]' -IMGOPTS="compat=1.1" IMG_SIZE=64M echo diff --git a/tests/qemu-iotests/066 b/tests/qemu-iotests/066 index cacbdb6ae0..71e8df598a 100755 --- a/tests/qemu-iotests/066 +++ b/tests/qemu-iotests/066 @@ -39,9 +39,10 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto generic +# We need zero clusters and snapshots +_unsupported_imgopts 'compat=0.10' 'refcount_bits=1[^0-9]' # Intentionally create an unaligned image -IMGOPTS="compat=1.1" IMG_SIZE=$((64 * 1024 * 1024 + 512)) echo diff --git a/tests/qemu-iotests/068 b/tests/qemu-iotests/068 index c164ccc64a..fe9d7ae1be 100755 --- a/tests/qemu-iotests/068 +++ b/tests/qemu-iotests/068 @@ -39,8 +39,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto generic +# Internal snapshots are (currently) impossible with refcount_bits=1 +_unsupported_imgopts 'compat=0.10' 'refcount_bits=1[^0-9]' -IMGOPTS="compat=1.1" IMG_SIZE=128K case "$QEMU_DEFAULT_MACHINE" in diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098 index 1c1d1c468f..700068b328 100755 --- a/tests/qemu-iotests/098 +++ b/tests/qemu-iotests/098 @@ -40,8 +40,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file - -IMGOPTS="compat=1.1" +# The code path we want to test here only works for compat=1.1 images +_unsupported_imgopts 'compat=0.10' for event in l1_update empty_image_prepare reftable_update refblock_alloc; do -- 2.23.0
[PATCH v3 08/22] iotests: Let _make_test_img parse its parameters
This will allow us to add more options than just -b.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/common.rc | 28
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index fa7bae2422..e12216b5f2 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -302,12 +302,12 @@ _make_test_img()
# extra qemu-img options can be added by tests
# at least one argument (the image size) needs to be added
local extra_img_options=""
-local image_size=$*
local optstr=""
local img_name=""
local use_backing=0
local backing_file=""
local object_options=""
+local misc_params=()
if [ -n "$TEST_IMG_FILE" ]; then
img_name=$TEST_IMG_FILE
@@ -323,11 +323,23 @@ _make_test_img()
optstr=$(_optstr_add "$optstr" "key-secret=keysec0")
fi
-if [ "$1" = "-b" ]; then
-use_backing=1
-backing_file=$2
-image_size=$3
-fi
+for param; do
+if [ "$use_backing" = "1" -a -z "$backing_file" ]; then
+backing_file=$param
+continue
+fi
+
+case "$param" in
+-b)
+use_backing=1
+;;
+
+*)
+misc_params=("${misc_params[@]}" "$param")
+;;
+esac
+done
+
if [ \( "$IMGFMT" = "qcow2" -o "$IMGFMT" = "qed" \) -a -n "$CLUSTER_SIZE"
]; then
optstr=$(_optstr_add "$optstr" "cluster_size=$CLUSTER_SIZE")
fi
@@ -343,9 +355,9 @@ _make_test_img()
# XXX(hch): have global image options?
(
if [ $use_backing = 1 ]; then
-$QEMU_IMG create $object_options -f $IMGFMT $extra_img_options -b
"$backing_file" "$img_name" $image_size 2>&1
+$QEMU_IMG create $object_options -f $IMGFMT $extra_img_options -b
"$backing_file" "$img_name" "${misc_params[@]}" 2>&1
else
-$QEMU_IMG create $object_options -f $IMGFMT $extra_img_options
"$img_name" $image_size 2>&1
+$QEMU_IMG create $object_options -f $IMGFMT $extra_img_options
"$img_name" "${misc_params[@]}" 2>&1
fi
) | _filter_img_create
--
2.23.0
Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test
- Original Message - > From: "Eric Blake" > To: "Cleber Rosa" , qemu-devel@nongnu.org > Cc: "Peter Maydell" , "Eduardo Habkost" > , "Philippe Mathieu-Daudé" > , "Wainer dos Santos Moschetta" , > "Laurent Vivier" , > "Willian Rampazzo" , "Philippe Mathieu-Daudé" > > Sent: Thursday, November 7, 2019 10:43:08 AM > Subject: Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test > > On 10/29/19 6:23 PM, Cleber Rosa wrote: > > The boot_linux_console.py:BootLinuxConsole.test_m68k_q800 was very > > recently merged, but between its last review and now, the Kernel > > package used went missing. > > > > meta-question: Why was this series posted in-reply-to the pull request, > rather than as a new top-level thread? I nearly missed it because I > don't expect to see unreviewed patches buried in threading like that. > My workflow would have been to post the series in isolation, then > manually reply to the pull request to mention the message-id of the > related series proposed as a followup. > Hi Eric, That was my attempt to signal that it was a fix to something which had *just* being merged as part of that pull request (though now caused by it). I basically did not know how to act properly, so I thank you for the workflow suggestion. I'll certainly follow it next time. Thanks a lot! - Cleber. > -- > Eric Blake, Principal Software Engineer > Red Hat, Inc. +1-919-301-3226 > Virtualization: qemu.org | libvirt.org >
Re: [PATCH v2 20/21] iotests: Disable data_file where it cannot be used
On 07.11.19 16:19, Maxim Levitsky wrote: > On Thu, 2019-11-07 at 12:36 +0100, Max Reitz wrote: >> On 06.11.19 16:52, Maxim Levitsky wrote: >>> On Tue, 2019-10-15 at 16:27 +0200, Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/007 | 5 +++-- tests/qemu-iotests/014 | 2 ++ tests/qemu-iotests/015 | 5 +++-- tests/qemu-iotests/026 | 5 - tests/qemu-iotests/029 | 5 +++-- tests/qemu-iotests/031 | 6 +++--- tests/qemu-iotests/036 | 5 +++-- tests/qemu-iotests/039 | 3 +++ tests/qemu-iotests/046 | 2 ++ tests/qemu-iotests/048 | 2 ++ tests/qemu-iotests/051 | 5 +++-- tests/qemu-iotests/058 | 5 +++-- tests/qemu-iotests/060 | 6 -- tests/qemu-iotests/061 | 6 -- tests/qemu-iotests/062 | 2 +- tests/qemu-iotests/066 | 2 +- tests/qemu-iotests/067 | 6 -- tests/qemu-iotests/068 | 5 +++-- tests/qemu-iotests/071 | 3 +++ tests/qemu-iotests/073 | 2 ++ tests/qemu-iotests/074 | 2 ++ tests/qemu-iotests/080 | 5 +++-- tests/qemu-iotests/090 | 2 ++ tests/qemu-iotests/098 | 6 -- tests/qemu-iotests/099 | 3 ++- tests/qemu-iotests/103 | 5 +++-- tests/qemu-iotests/108 | 6 -- tests/qemu-iotests/112 | 5 +++-- tests/qemu-iotests/114 | 2 ++ tests/qemu-iotests/121 | 3 +++ tests/qemu-iotests/138 | 2 ++ tests/qemu-iotests/156 | 2 ++ tests/qemu-iotests/176 | 7 +-- tests/qemu-iotests/191 | 2 ++ tests/qemu-iotests/201 | 6 +++--- tests/qemu-iotests/214 | 3 ++- tests/qemu-iotests/217 | 3 ++- tests/qemu-iotests/220 | 5 +++-- tests/qemu-iotests/243 | 6 -- tests/qemu-iotests/244 | 5 +++-- tests/qemu-iotests/250 | 2 ++ tests/qemu-iotests/267 | 5 +++-- 42 files changed, 117 insertions(+), 52 deletions(-) >> >> [...] >> diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031 index c44fcf91bb..646ecd593f 100755 --- a/tests/qemu-iotests/031 +++ b/tests/qemu-iotests/031 @@ -40,9 +40,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file -# We want to test compat=0.10, which does not support refcount widths -# other than 16 -_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' +# We want to test compat=0.10, which does not support external data +# files or refcount widths other than 16 +_unsupported_imgopts data_file 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' >>> >>> This is maybe another reason to split this test for compat=0.10 and for >>> compat=1.1 >>> But still can be done later of course. >> >> Hm, but I don’t really think this test is important for external data >> files. There is no I/O. > I guess both yes and no, the external data file is a header extension as well. Yes, but the test already involves a header extension. diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036 index bbaf0ef45b..512598421c 100755 --- a/tests/qemu-iotests/036 +++ b/tests/qemu-iotests/036 @@ -43,8 +43,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 _supported_proto file -# Only qcow2v3 and later supports feature bits -_unsupported_imgopts 'compat=0.10' +# Only qcow2v3 and later supports feature bits; +# qcow2.py does not support external data files >>> >>> Minor nitpick, maybe tag this with TODO or so. No need to do now. >> >> Hm, well, and the same applies here. (Just not a very important test.) > Same here, in theory external data file is a feature, and it could > 'interact' with other features, but most likely you are right here as well. Well, but the test currently doesn’t involve any known feature bits. It’s mostly about checking what our qcow2 driver does with unknown feature bits. (If it wanted to involve known feature bits, it could have easily used e.g. the dirty feature.) diff --git a/tests/qemu-iotests/048 b/tests/qemu-iotests/048 index a8feb76184..2af6b74b41 100755 --- a/tests/qemu-iotests/048 +++ b/tests/qemu-iotests/048 @@ -49,6 +49,8 @@ _compare() _supported_fmt raw qcow2 qed luks _supported_proto file _supported_os Linux +# Using 'cp' is incompatible with external data files +_unsupported_imgopts data_file >>> >>> You could compare the external files instead in theory *I think*. >>> Another item on some TODO list I guess. >> >> This is a test of qemu-img compare, not of the image format. So it >> doesn’t make much sense to me to compare the external files, and also it >> should be completely sufficient to run this test only without external >> data files. > Yes but on the other hand, its is kind of nice to test that it can compare > correctly > two qcow2 files which have external data
Re: [PATCH v3 21/22] iotests: Disable data_file where it cannot be used
On 07.11.19 17:37, Max Reitz wrote: > Signed-off-by: Max Reitz > --- > tests/qemu-iotests/007 | 5 +++-- > tests/qemu-iotests/014 | 2 ++ > tests/qemu-iotests/015 | 5 +++-- > tests/qemu-iotests/026 | 5 - > tests/qemu-iotests/029 | 5 +++-- > tests/qemu-iotests/031 | 6 +++--- > tests/qemu-iotests/036 | 5 +++-- > tests/qemu-iotests/039 | 3 +++ > tests/qemu-iotests/046 | 2 ++ > tests/qemu-iotests/048 | 2 ++ > tests/qemu-iotests/051 | 5 +++-- > tests/qemu-iotests/058 | 5 +++-- > tests/qemu-iotests/060 | 6 -- > tests/qemu-iotests/061 | 6 -- > tests/qemu-iotests/062 | 2 +- > tests/qemu-iotests/066 | 4 +++- > tests/qemu-iotests/067 | 6 -- > tests/qemu-iotests/068 | 5 +++-- > tests/qemu-iotests/071 | 3 +++ > tests/qemu-iotests/073 | 4 > tests/qemu-iotests/074 | 2 ++ > tests/qemu-iotests/080 | 5 +++-- > tests/qemu-iotests/090 | 2 ++ > tests/qemu-iotests/098 | 6 -- > tests/qemu-iotests/099 | 3 ++- > tests/qemu-iotests/103 | 5 +++-- > tests/qemu-iotests/108 | 6 -- > tests/qemu-iotests/112 | 5 +++-- > tests/qemu-iotests/114 | 2 ++ > tests/qemu-iotests/121 | 3 +++ > tests/qemu-iotests/138 | 3 +++ > tests/qemu-iotests/156 | 2 ++ > tests/qemu-iotests/176 | 7 +-- > tests/qemu-iotests/191 | 2 ++ > tests/qemu-iotests/201 | 6 +++--- > tests/qemu-iotests/214 | 3 ++- > tests/qemu-iotests/217 | 3 ++- > tests/qemu-iotests/220 | 5 +++-- > tests/qemu-iotests/243 | 6 -- > tests/qemu-iotests/244 | 5 +++-- > tests/qemu-iotests/250 | 2 ++ > tests/qemu-iotests/261 | 3 ++- > tests/qemu-iotests/267 | 5 +++-- > 43 files changed, 124 insertions(+), 53 deletions(-) [...] > diff --git a/tests/qemu-iotests/067 b/tests/qemu-iotests/067 > index 926c79b37c..3bc6e719eb 100755 > --- a/tests/qemu-iotests/067 > +++ b/tests/qemu-iotests/067 > @@ -32,8 +32,10 @@ status=1 # failure is the default! > > _supported_fmt qcow2 > _supported_proto file > -# Because anything other than 16 would change the output of query-block > -_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' > +# Because anything other than 16 would change the output of query-block, > +# and external data files would change the output of > +# query-named-block-ndoes s/ndoes/nodes/... signature.asc Description: OpenPGP digital signature
[PATCH v3 18/22] iotests: Make 110 work with data_file
The only difference is that the json:{} filename of the image looks
different. We actually do not care about that filename in this test, we
are only interested in (1) that there is a json:{} filename, and (2)
whether the backing filename can be constructed.
So just filter out the json:{} data, thus making this test pass both
with and without data_file.
Signed-off-by: Max Reitz
Reviewed-by: Maxim Levitsky
---
tests/qemu-iotests/110 | 7 +--
tests/qemu-iotests/110.out | 4 ++--
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index f78df0e6e1..139c02c2cf 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -67,6 +67,7 @@ echo
# Across blkdebug without a config file, you cannot reconstruct filenames, so
# qemu is incapable of knowing the directory of the top image from the filename
# alone. However, using bdrv_dirname(), it should still work.
+# (Filter out the json:{} filename so this test works with external data files)
TEST_IMG="json:{
'driver': '$IMGFMT',
'file': {
@@ -82,7 +83,8 @@ TEST_IMG="json:{
}
]
}
-}" _img_info | _filter_img_info | grep -v 'backing file format'
+}" _img_info | _filter_img_info | grep -v 'backing file format' \
+| _filter_json_filename
echo
echo '=== Backing name is always relative to the backed image ==='
@@ -114,7 +116,8 @@ TEST_IMG="json:{
}
]
}
-}" _img_info | _filter_img_info | grep -v 'backing file format'
+}" _img_info | _filter_img_info | grep -v 'backing file format' \
+| _filter_json_filename
# success, all done
diff --git a/tests/qemu-iotests/110.out b/tests/qemu-iotests/110.out
index f60b26390e..f835553a99 100644
--- a/tests/qemu-iotests/110.out
+++ b/tests/qemu-iotests/110.out
@@ -11,7 +11,7 @@ backing file: t.IMGFMT.base (actual path:
TEST_DIR/t.IMGFMT.base)
=== Non-reconstructable filename ===
-image: json:{"driver": "IMGFMT", "file": {"set-state.0.event": "read_aio",
"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver":
"blkdebug", "set-state.0.new_state": 42}}
+image: json:{ /* filtered */ }
file format: IMGFMT
virtual size: 64 MiB (67108864 bytes)
backing file: t.IMGFMT.base (actual path: TEST_DIR/t.IMGFMT.base)
@@ -22,7 +22,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
backing_file=t.IMGFMT.b
=== Nodes without a common directory ===
-image: json:{"driver": "IMGFMT", "file": {"children": [{"driver": "file",
"filename": "TEST_DIR/t.IMGFMT"}, {"driver": "file", "filename":
"TEST_DIR/t.IMGFMT.copy"}], "driver": "quorum", "vote-threshold": 1}}
+image: json:{ /* filtered */ }
file format: IMGFMT
virtual size: 64 MiB (67108864 bytes)
backing file: t.IMGFMT.base (cannot determine actual path)
--
2.23.0
Re: [PATCH] ppc/pnv: Add a "/qemu" device tree node
On Wed, Nov 06, 2019 at 03:21:29PM +0100, Cédric Le Goater wrote: > It helps skiboot identifying that is running on a QEMU platform. The > compatible string will define the POWERPC processor version. > > Signed-off-by: Cédric Le Goater Applied to ppc-for-4.3. > --- > hw/ppc/pnv.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c > index 627c08e5b985..4c3d5184126a 100644 > --- a/hw/ppc/pnv.c > +++ b/hw/ppc/pnv.c > @@ -479,6 +479,9 @@ static void *pnv_dt_create(MachineState *machine) > fdt = g_malloc0(FDT_MAX_SIZE); > _FDT((fdt_create_empty_tree(fdt, FDT_MAX_SIZE))); > > +/* /qemu node */ > +_FDT((fdt_add_subnode(fdt, 0, "qemu"))); > + > /* Root node */ > _FDT((fdt_setprop_cell(fdt, 0, "#address-cells", 0x2))); > _FDT((fdt_setprop_cell(fdt, 0, "#size-cells", 0x2))); -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
Re: [PATCH 2/5] ipmi: Add support to customize OEM functions
>>> What's the plan for merging this, once it's ready? Is there an IPMI >>> tree for it to be staged in? If not I could take it through the ppc >>> tree, but I'd need some Acked-bys in that case. >> >> I have an IPMI tree for this. I was assuming it was going in to the PPC >> tree, but it's not big deal. > > I'd be more comfortable if the generic ipmi changes went through the > ipmi tree. Here is the patch : http://patchwork.ozlabs.org/patch/1185187/ > Note that I've moved the initial ppc specific patch from > my ppc-for-4.2 tree to my ppc-for-4.3 tree, since it missed my > previous pull request and it's not really post-freeze material. OK. I was wondering where it had gone. Thanks, C.
Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test
On 11/7/19 6:18 PM, Laurent Vivier wrote: Le 07/11/2019 à 17:38, Cleber Rosa a écrit : - Original Message - From: "Eric Blake" To: "Cleber Rosa" , qemu-devel@nongnu.org Cc: "Peter Maydell" , "Eduardo Habkost" , "Philippe Mathieu-Daudé" , "Wainer dos Santos Moschetta" , "Laurent Vivier" , "Willian Rampazzo" , "Philippe Mathieu-Daudé" Sent: Thursday, November 7, 2019 10:43:08 AM Subject: Re: [PATCH 0/2] Acceptance test: update kernel used on m68k/q800 test On 10/29/19 6:23 PM, Cleber Rosa wrote: The boot_linux_console.py:BootLinuxConsole.test_m68k_q800 was very recently merged, but between its last review and now, the Kernel package used went missing. meta-question: Why was this series posted in-reply-to the pull request, rather than as a new top-level thread? I nearly missed it because I don't expect to see unreviewed patches buried in threading like that. My workflow would have been to post the series in isolation, then manually reply to the pull request to mention the message-id of the related series proposed as a followup. Hi Eric, That was my attempt to signal that it was a fix to something which had *just* being merged as part of that pull request (though now caused by it). I basically did not know how to act properly, so I thank you for the workflow suggestion. I'll certainly follow it next time. IMHO, you should send your series and then replies to the pull request to tell you have sent your series that fixes the patch in the pull request, or vice-versa. But your series has been queued by Alex, so there is no problem... I prepared a different fix around the same time, but closed my laptop before the patch was sent and noticed the next day: https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg08120.html Laurent, are you OK with the new kernel being tested?
[Bug 1851664] Re: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine
Please find the above qemu command to lunch guest machine -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1851664 Title: qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine Status in QEMU: Incomplete Bug description: We are trying to attach 6 VF's to the guest machine on 4.1.1 qemu emulator. We are observing "VFIO_MAP_DMA : -28" error. We are using w-bits=48 bits while lunching VM. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1851664/+subscriptions
Re: [PATCH 2/3] Acceptance tests: introduce utility method for tags unique vals
On 10/28/19 8:02 PM, Cleber Rosa wrote:
On Thu, Oct 24, 2019 at 06:12:25PM -0300, Wainer dos Santos Moschetta wrote:
Hi Cleber,
On 9/24/19 4:45 PM, Cleber Rosa wrote:
Currently a test can describe the target architecture binary that it
should primarily be run with, be setting a single tag value.
The same approach is expected to be done with other QEMU aspects to be
tested, for instance, the machine type and accelerator, so let's
generalize the logic into a utility method.
Signed-off-by: Cleber Rosa
---
tests/acceptance/avocado_qemu/__init__.py | 19 +--
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/tests/acceptance/avocado_qemu/__init__.py
b/tests/acceptance/avocado_qemu/__init__.py
index bd41e0443c..02775bafcf 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -54,14 +54,21 @@ def pick_default_qemu_bin(arch=None):
class Test(avocado.Test):
+def _get_unique_tag_val(self, tag_name):
+"""
+Gets a tag value, if unique for a key
+"""
+vals = self.tags.get(tag_name, [])
+if len(vals) == 1:
An small optimization:
if vals:
return vals.pop()
IIUC, this would break the idea of uniqueness that this method, for
now, has. Read on.
+return vals.pop()
+return None
Does it allows to express a scenario like "I want my test method to run on
x86_64 and aarch64" using tags? If so, _get_unique_tag_val logic returns
None for multi-value tags (e.g. 'tags=arch:x86_64,arch:aarch64').
I thought that initially we should attempt to pick a default arch or
machine type only of len(vals) == 1. Not because what you describe
can't be done, but because I would like to go through the tests and
make sure we run them in all the given tagged arches when we allow
that.
Ok, understood the rationale now.
Reviewed-by: Wainer dos Santos Moschetta
Thanks,
- Cleber.
Thanks,
Wainer
+
def setUp(self):
self._vms = {}
-arches = self.tags.get('arch', [])
-if len(arches) == 1:
-arch = arches.pop()
-else:
-arch = None
-self.arch = self.params.get('arch', default=arch)
+
+self.arch = self.params.get('arch',
+default=self._get_unique_tag_val('arch'))
+
default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
self.qemu_bin = self.params.get('qemu_bin',
default=default_qemu_bin)
Re: [qemu-web PATCH] Add device fuzzing blog post
- Original Message - > From: "Stefan Hajnoczi" > Sent: Thursday, November 7, 2019 10:11:36 AM > > This blog post covers the device fuzzing GSoC project that Alexander > Olenik did in 2019. [...] > +This article was contributed by Stefan Hajnoczi and Alexander Oleinik. You could also use the "author:" field in the header for this. > diff --git a/screenshots/fuzzing-intro.png b/screenshots/fuzzing-intro.png > new file mode 100644 > index 000..e130027 > Binary files /dev/null and b/screenshots/fuzzing-intro.png differ > diff --git a/screenshots/fuzzing.png b/screenshots/fuzzing.png > new file mode 100644 > index 000..2f15ecb > Binary files /dev/null and b/screenshots/fuzzing.png differ Seems like the images are missing ... can you please attach them? Thanks, Thomas
Re: [PATCH v2 10/21] iotests: Replace IMGOPTS= by -o
On Thu, 2019-11-07 at 10:20 +0100, Max Reitz wrote: > On 06.11.19 16:47, Maxim Levitsky wrote: > > On Tue, 2019-10-15 at 16:27 +0200, Max Reitz wrote: > > > Tests should not overwrite all user-supplied image options, but only add > > > to it (which will effectively overwrite conflicting values). Accomplish > > > this by passing options to _make_test_img via -o instead of $IMGOPTS. > > > > > > For some tests, there is no functional change because they already only > > > appended options to IMGOPTS. For these, this patch is just a > > > simplification. > > > > > > For others, this is a change, so they now heed user-specified $IMGOPTS. > > > Some of those tests do not work with all image options, though, so we > > > need to disable them accordingly. > > > > > > Signed-off-by: Max Reitz > > > --- > > > tests/qemu-iotests/031 | 9 --- > > > tests/qemu-iotests/039 | 24 ++ > > > tests/qemu-iotests/059 | 18 ++--- > > > tests/qemu-iotests/060 | 6 ++--- > > > tests/qemu-iotests/061 | 57 ++ > > > tests/qemu-iotests/079 | 3 +-- > > > tests/qemu-iotests/106 | 2 +- > > > tests/qemu-iotests/108 | 2 +- > > > tests/qemu-iotests/112 | 32 > > > tests/qemu-iotests/115 | 3 +-- > > > tests/qemu-iotests/121 | 6 ++--- > > > tests/qemu-iotests/125 | 2 +- > > > tests/qemu-iotests/137 | 2 +- > > > tests/qemu-iotests/138 | 3 +-- > > > tests/qemu-iotests/175 | 2 +- > > > tests/qemu-iotests/190 | 2 +- > > > tests/qemu-iotests/191 | 3 +-- > > > tests/qemu-iotests/220 | 4 ++- > > > tests/qemu-iotests/243 | 6 +++-- > > > tests/qemu-iotests/244 | 10 +--- > > > tests/qemu-iotests/250 | 3 +-- > > > tests/qemu-iotests/265 | 2 +- > > > 22 files changed, 100 insertions(+), 101 deletions(-) > > [...] > > > > @@ -161,7 +161,7 @@ _cleanup_test_img > > > > > > echo > > > echo "=== Testing 4TB monolithicFlat creation and IO ===" > > > -IMGOPTS="subformat=monolithicFlat" _make_test_img 4T > > > +_make_test_img -o "subformat=monolithicFlat" 4T > > > _img_info > > > $QEMU_IO -c "write -P 0xa 900G 512" "$TEST_IMG" | _filter_qemu_io > > > $QEMU_IO -c "read -v 900G 1024" "$TEST_IMG" | _filter_qemu_io > > > @@ -170,7 +170,7 @@ _cleanup_test_img > > > echo > > > echo "=== Testing qemu-img map on extents ===" > > > for fmt in monolithicSparse twoGbMaxExtentSparse; do > > > -IMGOPTS="subformat=$fmt" _make_test_img 31G > > > +_make_test_img -o "subformat=$fmt" 31G > > > $QEMU_IO -c "write 65024 1k" "$TEST_IMG" | _filter_qemu_io > > > $QEMU_IO -c "write 2147483136 1k" "$TEST_IMG" | _filter_qemu_io > > > $QEMU_IO -c "write 5G 1k" "$TEST_IMG" | _filter_qemu_io > > > > Looks good. Another test that pokes at guessed locations... :-) > > Actually, no. These are writes on the format, not the file itself. The > monolithicSparse subformat will store everything in a single file, > whereas twoGbMaxExtentSparse will create one file per 2 GB of guest > disk. So the locations are chosen accordingly to that 2 GB limit ((1) > something somewhere in the first extent, (2) something that wraps around > the first 2 GB limit, so hits extents #0 and #1, and (3) something in > the middle of extent #2.) > > (The following qemu-img map call then verifies that it lands in the > different files for twoGbMaxExtentSparse, and that monolithicSparse is > at least indeed sparsely allocated.) > > Max Good to know, I missed this one. Best regards, Maxim Levitsky
[Bug 1848556] Re: qemu-img check failing on remote image in Eoan
This bug was fixed in the package qemu - 1:4.0+dfsg-0ubuntu10 --- qemu (1:4.0+dfsg-0ubuntu10) focal; urgency=medium * d/p/ubuntu/lp-1848556-curl-Handle-success-in-multi_check_completion.patch: fix a potential hang when qemu or qemu-img where accessing http backed disks via libcurl (LP: #1848556) * d/p/u/lp-1848497-virtio-balloon-fix-QEMU-4.0-config-size-migration-in.patch: fix migration issue from qemu <4.0 when using virtio-balloon (LP: #1848497) -- Christian Ehrhardt Mon, 21 Oct 2019 14:51:45 +0200 ** Changed in: qemu (Ubuntu Focal) Status: Triaged => Fix Released -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1848556 Title: qemu-img check failing on remote image in Eoan Status in QEMU: Fix Released Status in qemu package in Ubuntu: Fix Released Status in qemu source package in Eoan: Triaged Status in qemu source package in Focal: Fix Released Bug description: Ubuntu SRU Template: [Impact] * There is fallout due to changes in libcurl that affect qemu and might lead to a hang. * Fix by backporting the upstream fix [Test Case] * If you have network just run $ qemu-img check http://10.193.37.117/cloud/eoan-server-cloudimg-amd64.img * Without network, install apache2, and get a complex qemu file (like a cloud image) onto the system. Then access the file via apache http but not localhost (that would work) [Regression Potential] * The change is local to the libcurl usage of qemu, so that could be affected. But then this is what has been found to not work here, so I'd expect not too much trouble. But if so then in the curl usage (which means disks on http) [Other Info] * n/a --- The "qemu-img check" function is failing on remote (HTTP-hosted) images, beginning with Ubuntu 19.10 (qemu-utils version 1:4.0+dfsg- 0ubuntu9). With previous versions, through Ubuntu 19.04/qemu-utils version 1:3.1+dfsg-2ubuntu3.5, the following worked: $ /usr/bin/qemu-img check http://10.193.37.117/cloud/eoan-server-cloudimg-amd64.img No errors were found on the image. 19778/36032 = 54.89% allocated, 90.34% fragmented, 89.90% compressed clusters Image end offset: 514064384 The 10.193.37.117 server holds an Apache server that hosts the cloud images on a LAN. Beginning with Ubuntu 19.10/qemu-utils 1:4.0+dfsg- 0ubuntu9, the same command never returns. (I've left it for up to an hour with no change.) I'm able to wget the image from the same server and installation on which qemu-img check fails. I've tried several .img files on the server, ranging from Bionic to Eoan, with the same results with all of them. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1848556/+subscriptions
Re: [PULL 0/1] Audio 20191106 patches
On Wed, 6 Nov 2019 at 08:27, Gerd Hoffmann wrote: > > The following changes since commit 36609b4fa36f0ac934874371874416f7533a5408: > > Merge remote-tracking branch > 'remotes/palmer/tags/palmer-for-master-4.2-sf1' into staging (2019-11-02 > 17:59:03 +) > > are available in the Git repository at: > > git://git.kraxel.org/qemu tags/audio-20191106-pull-request > > for you to fetch changes up to 14d4f01191354e9520c47c692007344c30ab358b: > > audio: add -audiodev pa,in|out.latency= to documentation (2019-11-06 > 08:08:10 +0100) > > > audio: documentation update > > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/4.2 for any user-visible changes. -- PMM
Re: QEMU HTML documentation now on qemu.org
On Wed, Nov 06, 2019 at 05:19:28PM +0100, Stefan Hajnoczi wrote: > Hi, > You can now access the latest QEMU HTML documentation built from > qemu.git/master nightly at: > > https://wiki.qemu.org/docs/qemu-doc.html > https://wiki.qemu.org/docs/qemu-qmp-ref.html > https://wiki.qemu.org/docs/qemu-ga-ref.html > ...as well as interop/ and specs/ > > Feel free to link to the documentation from the QEMU website and/or > wiki! What's the reason for putting on wiki.qemu.org URL ? It feels like having it under www.qemu.org would be a more natural home, especially if we can then make it pick up the jekyll theme around the pages. Ideally we should publish the docs under versioned URL when we make a release. eg /docs/latest/ for current GIT master which I presume the above is tracking, and then a /docs/$VERSION/... for each major release we cut. That way users can get an accurate view of features in the QEMU they are actually using. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [RFC PATCH 18/18] qemu-storage-daemon: Add --monitor option
Am 06.11.2019 um 15:32 hat Max Reitz geschrieben:
> On 17.10.19 15:02, Kevin Wolf wrote:
> > This adds and parses the --monitor option, so that a QMP monitor can be
> > used in the storage daemon. The monitor offers commands defined in the
> > QAPI schema at storage-daemon/qapi/qapi-schema.json.
> >
> > Signed-off-by: Kevin Wolf
> > ---
> > storage-daemon/qapi/qapi-schema.json | 15
> > qemu-storage-daemon.c| 34
> > Makefile | 30
> > Makefile.objs| 4 ++--
> > monitor/Makefile.objs| 2 ++
> > qapi/Makefile.objs | 5
> > qom/Makefile.objs| 1 +
> > scripts/qapi/gen.py | 5
> > storage-daemon/Makefile.objs | 1 +
> > storage-daemon/qapi/Makefile.objs| 1 +
> > 10 files changed, 96 insertions(+), 2 deletions(-)
> > create mode 100644 storage-daemon/qapi/qapi-schema.json
> > create mode 100644 storage-daemon/Makefile.objs
> > create mode 100644 storage-daemon/qapi/Makefile.objs
>
> [...]
>
> > diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
> > index 3e04e299ed..03d256f0a4 100644
> > --- a/qapi/Makefile.objs
> > +++ b/qapi/Makefile.objs
> > @@ -30,3 +30,8 @@ obj-y += $(QAPI_TARGET_MODULES:%=qapi-events-%.o)
> > obj-y += qapi-events.o
> > obj-y += $(QAPI_TARGET_MODULES:%=qapi-commands-%.o)
> > obj-y += qapi-commands.o
> > +
> > +QAPI_MODULES_STORAGE_DAEMON = block block-core char common crypto
> > introspect
> > +QAPI_MODULES_STORAGE_DAEMON += job monitor qom sockets pragma transaction
>
> I took a look into the rest, and I wonder whether query-iothreads from
> misc.json would be useful, too.
Possibly. It would be a separate patch, but I can add it.
The question is just where to move query-iothreads. Do we have a good
place, or do I need to separate misc.json and a new misc-sysemu.json?
> > diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
> > index 796c17c38a..c25634f673 100644
> > --- a/scripts/qapi/gen.py
> > +++ b/scripts/qapi/gen.py
> > @@ -44,6 +44,11 @@ class QAPIGen(object):
> > return ''
> >
> > def write(self, output_dir):
> > +# Include paths starting with ../ are used to reuse modules of the
> > main
> > +# schema in specialised schemas. Don't overwrite the files that are
> > +# already generated for the main schema.
> > +if self.fname.startswith('../'):
> > +return
>
> Sorry, but I’m about to ask a clueless question: How do we ensure that
> the main schema is generated before something else could make sure of
> the specialized schemas?
"Make sure"?
I think the order of the generation doesn't matter because generating
the storage daemon files doesn't actually access the main ones.
Generated C files shouldn't be a problem either because if we link an
object file into a binary, we have a make dependency for it.
Maybe the only a bit trickier question is whether we have the
dependencies right so that qemu-storage-daemon.c is only built after the
header files of both the main schema and the specific one have been
generated. If I understand the Makefile correctly, generated-files-y
takes care of this, and this patch adds all new header files to it if I
didn't miss any.
Kevin
signature.asc
Description: PGP signature
Re: privileged entropy sources in QEMU/KVM guests
* Laszlo Ersek (ler...@redhat.com) wrote:
> Hi,
>
> related TianoCore BZ:
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=1871
>
> (I'm starting this thread separately because at least some of the topics
> are specific to QEMU, and I didn't want to litter the BZ with a
> discussion that may not be interesting to all participants CC'd on the
> BZ. I am keeping people CC'd on this initial posting; please speak up if
> you'd like to be dropped from the email thread.)
>
> QEMU provides guests with the virtio-rng device, and the OVMF and
> ArmVirtQemu* edk2 platforms build EFI_RNG_PROTOCOL on top of that
> device. But, that doesn't seem enough for all edk2 use cases.
>
> Also, virtio-rng (hence EFI_RNG_PROTOCOL too) is optional, and its
> absence may affect some other use cases.
>
>
> (1) For UEFI HTTPS boot, TLS would likely benefit from good quality
> entropy. If the VM config includes virtio-rng (hence the guest firmware
> has EFI_RNG_PROTOCOL), then it should be used as a part of HTTPS boot.
>
> However, what if virtio-rng (hence EFI_RNG_PROTOCOL) are absent? Should
> UEFI HTTPS boot be disabled completely (or prevented / rejected
> somehow), blaming lack of good entropy? Or should TLS silently fall back
> to "mixing some counters [such as TSC] together and applying a
> deterministic cryptographic transformation"?
>
> IOW, knowing that the TLS setup may not be based on good quality
> entropy, should we allow related firmware services to "degrade silently"
> (not functionally, but potentially in security), or should we deny the
> services altogether?
I don't see a downside to insisting that if you want to use https then
you must provide an entropy source; they're easy enough to add using
virtio-rng if the CPU doesn't provide it.
>
> (2) It looks like the SMM driver implementing the privileged part of the
> UEFI variable runtime service could need access to good quality entropy,
> while running in SMM; in the future.
>
> This looks problematic on QEMU. Entropy is a valuable resource, and
> whatever resource SMM drivers depend on, should not be possible for e.g.
> a 3rd party UEFI driver (or even for the runtime OS) to exhaust.
> Therefore, it's not *only* the case that SMM drivers must not consume
> EFI_RNG_PROTOCOL (which exists at a less critical privilege level, i.e.
> outside of SMM/SMRAM), but also that SMM drivers must not depend on the
> same piece of *hardware* that feeds EFI_RNG_PROTOCOL.
>
> Furthermore, assuming we dedicate a hardware entropy device specifically
> to SMM drivers, such a device cannot be PCI(e). It would have to be a
> platform device at a fixed location (IO port or MMIO) that is only
> accessible to such guest code that executes in SMM. IOW, device access
> would have to be restricted similarly to pflash. (In fact the variable
> SMM driver will need, AIUI, the entropy for encrypting various variable
> contents, which are then written into pflash.)
Ewww. I guess a virtio-rng instance wired to virtio-mmio could do that.
It's a bit grim though.
Dave
> Alternatively, CPU instructions could exist that return entropy, and are
> executable only inside SMM. It seems that e.g. RDRAND can be trapped in
> guests ("A VMEXIT due to RDRAND will have exit reason 57 (decimal)").
> Then KVM / QEMU could provide any particular implementation we wanted --
> for example an exception could be injected unless RDRAND had been
> executed from within SMM. Unfortunately, such an arbitrary restriction
> (of RDRAND to SMM) would diverge from the Intel SDM, and would likely
> break other (non-SMM) guest code.
>
> Does a platform device that is dynamically detectable and usable in SMM
> only seem like an acceptable design for QEMU?
>
> Thanks,
> Laszlo
>
>
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
Re: privileged entropy sources in QEMU/KVM guests
Hi Laszlo,
Thanks for starting this thread.
On Thu, 7 Nov 2019 at 11:11, Laszlo Ersek wrote:
>
> Hi,
>
> related TianoCore BZ:
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=1871
>
> (I'm starting this thread separately because at least some of the topics
> are specific to QEMU, and I didn't want to litter the BZ with a
> discussion that may not be interesting to all participants CC'd on the
> BZ. I am keeping people CC'd on this initial posting; please speak up if
> you'd like to be dropped from the email thread.)
>
> QEMU provides guests with the virtio-rng device, and the OVMF and
> ArmVirtQemu* edk2 platforms build EFI_RNG_PROTOCOL on top of that
> device. But, that doesn't seem enough for all edk2 use cases.
>
> Also, virtio-rng (hence EFI_RNG_PROTOCOL too) is optional, and its
> absence may affect some other use cases.
>
>
> (1) For UEFI HTTPS boot, TLS would likely benefit from good quality
> entropy. If the VM config includes virtio-rng (hence the guest firmware
> has EFI_RNG_PROTOCOL), then it should be used as a part of HTTPS boot.
>
> However, what if virtio-rng (hence EFI_RNG_PROTOCOL) are absent? Should
> UEFI HTTPS boot be disabled completely (or prevented / rejected
> somehow), blaming lack of good entropy? Or should TLS silently fall back
> to "mixing some counters [such as TSC] together and applying a
> deterministic cryptographic transformation"?
>
> IOW, knowing that the TLS setup may not be based on good quality
> entropy, should we allow related firmware services to "degrade silently"
> (not functionally, but potentially in security), or should we deny the
> services altogether?
>
TLS uses a source of randomness to establish symmetric session keys
for encryption. So it really depends on the use case whether HTTPS is
used for authentication or for confidentiality, and it seems to me
that it would typically be the former. So disabling HTTPS boot in this
case seems counterproductive to me.
>
> (2) It looks like the SMM driver implementing the privileged part of the
> UEFI variable runtime service could need access to good quality entropy,
> while running in SMM; in the future.
>
> This looks problematic on QEMU. Entropy is a valuable resource, and
> whatever resource SMM drivers depend on, should not be possible for e.g.
> a 3rd party UEFI driver (or even for the runtime OS) to exhaust.
> Therefore, it's not *only* the case that SMM drivers must not consume
> EFI_RNG_PROTOCOL (which exists at a less critical privilege level, i.e.
> outside of SMM/SMRAM), but also that SMM drivers must not depend on the
> same piece of *hardware* that feeds EFI_RNG_PROTOCOL.
>
The typical model is to seed a DRBG [deterministic pseudorandom
sequence generator] using a sufficient amount of high quality entropy.
Once you have done that, it is rather hard to exhaust a DRBG - it is a
mathematical construction that is designed to last for a long time (<=
2^48 invocations [not bytes] according to the NIST spec), after which
it does not degrade although it may have generated so much output that
its internal state may be inferred if you have captured enough of it
(which is a rather theoretical issue IMHO)
The problem is that using the output of a DRBG as a seed is
non-trivial - the spec describes ways to do this, but wiring
virtio-rng to a DRBG in the host and using its output to seed a DRBG
in the guest is slighly problematic.
So it seems to me that the correct way to model this is to make the
host's true entropy source a shared resource like any other.
> Furthermore, assuming we dedicate a hardware entropy device specifically
> to SMM drivers, such a device cannot be PCI(e). It would have to be a
> platform device at a fixed location (IO port or MMIO) that is only
> accessible to such guest code that executes in SMM. IOW, device access
> would have to be restricted similarly to pflash. (In fact the variable
> SMM driver will need, AIUI, the entropy for encrypting various variable
> contents, which are then written into pflash.)
>
> Alternatively, CPU instructions could exist that return entropy, and are
> executable only inside SMM. It seems that e.g. RDRAND can be trapped in
> guests ("A VMEXIT due to RDRAND will have exit reason 57 (decimal)").
> Then KVM / QEMU could provide any particular implementation we wanted --
> for example an exception could be injected unless RDRAND had been
> executed from within SMM. Unfortunately, such an arbitrary restriction
> (of RDRAND to SMM) would diverge from the Intel SDM, and would likely
> break other (non-SMM) guest code.
>
> Does a platform device that is dynamically detectable and usable in SMM
> only seem like an acceptable design for QEMU?
>
Re: [RFC PATCH 00/18] Add qemu-storage-daemon
On Thu, Oct 17, 2019 at 03:01:46PM +0200, Kevin Wolf wrote: > This series adds a new tool 'qemu-storage-daemon', which can be used to > export and perform operations on block devices. There is some overlap > between qemu-img/qemu-nbd and the new qemu-storage-daemon, but there are > a few important differences: > > * The qemu-storage-daemon has QMP support. The command set is obviously > restricted compared to the system emulator because there is no guest, > but all of the block operations are present. > > This means that it can access advanced options or operations that the > qemu-img command line doesn't expose. For example, blockdev-create is > a lot more powerful than 'qemu-img create', and qemu-storage-daemon > allows to execute it without starting a guest. > > Compared to qemu-nbd it means that, for example, block jobs can now be > executed on the server side, and backing chains shared by multiple VMs > can be modified this way. > > * The existing tools all have a separately invented one-off syntax for > the job at hand, which usually comes with restrictions compared to the > system emulator. qemu-storage-daemon shares the same syntax with the > system emulator for most options and prefers QAPI based interfaces > where possible (such as --blockdev), so it should be easy to make use > of in libvirt. > > * While this series implements only NBD exports, the storage daemon is > intended to serve multiple protocols and its syntax reflects this. In > the past, we had proposals to add new one-off tools for exporting over > new protocols like FUSE or TCMU. > > With a generic storage daemon, additional export methods have a home > without adding a new tool for each of them. > > I'm posting this as an RFC mainly for two reasons: > > 1. The monitor integration, which could be argued to be a little hackish >(some generated QAPI source files are built from a separate QAPI >schema, but the per-module ones are taken from the system emulator) >and Markus will want to have a closer look there. But from the IRC >discussions we had, we seem to agree on the general approach here. > > 2. I'm not completely sure if the command line syntax is the final >version that we want to support long-term. Many options directly use >QAPI visitors (--blockdev, --export, --nbd-server) and should be >fine. However, others use QemuOpts (--chardev, --monitor, --object). > >This is the same as in the system emulator, so we wouldn't be adding >a new problem, but as there was talk about QAPIfying the command >line, and I wouldn't want later syntax changes or adding lots of >compatibility code to a new tool, I thought we should probably >discuss whether QAPIfying from the start would be an option. I think that following what the QEMU emulators currently do for CLI args should be an explicit anti-goal, because we know that it is a long standing source of pain. Fixing it in the emulator binaries is hard due to backward compatibility, but for this new binary we have a clean slate. This feels like a good opportunity to implement & demonstrate what we think QEMU configuration ought to look like. Work done for this in the qemu-storage-daemon may well help us understand how we'll be able to move the QEMU emulators into a new scheme later. My personal wish would be to have no use of QemuOpts at all. Use GOptionContext *only* for parsing command line arguments related to execution of the daemon - ie things like --help, --version, --daemon, --pid-file. The use a "--config /path/to/json/file" arg to point to the config file for everything else using QAPI schema to describe it fully. When loading the config file, things should be created in order in which they are specified. ie don't try and group things, otherwise we end up back with the horrific hacks for objects where some are created early & some late. For an ambitious stretch goal, I think we should seriously consider whether our use of chardevs is appropriate in all cases that exist, and whether we can avoid the trap of over-using chardev in the new storage daemon since it is a clean slate in terms of user facing CLI config. chardevs are designed for & reasonably well suited to attaching to devices like serial ports, parallel ports, etc. You have a 1:1 remote:local peer relationship. The transport is a dumb byte stream, nothing special needed on top & the user can cope with any type of chardev. Many cases where we've used chardevs as a backend in QEMU are a poor fit. We just used chardevs as an "easy" way to configure a UNIX or TCP socket from the CLI, and don't care about, nor work with, any othuer chardev backends. As a result of this misuse we've had to put in an increasing number of hacks in the chardev code to deal with fact that callers want to know about & use socket semantics. eg FD passing, the chardev reconnection polling code. The monitor is a prime example of a bad fit - it
Re: Looking for issues/features for my first contribution
On Thursday, November 7, 2019, Rajath Shashidhara wrote: > Hi all, > > I am a Computer Science graduate student at The University of Texas at > Austin (UT, Austin). I am looking forward to contributing to qemu ! > > This semester, I am taking a class in Virtualization ( > https://github.com/vijay03/cs378-f19) and contributing to a > virtualization related open-source project is a significant part of the > course. > I would be interested in contributing a patchset to qemu - possibly a > self-contained feature or a reasonably complex bug fix that can be > completed in under a month's time. I did look at both the bugtracker and > the QEMU Google Summer of Code 2019 page [https://wiki.qemu.org/Google_ > Summer_of_Code_2019] for ideas. However, I would be interested in hearing > from the community and I would be delighted if somebody can be suggest a > suitable project ! > > Hello, Rajath! Thank you for expressing interest in QEMU open source project. There is certainly a place for you and your contributions in QEMU, and you are very welcomed! It looks to me the following project would fit your description: 'Implement emulation of DS3231 real time clock in QEMU' Datasheet: https://datasheets.maximintegrated.com/en/ds/DS3231.pdf The steps needed to complete it (in my opinion): - collect datasheets of as many as possible RTC chips already emulated in QEMU (there are around of dozen of them, see folder hw/rtc) - do a comparative analysis of selected RTC implementations in QEMU - get to know general QEMU device model - design and implement DS3231 emulation I can give you (unfortunately constrained by tight time limits) some help and guidance. But there are other people in community too (more knowledgable in the area than me). I salute your initiative! Yours, Aleksandar > I am an advanced C programmer with both professional and academic > background in systems design & implementation - especially OS & Networks. > Given my background, I feel fairly confident that I can pickup the QEMU > codebase quickly. > > Eagerly looking forward to hearing from the community ! > > Thanks, > Rajath Shashidhara > > >
Re: [PATCH v3 11/16] hw/arm/raspi: Use -smp cores= option to restrict enabled cores
On 10/20/19 1:47 AM, Philippe Mathieu-Daudé wrote: > The abstract TYPE_BCM283X device provides a 'enabled-cpus' property > to restrict the number of cores powered on reset. This because on > real hardware the GPU is responsible of starting the cores and keep > them spinning until the Linux kernel is ready to use them. > When using the -kernel paramenter, QEMU does this by installing the > 'raspi_smpboot' code when arm_boot_info::write_board_setup() is > called. This is a special feature to help the Linux kernel, and can > only be used with a Linux kernel. > > Even if loaded with the -kernel option, U-boot is not Linux, thus > is not recognized as it and the raspi_smpboot code is not installed. > > Upon introduction of this machine in commit 1df7d1f9303, the -smp > option allowd to limit the number of cores powered on reset. > Unfortunately later commit 72649619341 added a check which made this > feature unusable: > >$ qemu-system-aarch64 -M raspi3 -smp 1 >qemu-system-aarch64: Invalid SMP CPUs 1. The min CPUs supported by machine > 'raspi3' is 4 > > Fortunately, the -smp option allow various kind of CPU topology: > >-smp > [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets] > set the number of CPUs to 'n' [default=1] > maxcpus= maximum number of total cpus, including > offline CPUs for hotplug, etc > cores= number of CPU cores on one socket (for PC, it's on one die) > threads= number of threads on one CPU core > dies= number of CPU dies on one socket (for PC only) > sockets= number of discrete sockets in the system > > Let's use the 'cores' argument to specify the number of cores powered > at reset to restore this feature, and allow to boot U-boot. > > We can now run U-boot using: > >$ qemu-system-aarch64 -M raspi3 -smp 4,cores=1 ... > > Reported-by: Laurent Bonnans > Signed-off-by: Philippe Mathieu-Daudé > --- > hw/arm/raspi.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c > index 569d85c11a..45d3f91f95 100644 > --- a/hw/arm/raspi.c > +++ b/hw/arm/raspi.c > @@ -190,8 +190,8 @@ static void raspi_init(MachineState *machine, int version) > /* Setup the SOC */ > object_property_add_const_link(OBJECT(>soc), "ram", OBJECT(>ram), > _abort); > -object_property_set_int(OBJECT(>soc), machine->smp.cpus, > "enabled-cpus", > -_abort); > +object_property_set_int(OBJECT(>soc), machine->smp.cores, > +"enabled-cpus", _abort); > int board_rev = version == 3 ? 0xa02082 : 0xa21041; > object_property_set_int(OBJECT(>soc), board_rev, "board-rev", > _abort); Hi Phil, Thanks for the patch, I finally got the chance to make some tests (sorry about the delay). Using the proposed -smp options indeed helps to run u-boot elfs directly. However, the cores fail to start when switching to linux (tested on raspi2): [ 0.071030] smp: Bringing up secondary CPUs ... [ 1.157876] CPU1: failed to come online [ 2.219899] CPU2: failed to come online [ 3.285412] CPU3: failed to come online [ 3.286137] smp: Brought up 1 node, 1 CPU [ 3.286766] SMP: Total of 1 processors activated (125.00 BogoMIPS). [ 3.287442] CPU: All CPU(s) started in SVC mode. The behavior persist even without using the option on the command line. The normal behavior is restored if I use "-smp 4, cores=4" Greetings, Laurent
Re: [RFC PATCH 18/18] qemu-storage-daemon: Add --monitor option
On 07.11.19 11:12, Kevin Wolf wrote:
> Am 06.11.2019 um 15:32 hat Max Reitz geschrieben:
>> On 17.10.19 15:02, Kevin Wolf wrote:
>>> This adds and parses the --monitor option, so that a QMP monitor can be
>>> used in the storage daemon. The monitor offers commands defined in the
>>> QAPI schema at storage-daemon/qapi/qapi-schema.json.
>>>
>>> Signed-off-by: Kevin Wolf
>>> ---
>>> storage-daemon/qapi/qapi-schema.json | 15
>>> qemu-storage-daemon.c| 34
>>> Makefile | 30
>>> Makefile.objs| 4 ++--
>>> monitor/Makefile.objs| 2 ++
>>> qapi/Makefile.objs | 5
>>> qom/Makefile.objs| 1 +
>>> scripts/qapi/gen.py | 5
>>> storage-daemon/Makefile.objs | 1 +
>>> storage-daemon/qapi/Makefile.objs| 1 +
>>> 10 files changed, 96 insertions(+), 2 deletions(-)
>>> create mode 100644 storage-daemon/qapi/qapi-schema.json
>>> create mode 100644 storage-daemon/Makefile.objs
>>> create mode 100644 storage-daemon/qapi/Makefile.objs
>>
>> [...]
>>
>>> diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
>>> index 3e04e299ed..03d256f0a4 100644
>>> --- a/qapi/Makefile.objs
>>> +++ b/qapi/Makefile.objs
>>> @@ -30,3 +30,8 @@ obj-y += $(QAPI_TARGET_MODULES:%=qapi-events-%.o)
>>> obj-y += qapi-events.o
>>> obj-y += $(QAPI_TARGET_MODULES:%=qapi-commands-%.o)
>>> obj-y += qapi-commands.o
>>> +
>>> +QAPI_MODULES_STORAGE_DAEMON = block block-core char common crypto
>>> introspect
>>> +QAPI_MODULES_STORAGE_DAEMON += job monitor qom sockets pragma transaction
>>
>> I took a look into the rest, and I wonder whether query-iothreads from
>> misc.json would be useful, too.
>
> Possibly. It would be a separate patch, but I can add it.
>
> The question is just where to move query-iothreads. Do we have a good
> place, or do I need to separate misc.json and a new misc-sysemu.json?
I’d just put it in block.json because of the “io”... O:-)
>>> diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
>>> index 796c17c38a..c25634f673 100644
>>> --- a/scripts/qapi/gen.py
>>> +++ b/scripts/qapi/gen.py
>>> @@ -44,6 +44,11 @@ class QAPIGen(object):
>>> return ''
>>>
>>> def write(self, output_dir):
>>> +# Include paths starting with ../ are used to reuse modules of the
>>> main
>>> +# schema in specialised schemas. Don't overwrite the files that are
>>> +# already generated for the main schema.
>>> +if self.fname.startswith('../'):
>>> +return
>>
>> Sorry, but I’m about to ask a clueless question: How do we ensure that
>> the main schema is generated before something else could make sure of
>> the specialized schemas?
>
> "Make sure"?
Oops, s/ sure/ use/.
> I think the order of the generation doesn't matter because generating
> the storage daemon files doesn't actually access the main ones.
> Generated C files shouldn't be a problem either because if we link an
> object file into a binary, we have a make dependency for it.
I was mostly wondering about the fact that make mustn’t try to compile
the “generated files” (which aren’t really generated here) before they
are actually generated when the main schema is processed.
Max
> Maybe the only a bit trickier question is whether we have the
> dependencies right so that qemu-storage-daemon.c is only built after the
> header files of both the main schema and the specific one have been
> generated. If I understand the Makefile correctly, generated-files-y
> takes care of this, and this patch adds all new header files to it if I
> didn't miss any.
>
> Kevin
>
signature.asc
Description: OpenPGP digital signature
[Bug 1851547] Re: qemu 4 crashes with this parameter attached -usb -device usb-host, hostbus=1, hostaddr=7 \
Hi Marietto, Can you attach an lsusb output from your host? I'm curious what host bug 1, addr 7 and 8 are. Dave -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1851547 Title: qemu 4 crashes with this parameter attached -usb -device usb- host,hostbus=1,hostaddr=7 \ Status in QEMU: New Bug description: Hello. qemu / kvm does not start anymore after upgrading ubuntu from 19.04 to 19.10 and qemu from 3 to 4,as you can see below. what can I do ? root@ziomario-Z390-AORUS-PRO:/home/ziomario/Scrivania/OS-KVM# ./boot- OS-HSP2.sh > qemu-system-x86_64: /build/qemu- UryNDZ/qemu-4.0+dfsg/hw/usb/core.c:720: usb_ep_get: asserzione "dev != NULL" non riuscita. ./boot-OS-HSP2.sh: riga 40: 26312 Annullato (core dump creato) qemu- system-x86_64 -enable-kvm -m 16000 -cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid- freq=on,$MY_OPTIONS -machine pc-q35-2.9 -smp 4,cores=2 -vga none -device vfio-pci,host=01:00.0,bus=pcie.0,multifunction=on -device vfio-pci,host=01:00.1,bus=pcie.0 -device vfio- pci,host=01:00.2,bus=pcie.0 -device vfio-pci,host=01:00.3,bus=pcie.0 -usb -device usb-host,hostbus=1,hostaddr=7 -drive if=pflash,format=raw,readonly,file=$OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,file=$OVMF/OVMF_VARS-1024x768.fd -smbios type=2 -device ich9-ahci,id=sata -drive id=Clover,if=none,snapshot=on,format=qcow2,file=./'Mo/CloverNG.qcow2' -device ide-hd,bus=sata.2,drive=Clover -device ide- hd,bus=sata.3,drive=InstallMedia -drive id=InstallMedia,if=none,file=BaseSystemHS.img,format=raw -drive id=BsdHDD,if=none,file=/dev/sdg,format=raw -device ide- hd,bus=sata.4,drive=BsdHDD -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -device e1000-82545em,netdev=net0,id=net0,mac=52:54:00:c9:18:27 -monitor stdio It seems that this line is not good anymore (it worked with qemu 3.x) : -usb -device usb-host,hostbus=1,hostaddr=7 \ when I removed it,it works. But I need that. With what can I change it ? You can reproduce that upgrading ubuntu 19.04 to 19.10 because in that way also qemu will be upgraded from 3 to 4. These are the packages that I'm using : root@ziomario-Z390-AORUS-PRO:/home/ziomario# qemu-system-x86_64 --version QEMU emulator version 4.0.0 (Debian 1:4.0+dfsg-0ubuntu9) To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1851547/+subscriptions
Re: [PATCH v2 05/21] iotests: Replace IMGOPTS by _unsupported_imgopts
On Thu, 2019-11-07 at 10:08 +0100, Max Reitz wrote: > On 06.11.19 16:45, Maxim Levitsky wrote: > > On Tue, 2019-10-15 at 16:27 +0200, Max Reitz wrote: > > > Some tests require compat=1.1 and thus set IMGOPTS='compat=1.1' > > > globally. That is not how it should be done; instead, they should > > > simply set _unsupported_imgopts to compat=0.10 (compat=1.1 is the > > > default anyway). > > > > > > This makes the tests heed user-specified $IMGOPTS. Some do not work > > > with all image options, though, so we need to disable them accordingly. > > > > > > Signed-off-by: Max Reitz > > > --- > > > tests/qemu-iotests/036 | 3 +-- > > > tests/qemu-iotests/060 | 4 ++-- > > > tests/qemu-iotests/062 | 3 ++- > > > tests/qemu-iotests/066 | 3 ++- > > > tests/qemu-iotests/068 | 3 ++- > > > tests/qemu-iotests/098 | 4 ++-- > > > 6 files changed, 11 insertions(+), 9 deletions(-) > > > > > > diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036 > > > index 5f929ad3be..bbaf0ef45b 100755 > > > --- a/tests/qemu-iotests/036 > > > +++ b/tests/qemu-iotests/036 > > > @@ -43,9 +43,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 > > > # This tests qcow2-specific low-level functionality > > > _supported_fmt qcow2 > > > _supported_proto file > > > - > > > # Only qcow2v3 and later supports feature bits > > > -IMGOPTS="compat=1.1" > > > +_unsupported_imgopts 'compat=0.10' > > > > > > echo > > > echo === Image with unknown incompatible feature bit === > > > diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 > > > index b91d8321bb..9c2ef42522 100755 > > > --- a/tests/qemu-iotests/060 > > > +++ b/tests/qemu-iotests/060 > > > @@ -48,6 +48,8 @@ _filter_io_error() > > > _supported_fmt qcow2 > > > _supported_proto file > > > _supported_os Linux > > > +# These tests only work for compat=1.1 images with refcount_bits=16 > > > +_unsupported_imgopts 'compat=0.10' 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' > > > > Looks like the reason for that is that the test hardcodes (or guesses that > > is) various qcow2 structures > > thing I have seen few times already in the iotests. > > Not now but sometime in the future it would be nice to extend qcow2.py (or > > something > > like that) to dump location of all qcow2 structures so that the guesswork > > could be eliminated. > > With the peek_file* functions we have now it’s actually simple to dump > that location ($(peek_file_be "$TEST_IMG" 48 8) for the refcount table > offset, for example). > > But it wouldn’t help, because compat=0.10 or refcount_bits != 16 won’t > change those locations. So the locations aren’t the reason why we need > to forbid those options here. > > The reason we need refcount_bits=16 is that we’re going to directly > manipulate a refcount block. To do so, we need to know the refcount > width, and I don’t think it’s worth trying to implement something generic. > > We need compat=1.1 because compat=0.10 doesn’t have feature bits, so > there’s no “corrupt” bit there. > > Max > This makes sense! Sorry for the noise! Best regards, Maxim Levitsky
