Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Am 26.03.2011 22:56, schrieb Dushyant Bansal: On the other hand, I think the starting point for a generic in-place converter would be a loop that does something like bdrv_is_allocated() but translates the guest position in the block device into an offset into the image file. That, together with some sort of free map or space allocation bitmap would allow a generic approach to figuring out the data mapping and which parts of the file can be safely used. We can discuss the detailed API later, but I agree that the critical thing to convert is the mapping. You would probably open the file with the source format driver read-only and with the destination driver read-write. For qcow2 you would start with writing a refcount table that marks the whole file as used, other formats use the file size anyway. Then you can start creating L1 and L2 tables and copy the mapping over. Once this is done, you do an fsck to free the metadata of the old format. One thing that may become tricky is the image header which both drivers may want to use and which is fixed at offset 0. And of course, you must make sure that the image is safe at any point if the converter crashes. For image header issue, this is the approach that comes to mind. Lets say, destination format is qcow2. BDRVQcowState is responsible for header fields inside BlockDriverState. We need qcow2 image header to initiliaze all the fields of BDRVQcowState, which is done by bdrv_open(qcow2_open()). So initially, for the qcow2 driver, we do not copy the qcow2 image header (we keep the source header). We can then manually set fields of BDRVQcowState with the desired header fields. And after all other metadata has been copied for the qcow2 format, we can replace the source image header with the qcow2 header. The question is if you can make sure that while you convert the image, qcow2 will never try to modify the header. It will definitely do so if you have to increase the L1 or refcount table, for example. In the end, we may have to use tricks like using a protocol which keeps the first cluster in a ramdisk until the conversion is complete. Kevin
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On the other hand, I think the starting point for a generic in-place converter would be a loop that does something like bdrv_is_allocated() but translates the guest position in the block device into an offset into the image file. That, together with some sort of free map or space allocation bitmap would allow a generic approach to figuring out the data mapping and which parts of the file can be safely used. We can discuss the detailed API later, but I agree that the critical thing to convert is the mapping. You would probably open the file with the source format driver read-only and with the destination driver read-write. For qcow2 you would start with writing a refcount table that marks the whole file as used, other formats use the file size anyway. Then you can start creating L1 and L2 tables and copy the mapping over. Once this is done, you do an fsck to free the metadata of the old format. One thing that may become tricky is the image header which both drivers may want to use and which is fixed at offset 0. And of course, you must make sure that the image is safe at any point if the converter crashes. For image header issue, this is the approach that comes to mind. Lets say, destination format is qcow2. BDRVQcowState is responsible for header fields inside BlockDriverState. We need qcow2 image header to initiliaze all the fields of BDRVQcowState, which is done by bdrv_open(qcow2_open()). So initially, for the qcow2 driver, we do not copy the qcow2 image header (we keep the source header). We can then manually set fields of BDRVQcowState with the desired header fields. And after all other metadata has been copied for the qcow2 format, we can replace the source image header with the qcow2 header. Thanks, Dushyant
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Am 16.03.2011 18:47, schrieb Stefan Hajnoczi: On Tue, Mar 15, 2011 at 10:27 AM, Kevin Wolf kw...@redhat.com wrote: Am 14.03.2011 16:13, schrieb Dushyant Bansal: Nice that qemu-img convert isn't that far out by default on raw :). About Google Summer of Code, I have posted my take on applying and want to share that with you and qemu-devel: http://blog.vmsplice.net/2011/03/advice-for-students-applying-to-google.html Thanks for sharing your experiences. After reading about qcow2 and qed and how they organize data (thanks to the newly added qcow2 doc and discussions on the mailing list), this is what I understand. So, the main difference between qed and qcow2 is the absence of reference count structure in qed(means less meta data). It improves performance due to: 1. For write operations, less or no metadata to update. 2. Data write and metadata write can be in any order This also means these features are no longer supported: 1. Internal snapshots, 2. CPU/device state snapshots, 3. Compression, 4. Encryption Now, coming to qed--qcow2 conversion, I want to clarify some things. 1. header_size: variable in qed, equals to cluster size in qcow2: When will it be larger than 1 cluster in qed? So, what will happen to that extra data on qed-qcow2 conversion. If you have an feature that is used in the original image, but cannot be represented in the new format, I think you should just get an error. 2. L2 table size: equals to L1 table size in qed, equals to cluster size in qcow2: we need to take it into account during conversion. Right. I think we'll have to rewrite all of the metadata. I wonder if we can manage to have a nice block driver interface for in-place image conversions so that we don't only get a qed-qcow2 converter, but also can implement the interface in e.g. VMDK and get VMDK-qcow2 and VMDK-qed as well. I think this will be tricky but would be very interested if someone has ideas. Code-wise an in-place converter probably needs access to both format's on-disk structures or internal functions. I don't think abstracting this is easy because the more you abstract the less control you have over keeping things in-place and cleanly putting the new structures in place. Well, if it was easy, I would have suggested a specific way of doing it. ;-) But it would be a really cool thing to have, and I think it's more fun for a GSoC participant to actually think about a hard problem than just doing mostly mechanical work. On the other hand, I think the starting point for a generic in-place converter would be a loop that does something like bdrv_is_allocated() but translates the guest position in the block device into an offset into the image file. That, together with some sort of free map or space allocation bitmap would allow a generic approach to figuring out the data mapping and which parts of the file can be safely used. We can discuss the detailed API later, but I agree that the critical thing to convert is the mapping. You would probably open the file with the source format driver read-only and with the destination driver read-write. For qcow2 you would start with writing a refcount table that marks the whole file as used, other formats use the file size anyway. Then you can start creating L1 and L2 tables and copy the mapping over. Once this is done, you do an fsck to free the metadata of the old format. One thing that may become tricky is the image header which both drivers may want to use and which is fixed at offset 0. And of course, you must make sure that the image is safe at any point if the converter crashes. The big benefit doing an interface for in-place conversion is that we can support n-to-n conversions with at most n converter code rather than having to code n * n - n different in-place converters. Yes, this was the idea. Kevin
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
1. header_size: variable in qed, equals to cluster size in qcow2:
When will it be larger than 1 cluster in qed? So, what will happen to
that extra data on qed-qcow2 conversion.
If you have an feature that is used in the original image, but cannot be
represented in the new format, I think you should just get an error.
2. L2 table size: equals to L1 table size in qed, equals to cluster size
in qcow2:
we need to take it into account during conversion.
Right. I think we'll have to rewrite all of the metadata.
I wonder if we can manage to have a nice block driver interface for
in-place image conversions so that we don't only get a qed-qcow2
converter, but also can implement the interface in e.g. VMDK and get
VMDK-qcow2 and VMDK-qed as well.
3. refcount table:
qcow2-qed:we do not keep refcount structure
qed-qcow2: initialize refcount structure
Yes, refcounts can be rebuilt after the mapping has been converted.
So, a qcow2-qed-qcow2 conversion means if earlier, qcow2 image was
using additional features{1-4}, all information related to that will be
lost.
We shouldn't lose anything but just abort if a conversion isn't
possible. The user can still use qemu-img convert for the more
complicated cases, for example for removing encryption or compression.
Thanks for clearing up those issues.
--
Dushyant
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Tue, Mar 15, 2011 at 10:27 AM, Kevin Wolf kw...@redhat.com wrote:
Am 14.03.2011 16:13, schrieb Dushyant Bansal:
Nice that qemu-img convert isn't that far out by default on raw :).
About Google Summer of Code, I have posted my take on applying and
want to share that with you and qemu-devel:
http://blog.vmsplice.net/2011/03/advice-for-students-applying-to-google.html
Thanks for sharing your experiences.
After reading about qcow2 and qed and how they organize data (thanks to
the newly added qcow2 doc and discussions on the mailing list), this is
what I understand.
So, the main difference between qed and qcow2 is the absence of
reference count structure in qed(means less meta data).
It improves performance due to:
1. For write operations, less or no metadata to update.
2. Data write and metadata write can be in any order
This also means these features are no longer supported:
1. Internal snapshots,
2. CPU/device state snapshots,
3. Compression,
4. Encryption
Now, coming to qed--qcow2 conversion, I want to clarify some things.
1. header_size: variable in qed, equals to cluster size in qcow2:
When will it be larger than 1 cluster in qed? So, what will happen to
that extra data on qed-qcow2 conversion.
If you have an feature that is used in the original image, but cannot be
represented in the new format, I think you should just get an error.
2. L2 table size: equals to L1 table size in qed, equals to cluster size
in qcow2:
we need to take it into account during conversion.
Right. I think we'll have to rewrite all of the metadata.
I wonder if we can manage to have a nice block driver interface for
in-place image conversions so that we don't only get a qed-qcow2
converter, but also can implement the interface in e.g. VMDK and get
VMDK-qcow2 and VMDK-qed as well.
I think this will be tricky but would be very interested if someone
has ideas. Code-wise an in-place converter probably needs access to
both format's on-disk structures or internal functions. I don't think
abstracting this is easy because the more you abstract the less
control you have over keeping things in-place and cleanly putting the
new structures in place.
On the other hand, I think the starting point for a generic in-place
converter would be a loop that does something like bdrv_is_allocated()
but translates the guest position in the block device into an offset
into the image file. That, together with some sort of free map or
space allocation bitmap would allow a generic approach to figuring out
the data mapping and which parts of the file can be safely used.
The big benefit doing an interface for in-place conversion is that we
can support n-to-n conversions with at most n converter code rather
than having to code n * n - n different in-place converters.
3. refcount table:
qcow2-qed:we do not keep refcount structure
qed-qcow2: initialize refcount structure
Yes, refcounts can be rebuilt after the mapping has been converted.
So, a qcow2-qed-qcow2 conversion means if earlier, qcow2 image was
using additional features{1-4}, all information related to that will be
lost.
We shouldn't lose anything but just abort if a conversion isn't
possible. The user can still use qemu-img convert for the more
complicated cases, for example for removing encryption or compression.
I agree with Kevin on the points made.
Stefan
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Am 14.03.2011 16:13, schrieb Dushyant Bansal:
Nice that qemu-img convert isn't that far out by default on raw :).
About Google Summer of Code, I have posted my take on applying and
want to share that with you and qemu-devel:
http://blog.vmsplice.net/2011/03/advice-for-students-applying-to-google.html
Thanks for sharing your experiences.
After reading about qcow2 and qed and how they organize data (thanks to
the newly added qcow2 doc and discussions on the mailing list), this is
what I understand.
So, the main difference between qed and qcow2 is the absence of
reference count structure in qed(means less meta data).
It improves performance due to:
1. For write operations, less or no metadata to update.
2. Data write and metadata write can be in any order
This also means these features are no longer supported:
1. Internal snapshots,
2. CPU/device state snapshots,
3. Compression,
4. Encryption
Now, coming to qed--qcow2 conversion, I want to clarify some things.
1. header_size: variable in qed, equals to cluster size in qcow2:
When will it be larger than 1 cluster in qed? So, what will happen to
that extra data on qed-qcow2 conversion.
If you have an feature that is used in the original image, but cannot be
represented in the new format, I think you should just get an error.
2. L2 table size: equals to L1 table size in qed, equals to cluster size
in qcow2:
we need to take it into account during conversion.
Right. I think we'll have to rewrite all of the metadata.
I wonder if we can manage to have a nice block driver interface for
in-place image conversions so that we don't only get a qed-qcow2
converter, but also can implement the interface in e.g. VMDK and get
VMDK-qcow2 and VMDK-qed as well.
3. refcount table:
qcow2-qed:we do not keep refcount structure
qed-qcow2: initialize refcount structure
Yes, refcounts can be rebuilt after the mapping has been converted.
So, a qcow2-qed-qcow2 conversion means if earlier, qcow2 image was
using additional features{1-4}, all information related to that will be
lost.
We shouldn't lose anything but just abort if a conversion isn't
possible. The user can still use qemu-img convert for the more
complicated cases, for example for removing encryption or compression.
Kevin
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Nice that qemu-img convert isn't that far out by default on raw :).
About Google Summer of Code, I have posted my take on applying and
want to share that with you and qemu-devel:
http://blog.vmsplice.net/2011/03/advice-for-students-applying-to-google.html
Thanks for sharing your experiences.
After reading about qcow2 and qed and how they organize data (thanks to
the newly added qcow2 doc and discussions on the mailing list), this is
what I understand.
So, the main difference between qed and qcow2 is the absence of
reference count structure in qed(means less meta data).
It improves performance due to:
1. For write operations, less or no metadata to update.
2. Data write and metadata write can be in any order
This also means these features are no longer supported:
1. Internal snapshots,
2. CPU/device state snapshots,
3. Compression,
4. Encryption
Now, coming to qed--qcow2 conversion, I want to clarify some things.
1. header_size: variable in qed, equals to cluster size in qcow2:
When will it be larger than 1 cluster in qed? So, what will happen to
that extra data on qed-qcow2 conversion.
2. L2 table size: equals to L1 table size in qed, equals to cluster size
in qcow2:
we need to take it into account during conversion.
3. refcount table:
qcow2-qed:we do not keep refcount structure
qed-qcow2: initialize refcount structure
So, a qcow2-qed-qcow2 conversion means if earlier, qcow2 image was
using additional features{1-4}, all information related to that will be
lost.
What do you think? Please correct me if I am making some incorrect
statement.
Thanks,
Dushyant
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Mon, Feb 28, 2011 at 8:41 PM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: On Sunday 27 February 2011 04:19 PM, Stefan Hajnoczi wrote: On Sat, Feb 26, 2011 at 9:50 PM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: virtual size: 10G (10737418240 bytes) disk size: 569M convert- original time 0m52.522s convert- modified (resultant disk size: 5.3G) time 2m12.744s cp time 0m51.724s (same disk size) --- virtual size: 10G (10737418240 bytes) disk size: 3.6G convert- original time 1m52.249s convert- modified (resultant disk size: 7.1G) time 3m2.891s cp time 1m55.320s (same disk size) --- In these results, we can see that resultant disk size has increased. If I'm reading this correctly then qemu-img convert is within a few seconds of cp(1) for you? I have collected and attached some more time results for different operations on a 2.2G image. qemu-img convert is close to cp. qemu(modified): IO_BUF_SIZE = (2 * 1024 ) if any sector is non-null, write all sectors Nice that qemu-img convert isn't that far out by default on raw :). About Google Summer of Code, I have posted my take on applying and want to share that with you and qemu-devel: http://blog.vmsplice.net/2011/03/advice-for-students-applying-to-google.html Stefan
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Sunday 27 February 2011 04:19 PM, Stefan Hajnoczi wrote: On Sat, Feb 26, 2011 at 9:50 PM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: Disk block size is usually 512 bytes and in qemu-img, sector size is also 512B. And, this change would copy n sectors even if only one of them actually contains data (while cp checks and copies one block(=sector) at a time). Therefore, it will end up writing more data than cp. cp(1) from GNU coreutils 8.5 works in units of 32 KB on my system. It reads 32 KB and checks for all zeroes. If there are all zeroes, it seeks ahead 32 KB in the output file. Otherwise it writes the entire 32 KB. You can check what cp(1) is doing using strace(1). yes, you are right. I was reading from a much older coreutils source. virtual size: 10G (10737418240 bytes) disk size: 569M convert- original time0m52.522s convert- modified (resultant disk size: 5.3G) time2m12.744s cp time0m51.724s (same disk size) --- virtual size: 10G (10737418240 bytes) disk size: 3.6G convert- original time1m52.249s convert- modified (resultant disk size: 7.1G) time3m2.891s cp time1m55.320s (same disk size) --- In these results, we can see that resultant disk size has increased. If I'm reading this correctly then qemu-img convert is within a few seconds of cp(1) for you? I have collected and attached some more time results for different operations on a 2.2G image. qemu-img convert is close to cp. qemu(modified): IO_BUF_SIZE = (2 * 1024 ) if any sector is non-null, write all sectors -- Dushyant results.xls Description: MS-Excel spreadsheet
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Sat, Feb 26, 2011 at 9:50 PM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: Disk block size is usually 512 bytes and in qemu-img, sector size is also 512B. And, this change would copy n sectors even if only one of them actually contains data (while cp checks and copies one block(=sector) at a time). Therefore, it will end up writing more data than cp. cp(1) from GNU coreutils 8.5 works in units of 32 KB on my system. It reads 32 KB and checks for all zeroes. If there are all zeroes, it seeks ahead 32 KB in the output file. Otherwise it writes the entire 32 KB. You can check what cp(1) is doing using strace(1). virtual size: 10G (10737418240 bytes) disk size: 569M convert- original time 0m52.522s convert- modified (resultant disk size: 5.3G) time 2m12.744s cp time 0m51.724s (same disk size) --- virtual size: 10G (10737418240 bytes) disk size: 3.6G convert- original time 1m52.249s convert- modified (resultant disk size: 7.1G) time 3m2.891s cp time 1m55.320s (same disk size) --- In these results, we can see that resultant disk size has increased. If I'm reading this correctly then qemu-img convert is within a few seconds of cp(1) for you? Stefan
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Stefan Hajnoczi stefa...@gmail.com writes: On Sat, Feb 26, 2011 at 9:50 PM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: Disk block size is usually 512 bytes and in qemu-img, sector size is also 512B. And, this change would copy n sectors even if only one of them actually contains data (while cp checks and copies one block(=sector) at a time). Therefore, it will end up writing more data than cp. cp(1) from GNU coreutils 8.5 works in units of 32 KB on my system. It reads 32 KB and checks for all zeroes. If there are all zeroes, it seeks ahead 32 KB in the output file. Otherwise it writes the entire 32 KB. The latest version[*] of cp does better: cp now copies sparse files efficiently on file systems with FIEMAP support (ext4, btrfs, xfs, ocfs2). Before, it had to read 2^20 bytes when copying a 1MiB sparse file. Now, it copies bytes only for the non-sparse sections of a file. Similarly, to induce a hole in the output file, it had to detect a long sequence of zero bytes. Now, it knows precisely where each hole in an input file is, and can reproduce them efficiently in the output file. mv also benefits when it resorts to copying, e.g., between file systems. But beware of kernel bugs[**]. You can check what cp(1) is doing using strace(1). Often enlightening :) [...] [*] http://lists.gnu.org/archive/html/coreutils-announce/2011-02/msg0.html [**] http://lwn.net/Articles/429345/ Subscribera-only, should become public soon. In the meantime, try http://lwn.net/Articles/429349/
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Fri, Feb 25, 2011 at 5:42 PM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: On Saturday 29 January 2011 04:20 PM, Dushyant Bansal wrote: Or this: which is faster, qemu-img convert -fformat -Oformat src-image dst-image or cpsrc-image dst-image? What about for raw images, shouldn't that be the same speed as cp(1)? Poke around the source code, profile it, understand what it's doing, think about ways to improve it. No need to do everything, just doing part of this will give you background on QEMU's block layer. Contributing patches is a good way get up to speed and show your skills. If time doesn't permit that, just think about the problem and how you intend to solve it, and feel free to bounce ideas off me. I explored 'qemu-img create and convert' and got a basic understanding of how they work. Great, it's good to hear from you. cp faster than qemu-img convert Yes, I've experienced that too. For raw-raw In cp, it just copies all the disk blocks actually occupied by the file. And, with qemu-img convert, it checks all the sectors and copy those, which contains atleast one non-NUL byte. The better performance of cp over qemu-img convert is the result of overhead of this checking. How did you find out what cp(1) and qemu-img do? How does cp(1) know which disk blocks are actually occupied? I tried a few variations: 1. just copy all the sectors without checking So, actual size becomes equal to virtual size. Did that make qemu-img faster for the image file you tested? 2. In is_allocated_sectors,out of n sectors, if any sector has a non-NUL byte then break and copy all n sectors. As expected, resultant raw image was quite large in size. This is kind of like what cp(1) does, except it limits n to 32 KB maximum at a time. Maybe if you add this tweak they will show similar performance. The drawback is that the output image is larger than with the current approach. Stefan
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
In cp, it just copies all the disk blocks actually occupied by the file. And, with qemu-img convert, it checks all the sectors and copy those, which contains atleast one non-NUL byte. The better performance of cp over qemu-img convert is the result of overhead of this checking. How did you find out what cp(1) and qemu-img do? How does cp(1) know which disk blocks are actually occupied? I have looked into their source code. Yes, that was not correct. cp also checks each block and copy those which contains non-null byte. qemu-img does the same thing with sectors. I tried a few variations: 1. just copy all the sectors without checking So, actual size becomes equal to virtual size. Did that make qemu-img faster for the image file you tested? No, in fact it becomes slower. I guess it is due to the increase in disk write. 2. In is_allocated_sectors,out of n sectors, if any sector has a non-NUL byte then break and copy all n sectors. As expected, resultant raw image was quite large in size. This is kind of like what cp(1) does, except it limits n to 32 KB maximum at a time. Maybe if you add this tweak they will show similar performance. The drawback is that the output image is larger than with the current approach. Stefan Disk block size is usually 512 bytes and in qemu-img, sector size is also 512B. And, this change would copy n sectors even if only one of them actually contains data (while cp checks and copies one block(=sector) at a time). Therefore, it will end up writing more data than cp. virtual size: 10G (10737418240 bytes) disk size: 569M convert- original time0m52.522s convert- modified (resultant disk size: 5.3G) time2m12.744s cp time0m51.724s (same disk size) --- virtual size: 10G (10737418240 bytes) disk size: 3.6G convert- original time1m52.249s convert- modified (resultant disk size: 7.1G) time3m2.891s cp time1m55.320s (same disk size) --- In these results, we can see that resultant disk size has increased. Thanks, Dushyant
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Saturday 29 January 2011 04:20 PM, Dushyant Bansal wrote: Or this: which is faster, qemu-img convert -fformat -Oformat src-image dst-image or cpsrc-image dst-image? What about for raw images, shouldn't that be the same speed as cp(1)? Poke around the source code, profile it, understand what it's doing, think about ways to improve it. No need to do everything, just doing part of this will give you background on QEMU's block layer. Contributing patches is a good way get up to speed and show your skills. If time doesn't permit that, just think about the problem and how you intend to solve it, and feel free to bounce ideas off me. I explored 'qemu-img create and convert' and got a basic understanding of how they work. cp faster than qemu-img convert For raw-raw In cp, it just copies all the disk blocks actually occupied by the file. And, with qemu-img convert, it checks all the sectors and copy those, which contains atleast one non-NUL byte. The better performance of cp over qemu-img convert is the result of overhead of this checking. I tried a few variations: 1. just copy all the sectors without checking So, actual size becomes equal to virtual size. 2. In is_allocated_sectors,out of n sectors, if any sector has a non-NUL byte then break and copy all n sectors. As expected, resultant raw image was quite large in size. Looking forward to your comments. Thanks, Dushyant
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
http://www.google-melange.com/document/show/gsoc_program/google/gsoc2011/timeline I'd like to see an in-place QCOW2- QED image converter with tests. I'm interested in mentoring this year. Stefan Hi, I am interested in working on this project as part of gsoc. I have prior experience working with qemu. After reading some basic information on qcow2 and qed from: 1. http://lists.nongnu.org/archive/html/qemu-devel/2010-09/msg00310.html 2. http://people.gnome.org/~markmc/qcow-image-format.html I am now planning to start reading their source code. What do you suggest? Are there any other specific things that I should read about? P.S. I just found out another new qemu image format FVD(http://wiki.qemu.org/Features/FVD/Compare). I will try to read it too. Regards, Dushyant
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Sat, Jan 29, 2011 at 10:50 AM, Dushyant Bansal cs5070...@cse.iitd.ac.in wrote: I am interested in working on this project as part of gsoc. I have prior experience working with qemu. Great. I have put up a more detailed description on the QEMU GSoC 2011 page: http://wiki.qemu.org/Google_Summer_of_Code_2011#QCOW2_.3C-.3E_QED_image_converter Please be aware that organizations have not yet been selected by Google and that GSoC planning is still in the early stages. I want to make sure that your expectations and the amount of upfront effort you invest at this early stage reflect that so you're not disappointed if things don't come together. I am now planning to start reading their source code. What do you suggest? Are there any other specific things that I should read about? btrfs has an ext3 conversion mode that could be inspiring: https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3 You could tackle open bugs to familiarize yourself with the codebase: http://goo.gl/DMxOD Or this: which is faster, qemu-img convert -f format -O format src-image dst-image or cp src-image dst-image? What about for raw images, shouldn't that be the same speed as cp(1)? Poke around the source code, profile it, understand what it's doing, think about ways to improve it. No need to do everything, just doing part of this will give you background on QEMU's block layer. Contributing patches is a good way get up to speed and show your skills. If time doesn't permit that, just think about the problem and how you intend to solve it, and feel free to bounce ideas off me. Stefan
[Qemu-devel] Re: KVM call agenda for Jan 25
On 01/25/2011 04:35 PM, Stefan Hajnoczi wrote: On Tue, Jan 25, 2011 at 2:26 PM, Avi Kivitya...@redhat.com wrote: On 01/25/2011 12:06 AM, Anthony Liguori wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer I have a perpetually in progress branch for this, and would very much like to see this done. Seen this? http://repo.or.cz/w/qemu/stefanha.git/commit/8179e8ff20bb3f14f361109afe5b3bf2bac24f0d http://repo.or.cz/w/qemu/stefanha.git/shortlog/8179e8ff20bb3f14f361109afe5b3bf2bac24f0d And the qemu-devel thread: http://www.mail-archive.com/qemu-devel@nongnu.org/msg52522.html Thanks. Will follow up on the thread. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Mon, 24 Jan 2011 16:06:34 -0600 Anthony Liguori anth...@codemonkey.ws wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere - Let's start planning our next release in advance, here's a simple example: http://wiki.qemu.org/Planning/0.15-example Regards, Anthony Liguori thanks, -chris -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Tue, 25 Jan 2011 11:57:27 -0200 Luiz Capitulino lcapitul...@redhat.com wrote: On Mon, 24 Jan 2011 16:06:34 -0600 Anthony Liguori anth...@codemonkey.ws wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere - Let's start planning our next release in advance, here's a simple example: http://wiki.qemu.org/Planning/0.15-example Forgot: - Google summer of code 2011 is on, are we interested? (note: I just saw the news, I don't have any information yet) Regards, Anthony Liguori thanks, -chris -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Luiz Capitulino a écrit : On Mon, 24 Jan 2011 16:06:34 -0600 Anthony Liguori anth...@codemonkey.ws wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere - Let's start planning our next release in advance, here's a simple example: http://wiki.qemu.org/Planning/0.15-example What about planning the 0.14 release first? From what I see on the mailing list, we are more in a development phase than in a bug fix phase before a release. -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurel...@aurel32.net http://www.aurel32.net
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Tue, Jan 25, 2011 at 2:02 PM, Luiz Capitulino lcapitul...@redhat.com wrote: - Google summer of code 2011 is on, are we interested? (note: I just saw the news, I don't have any information yet) http://www.google-melange.com/document/show/gsoc_program/google/gsoc2011/timeline I'd like to see an in-place QCOW2 - QED image converter with tests. I'm interested in mentoring this year. Stefan
[Qemu-devel] Re: KVM call agenda for Jan 25
On 01/25/2011 12:06 AM, Anthony Liguori wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer I have a perpetually in progress branch for this, and would very much like to see this done. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On 01/25/2011 08:11 AM, Aurelien Jarno wrote: Luiz Capitulino a écrit : On Mon, 24 Jan 2011 16:06:34 -0600 Anthony Liguorianth...@codemonkey.ws wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere - Let's start planning our next release in advance, here's a simple example: http://wiki.qemu.org/Planning/0.15-example What about planning the 0.14 release first? From what I see on the mailing list, we are more in a development phase than in a bug fix phase before a release. Yeah, lesson learned is releasing before a holiday is probably a bad idea. What would be a reasonable window to get us in bug fixing mode to do a release? How would a 2/1 stable-0.14 branch sound? Regards, Anthony Liguori
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
Am 25.01.2011 15:11, schrieb Aurelien Jarno: Luiz Capitulino a écrit : On Mon, 24 Jan 2011 16:06:34 -0600 Anthony Liguori anth...@codemonkey.ws wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere - Let's start planning our next release in advance, here's a simple example: http://wiki.qemu.org/Planning/0.15-example What about planning the 0.14 release first? From what I see on the mailing list, we are more in a development phase than in a bug fix phase before a release. If we want to get into a bug fix phase, all we need to have is a stable-0.14 branch. It's not like creating a branch is a lot of work, it just didn't happen on the announced date. And I'm almost sure that when it happens, it's going to come as a surprise once again... Kevin
[Qemu-devel] Re: KVM call agenda for Jan 25
On Tue, Jan 25, 2011 at 2:26 PM, Avi Kivity a...@redhat.com wrote: On 01/25/2011 12:06 AM, Anthony Liguori wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer I have a perpetually in progress branch for this, and would very much like to see this done. Seen this? http://repo.or.cz/w/qemu/stefanha.git/commit/8179e8ff20bb3f14f361109afe5b3bf2bac24f0d http://repo.or.cz/w/qemu/stefanha.git/shortlog/8179e8ff20bb3f14f361109afe5b3bf2bac24f0d And the qemu-devel thread: http://www.mail-archive.com/qemu-devel@nongnu.org/msg52522.html Stefan
Re: [Qemu-devel] Re: KVM call agenda for Jan 25
On Tue, Jan 25, 2011 at 03:42:04PM +0100, Kevin Wolf wrote: Am 25.01.2011 15:11, schrieb Aurelien Jarno: Luiz Capitulino a écrit : On Mon, 24 Jan 2011 16:06:34 -0600 Anthony Liguori anth...@codemonkey.ws wrote: On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere - Let's start planning our next release in advance, here's a simple example: http://wiki.qemu.org/Planning/0.15-example What about planning the 0.14 release first? From what I see on the mailing list, we are more in a development phase than in a bug fix phase before a release. If we want to get into a bug fix phase, all we need to have is a stable-0.14 branch. It's not like creating a branch is a lot of work, it just didn't happen on the announced date. Agreed. That said when you see the branch date approaching you already focus more on identifying and fixing bugs than on new features, that's human. For example after seeing the mail from Anthony I started to test the main TCG targets (actually 8 of them) on the main supported hosts (7 of them if you count endianness). That's quite a lot of work that has been quite useful, that said given the number of patches that have been committed in the meanwhile, it's probably something to redo. And I'm almost sure that when it happens, it's going to come as a surprise once again... If we decide on a date in advance, it's something I can easily do if Anthony or someone else is busy. We just have to agree on that before. -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurel...@aurel32.net http://www.aurel32.net
[Qemu-devel] Re: KVM call agenda for Jan 25
On 01/24/2011 07:25 AM, Chris Wright wrote: Please send in any agenda items you are interested in covering. - coroutines for the block layer - glib everywhere Regards, Anthony Liguori thanks, -chris -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
