from:"qiaonuohan"

Re: [Qemu-devel] [PATCH 7/8] dump: Fix use-after-free in create_kdump_vmcore()

2014-05-26 Thread qiaonuohan


On 05/27/2014 09:40 AM, arei.gong...@huawei.com wrote:

From: Gongleiarei.gong...@huawei.com

Spotted by Coverity:

(7) Event closed_arg:  write_dump_pages(DumpState *) closes s-fd. [details]
Also see events:  [pass_closed_arg]

1490ret = write_dump_pages(s);
(8) Event cond_false:  Condition ret  0, taking false branch

1491if (ret  0) {
1492return -1;
(9) Event if_end:  End of if statement

1493}
1494
(10) Event pass_closed_arg:  Passing closed handle s-fd as an argument to function 
write_end_flat_header(int).
Also see events:  [closed_arg]

1495ret = write_end_flat_header(s-fd);
1496if (ret  0) {
1497dump_error(s, dump: failed to write end flat header.\n);
1498return -1;
1499}
1500
1501dump_completed(s);
1502
1503return 0;
1504}

Signed-off-by: Gongleiarei.gong...@huawei.com


Reviewed-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com


---
  dump.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/dump.c b/dump.c
index e56b7cf..3a704e9 100644
--- a/dump.c
+++ b/dump.c
@@ -1296,6 +1296,7 @@ static int write_dump_pages(DumpState *s)
  /* prepare buffer to store compressed data */
  len_buf_out = get_len_buf_out(s-page_size, s-flag_compress);
  if (len_buf_out == 0) {
+ret = -1;
  dump_error(s, dump: failed to get length of output buffer.\n);
  goto out;
  }



--
Regards
Qiao Nuohan

Re: [Qemu-devel] [PATCH] configure: Enable dead code (lzo, snappy, quorum)

2014-04-29 Thread qiaonuohan


On 04/29/2014 02:21 PM, Stefan Weil wrote:

Those options were not enabled by default, even when the build
environment would have supported them, so the corresponding
code was not compiled in normal test builds like on build bots.

Signed-off-by: Stefan Weils...@weilnetz.de


Reviewed-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com


---

I'm not sure whether this patch is trivial enough for qemu-trivial.
Are there any disadvantages when this code is enabled?

Stefan

  configure |   39 ---
  1 file changed, 24 insertions(+), 15 deletions(-)

diff --git a/configure b/configure
index 2fbec59..0faadd7 100755
--- a/configure
+++ b/configure
@@ -301,8 +301,8 @@ libusb=
  usb_redir=
  glx=
  zlib=yes
-lzo=no
-snappy=no
+lzo=
+snappy=
  guest_agent=
  guest_agent_with_vss=no
  vss_win32_sdk=
@@ -323,7 +323,7 @@ vte=
  tpm=no
  libssh2=
  vhdx=
-quorum=no
+quorum=

  # parse CC options first
  for opt do
@@ -1023,8 +1023,12 @@ for opt do
;;
--disable-zlib-test) zlib=no
;;
+  --disable-lzo) lzo=no
+  ;;
--enable-lzo) lzo=yes
;;
+  --disable-snappy) snappy=no
+  ;;
--enable-snappy) snappy=yes
;;
--enable-guest-agent) guest_agent=yes
@@ -1722,13 +1726,14 @@ if test $lzo != no ; then
  int main(void) { lzo_version(); return 0; }
  EOF
  if compile_prog  -llzo2 ; then
-:
+libs_softmmu=$libs_softmmu -llzo2
+lzo=yes
  else
-error_exit lzo check failed \
-Make sure to have the lzo libs and headers installed.
+if test $lzo = yes; then
+feature_not_found liblzo2 Install liblzo2 devel
+fi
+lzo=no
  fi
-
-libs_softmmu=$libs_softmmu -llzo2
  fi

  ##
@@ -1740,13 +1745,14 @@ if test $snappy != no ; then
  int main(void) { snappy_max_compressed_length(4096); return 0; }
  EOF
  if compile_prog  -lsnappy ; then
-:
+libs_softmmu=$libs_softmmu -lsnappy
+snappy=yes
  else
-error_exit snappy check failed \
-Make sure to have the snappy libs and headers installed.
+if test $snappy = yes; then
+feature_not_found libsnappy Install libsnappy devel
+fi
+snappy=no
  fi
-
-libs_softmmu=$libs_softmmu -lsnappy
  fi

  ##
@@ -2172,9 +2178,12 @@ if compile_prog $quorum_tls_cflags $quorum_tls_libs 
; then
libs_softmmu=$quorum_tls_libs $libs_softmmu
libs_tools=$quorum_tls_libs $libs_softmmu
QEMU_CFLAGS=$QEMU_CFLAGS $quorum_tls_cflags
+  quorum=yes
  else
-  echo gnutls  2.10.0 required to compile Quorum
-  exit 1
+  if test $quorum = yes; then
+feature_not_found gnutls gnutls  2.10.0 required to compile Quorum
+  fi
+  quorum=no
  fi
  fi




--
Regards
Qiao Nuohan

Re: [Qemu-devel] [PATCH v2 0/2] HMP: support specifying dump format for dump-guest-memory

2014-04-23 Thread qiaonuohan


On 04/23/2014 09:30 PM, Luiz Capitulino wrote:

On Wed, 23 Apr 2014 12:38:12 +0200
Christian Borntraegerborntrae...@de.ibm.com  wrote:


On 17/04/14 10:15, Qiao Nuohan wrote:

The last version is here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00018.html

ChangLog:
Changes from v7 to v8:
1. add a patch to fix doc of dump-guest-memory

Qiao Nuohan (2):
   HMP: fix doc of dump-guest-memory
   HMP: support specifying dump format for dump-guest-memory

  hmp-commands.hx | 30 +++---
  hmp.c   | 25 ++---
  2 files changed, 41 insertions(+), 14 deletions(-)



Luiz,

are you going to pick these patches for your HMP tree?


Yes, but I'm late with QMP review.
.



Hello,

Do you guys need I fix the 1st patch as Markus suggested?

--
Regards
Qiao Nuohan

Re: [Qemu-devel] [PATCH v2 0/2] HMP: support specifying dump format for dump-guest-memory

2014-04-21 Thread qiaonuohan


On 04/17/2014 04:15 PM, Qiao Nuohan wrote:

The last version is here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00018.html

ChangLog:
Changes from v7 to v8:
1. add a patch to fix doc of dump-guest-memory

Qiao Nuohan (2):
   HMP: fix doc of dump-guest-memory
   HMP: support specifying dump format for dump-guest-memory

  hmp-commands.hx | 30 +++---
  hmp.c   | 25 ++---
  2 files changed, 41 insertions(+), 14 deletions(-)



ping...

--
Regards
Qiao Nuohan

Re: [Qemu-devel] [PATCH v2 0/2] HMP: support specifying dump format for dump-guest-memory

2014-04-17 Thread qiaonuohan


On 04/16/2014 04:51 PM, Qiao Nuohan wrote:

The last version is here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00018.html

ChangLog:
Changes from v7 to v8:
1. add a patch to fix doc of dump-guest-memory

Qiao Nuohan (2):
   HMP: fix doc of dump-guest-memory
   HMP: support specifying dump format for dump-guest-memory

  hmp-commands.hx | 25 -
  hmp.c   | 25 ++---
  2 files changed, 38 insertions(+), 12 deletions(-)



please ignore this series. I have sent the new one

--
Regards
Qiao Nuohan

Re: [Qemu-devel] [PATCH] HMP: support specifying dump format for dump-guest-memory

2014-04-08 Thread qiaonuohan


ping...

On 04/03/2014 08:16 PM, Christian Borntraeger wrote:

On 01/04/14 15:25, Christian Borntraeger wrote:

On 01/04/14 10:33, Qiao Nuohan wrote:

Dumping guest memory is available to specify the dump format now. This patch
adds options '-z|-l|-s' to HMP command dump-guest-memory to specify dumping in
kdump-compression format, with zlib/lzo/snappy compression. And without these
options ELF format will be used.

The discussion about this feature is here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-03/msg04235.html

Signed-off-by: Qiao Nuohanqiaonuo...@cn.fujitsu.com
Suggested-by: Christian Borntraegerborntrae...@de.ibm.com


Looks good. I was able to take a zlib dump on s390.



In other words:
Acked-by: Christian Borntraegerborntrae...@de.ibm.com





---
  hmp-commands.hx | 11 +++
  hmp.c   | 25 -
  2 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index f3fc514..4b9989f 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -998,8 +998,8 @@ ETEXI

  {
  .name   = dump-guest-memory,
-.args_type  = paging:-p,filename:F,begin:i?,length:i?,
-.params = [-p] filename [begin] [length],
+.args_type  = 
paging:-p,zlib:-z,lzo:-l,snappy:-s,filename:F,begin:i?,length:i?,
+.params = [-p] [-z|-l|-s] filename [begin] [length],
  .help   = dump guest memory to file
\n\t\t\t begin(optional): the starting physical 
address
\n\t\t\t length(optional): the memory size, in bytes,
@@ -1008,12 +1008,15 @@ ETEXI


  STEXI
-@item dump-guest-memory [-p] @var{protocol} @var{begin} @var{length}
+@item dump-guest-memory [-p] [-z|-l|-s] @var{protocol} @var{begin} @var{length}
  @findex dump-guest-memory
  Dump guest memory to @var{protocol}. The file can be processed with crash or
-gdb.
+gdb. Without -z|-l|-s, the dump format is ELF.
filename: dump file name
  paging: do paging to get guest's memory mapping
+  zlib: dump in kdump-compressed format, with zlib compression
+   lzo: dump in kdump-compressed format, with lzo compression
+snappy: dump in kdump-compressed format, with snappy compression
   begin: the starting physical address. It's optional, and should be
  specified with length together.
  length: the memory size, in bytes. It's optional, and should be specified
diff --git a/hmp.c b/hmp.c
index 2f279c4..37c3961 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1308,16 +1308,39 @@ void hmp_dump_guest_memory(Monitor *mon, const QDict 
*qdict)
  {
  Error *errp = NULL;
  int paging = qdict_get_try_bool(qdict, paging, 0);
+int zlib = qdict_get_try_bool(qdict, zlib, 0);
+int lzo = qdict_get_try_bool(qdict, lzo, 0);
+int snappy = qdict_get_try_bool(qdict, snappy, 0);
  const char *file = qdict_get_str(qdict, filename);
  bool has_begin = qdict_haskey(qdict, begin);
  bool has_length = qdict_haskey(qdict, length);
-/* kdump-compressed format is not supported for HMP */
  bool has_format = false;
  int64_t begin = 0;
  int64_t length = 0;
  enum DumpGuestMemoryFormat dump_format = DUMP_GUEST_MEMORY_FORMAT_ELF;
  char *prot;

+if ((zlib + lzo + snappy)  1) {
+error_setg(errp, only one of '-z|-l|-s' can be set);
+hmp_handle_error(mon,errp);
+return;
+}
+
+if (zlib) {
+has_format = true;
+dump_format = DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB;
+}
+
+if (lzo) {
+has_format = true;
+dump_format = DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO;
+}
+
+if (snappy) {
+has_format = true;
+dump_format = DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY;
+}
+
  if (has_begin) {
  begin = qdict_get_int(qdict, begin);
  }






.




--
Regards
Qiao Nuohan

Re: [Qemu-devel] hmp interface for kdump compressed format

2014-04-01 Thread qiaonuohan


Hello guys,

I have sent the patch, please check here:

http://lists.nongnu.org/archive/html/qemu-devel/2014-04/msg00018.html

On 03/27/2014 04:38 PM, Markus Armbruster wrote:

qiaonuo...@cn.fujitsu.comqiaonuo...@cn.fujitsu.com  writes:


On 03/27/2014 01:04 AM, Markus Armbruster wrote:

So something like adding


  dump_guest_memory_set_formatformat

  would be the only possible solution with the hmp code as
is. Correct?


  Yes, one possibility would be to make the dump command stateful (=
  compression format), and to add a new command
setting/getting that state.

  Another option would be to leave the current command intact,
and add an
  independent command that wouldn't take begin, end, nor
paging, but
  would take compression format.


  Another possibility is to kill begin/length from the
  dump-guest-memory command.
  HMP is not stable, so if that's useful we can do it.


  AFAIK, kdump-compressed format can only be analyzed by crash-utility right
  now. ELF is big but we still need it, then begin/length will help us save
  time and space when only part of the memory is need.

  IMHO, a new HMP command will be a good choice and it is more simple than
  making dump format 'stateful'. I am not so clear about HMP, but I think
  the new HMP command can be 'dump-guest-memory-with-format', and still
  base on qmp_dump_guest_memory. If you guys like it, I will send a patch
  that add this command.

Paolo encouraged you to*break*  the existing HMP command instead of
adding a new one.  I'd like to second that.
.



Hello markus,

I have finish my patch as I stated in my last mail, but something is
wrong with my mail server, I have to wait a couple of days before
sending the patch. You said you prefer *breaking* the existing HMP
command. Since paging/begin/length is still usefull, would you please
give some reason and I will consider that.


I'm encouraging you to design a HMP command that gives you all you need.
Then call that dump-guest-memory, backward compatibility be damned.

Would

 dump-guest-memory [-p] filename [[format] begin length]

do?

An alternative could be two commands:

 dump-guest-memory filename [format [begin length]]
 dump-guest-memory-elf [-p] filename [begin length]

This bakes the restriction paging, begin and length work only with
format 'elf' into the interface.  Not sure that's a good idea, but I'm
leaving that to folks who actually know something about this dumping
business.

If you can find better command names, go right ahead.  You could pick
names that avoid incompatible change.  I care about avoiding
incompatible change to exotic HMP commands even less (a lot less, in
fact) than about naming of exotic HMP commands.

.




--
Regards
Qiao Nuohan

[Qemu-devel] [PATCH v9 00/14] Make 'dump-guest-memory' dump in kdump-compressed format

2014-02-17 Thread qiaonuohan

Hi, all

The last version is here:
http://lists.nongnu.org/archive/html/qemu-devel/2014-01/msg03669.html

Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

Compared with migration, the missing of compression feature means regression
to 'dump-guest-memory'. So we post these patches to make 'dump-guest-memory' be
able to dump guest's in kdump-compressed format. Then vmcore can be much
smaller, and easily to be delivered.

The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support. To get more detailed information about
kdump-compressed format, please refer to the following URL:
http://sourceforge.net/projects/makedumpfile/

Note:
1. The guest should be x86/x86_64/s390x. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
   available by default, and option '--enable-lzo' or '--enable-snappy'
   should be specified with 'configure' to make lzo or snappy available.

Changelog:
Changes from v8 to v9:
1. add Ekaterina's patch(support s390x) into series
2. modify query-dump-guest-memory-capability according to Paolo Bonzini's
   suggestion
3. fix compiling error on 32-bit host

Changes from v7 to v8:
1. rebase get_max_mapnr()
2. fix bug of using dh-block_size directly in create_header64()
3. abandon using static variables in get_next_page()
4. redefine 'struct PageDesc' to 'struct QEMU_PACKED PageDescriptor'
5. add return when format checking fails
Changes from v6 to v7:
1. support BE host
2. abandon non-flatten format to avoid using seek on vmcore
3. abandon using of very large array
4. use get_next_page to replace the iteration of guest's pages
5. abandon the support of HMP

Changes from v5 to v6:
1. add run-time check for compression format(lzo/snappy)
2. address Stefan's comments about reusing code and coding style
3. update the version of kdump-compressed format to 6th
4. resplit the patches
5. Add 'query-dump-guest-memory-capability' command

Changes from v4 to v5:
1. using flatten format to avoid using temporary files according to Stefan's
   comments
2. Address Andreas's comments about coding style

Changes from v3 to v4:
1. change to avoid conflict with Andreas's patches
2. rebase

Changes from v2 to v3:
1. Address Eric's comment

Changes from v1 to v2:
1. Address Eric  Daniel's comment: fix manner of string copy.
2. Address Eric's comment: replace reinventing new constants by using the
   ready-made ones accoring.
3. Address Andreas's comment: remove useless include.

qiaonuohan (14):
  dump: const-qualify the buf of WriteCoreDumpFunction
  dump: add argument to write_elfxx_notes
  dump: add API to write header of flatten format
  dump: add API to write vmcore
  dump: add API to write elf notes to buffer
  dump: add support for lzo/snappy
  dump: add members to DumpState and init some of them
  dump: add API to write dump header
  dump: add API to write dump_bitmap
  dump: add APIs to operate DataCache
  dump: add API to write dump pages
  dump: make kdump-compressed format available for 'dump-guest-memory'
  Define the architecture for compressed dump format
  dump: add 'query-dump-guest-memory-capability' command

 configure |   54 +++
 dump.c|  960 -
 hmp.c |5 +-
 include/qom/cpu.h |3 +-
 include/sysemu/dump.h |  138 +++
 qapi-schema.json  |   49 +++-
 qmp-commands.hx   |   27 ++-
 target-i386/cpu.h |2 +
 target-s390x/cpu.h|1 +
 9 files changed, 1219 insertions(+), 20 deletions(-)

[Qemu-devel] [PATCH v9 02/14] dump: add argument to write_elfxx_notes

2014-02-17 Thread qiaonuohan

write_elf32_notes/wirte_elf64_notes use fd_write_vmcore to write elf notes to
vmcore. Adding parameter WriteCoreDumpFunction f makes it available to choose
the method of writing elf notes

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   16 
 1 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/dump.c b/dump.c
index 42622de..c9d3492 100644
--- a/dump.c
+++ b/dump.c
@@ -271,7 +271,7 @@ static inline int cpu_index(CPUState *cpu)
 return cpu-cpu_index + 1;
 }
 
-static int write_elf64_notes(DumpState *s)
+static int write_elf64_notes(WriteCoreDumpFunction f, DumpState *s)
 {
 CPUState *cpu;
 int ret;
@@ -279,7 +279,7 @@ static int write_elf64_notes(DumpState *s)
 
 CPU_FOREACH(cpu) {
 id = cpu_index(cpu);
-ret = cpu_write_elf64_note(fd_write_vmcore, cpu, id, s);
+ret = cpu_write_elf64_note(f, cpu, id, s);
 if (ret  0) {
 dump_error(s, dump: failed to write elf notes.\n);
 return -1;
@@ -287,7 +287,7 @@ static int write_elf64_notes(DumpState *s)
 }
 
 CPU_FOREACH(cpu) {
-ret = cpu_write_elf64_qemunote(fd_write_vmcore, cpu, s);
+ret = cpu_write_elf64_qemunote(f, cpu, s);
 if (ret  0) {
 dump_error(s, dump: failed to write CPU status.\n);
 return -1;
@@ -321,7 +321,7 @@ static int write_elf32_note(DumpState *s)
 return 0;
 }
 
-static int write_elf32_notes(DumpState *s)
+static int write_elf32_notes(WriteCoreDumpFunction f, DumpState *s)
 {
 CPUState *cpu;
 int ret;
@@ -329,7 +329,7 @@ static int write_elf32_notes(DumpState *s)
 
 CPU_FOREACH(cpu) {
 id = cpu_index(cpu);
-ret = cpu_write_elf32_note(fd_write_vmcore, cpu, id, s);
+ret = cpu_write_elf32_note(f, cpu, id, s);
 if (ret  0) {
 dump_error(s, dump: failed to write elf notes.\n);
 return -1;
@@ -337,7 +337,7 @@ static int write_elf32_notes(DumpState *s)
 }
 
 CPU_FOREACH(cpu) {
-ret = cpu_write_elf32_qemunote(fd_write_vmcore, cpu, s);
+ret = cpu_write_elf32_qemunote(f, cpu, s);
 if (ret  0) {
 dump_error(s, dump: failed to write CPU status.\n);
 return -1;
@@ -574,7 +574,7 @@ static int dump_begin(DumpState *s)
 }
 
 /* write notes to vmcore */
-if (write_elf64_notes(s)  0) {
+if (write_elf64_notes(fd_write_vmcore, s)  0) {
 return -1;
 }
 
@@ -597,7 +597,7 @@ static int dump_begin(DumpState *s)
 }
 
 /* write notes to vmcore */
-if (write_elf32_notes(s)  0) {
+if (write_elf32_notes(fd_write_vmcore, s)  0) {
 return -1;
 }
 }
-- 
1.7.1

[Qemu-devel] [PATCH v9 13/14] Define the architecture for compressed dump format

2014-02-17 Thread qiaonuohan

Signed-off-by: Ekaterina Tumanova tuman...@linux.vnet.ibm.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
Reviewed-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c |7 +--
 target-i386/cpu.h  |2 ++
 target-s390x/cpu.h |1 +
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/dump.c b/dump.c
index fb0c896..2c81318 100644
--- a/dump.c
+++ b/dump.c
@@ -32,6 +32,9 @@
 #ifdef CONFIG_SNAPPY
 #include snappy-c.h
 #endif
+#ifndef ELF_MACHINE_UNAME
+#define ELF_MACHINE_UNAME Unknown
+#endif
 
 static uint16_t cpu_convert_to_target16(uint16_t val, int endian)
 {
@@ -817,7 +820,7 @@ static int create_header32(DumpState *s)
 dh-nr_cpus = cpu_convert_to_target32(s-nr_cpus, endian);
 bitmap_blocks = DIV_ROUND_UP(s-len_dump_bitmap, block_size) * 2;
 dh-bitmap_blocks = cpu_convert_to_target32(bitmap_blocks, endian);
-memcpy((dh-utsname.machine), i686, 4);
+strncpy(dh-utsname.machine, ELF_MACHINE_UNAME, 
sizeof(dh-utsname.machine));
 
 if (s-flag_compress  DUMP_DH_COMPRESSED_ZLIB) {
 status |= DUMP_DH_COMPRESSED_ZLIB;
@@ -924,7 +927,7 @@ static int create_header64(DumpState *s)
 dh-nr_cpus = cpu_convert_to_target32(s-nr_cpus, endian);
 bitmap_blocks = DIV_ROUND_UP(s-len_dump_bitmap, block_size) * 2;
 dh-bitmap_blocks = cpu_convert_to_target32(bitmap_blocks, endian);
-memcpy((dh-utsname.machine), x86_64, 6);
+strncpy(dh-utsname.machine, ELF_MACHINE_UNAME, 
sizeof(dh-utsname.machine));
 
 if (s-flag_compress  DUMP_DH_COMPRESSED_ZLIB) {
 status |= DUMP_DH_COMPRESSED_ZLIB;
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 1b94f0f..6abcd23 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -38,8 +38,10 @@
 
 #ifdef TARGET_X86_64
 #define ELF_MACHINE EM_X86_64
+#define ELF_MACHINE_UNAME x86_64
 #else
 #define ELF_MACHINE EM_386
+#define ELF_MACHINE_UNAME i686
 #endif
 
 #define CPUArchState struct CPUX86State
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 96c2b4a..6d46827 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -28,6 +28,7 @@
 #define TARGET_LONG_BITS 64
 
 #define ELF_MACHINEEM_S390
+#define ELF_MACHINE_UNAME S390X
 
 #define CPUArchState struct CPUS390XState
 
-- 
1.7.1

[Qemu-devel] [PATCH v9 14/14] dump: add 'query-dump-guest-memory-capability' command

2014-02-17 Thread qiaonuohan

'query-dump-guest-memory-capability' is used to query the available formats for
'dump-guest-memory'. The output of the command will be like:

- { execute: query-dump-guest-memory-capability }
- { return: { formats:
[elf, kdump-zlib, kdump-lzo, kdump-snappy] }

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Eric Blake ebl...@redhat.com
---
 dump.c   |   33 +
 qapi-schema.json |   24 
 qmp-commands.hx  |   20 
 3 files changed, 77 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 2c81318..14b3d1d 100644
--- a/dump.c
+++ b/dump.c
@@ -1791,3 +1791,36 @@ void qmp_dump_guest_memory(bool paging, const char 
*file, bool has_begin,
 
 g_free(s);
 }
+
+DumpGuestMemoryCapability *qmp_query_dump_guest_memory_capability(Error **errp)
+{
+DumpGuestMemoryFormatList *item;
+DumpGuestMemoryCapability *cap =
+  g_malloc0(sizeof(DumpGuestMemoryCapability));
+
+/* elf is always available */
+item = g_malloc0(sizeof(DumpGuestMemoryFormatList));
+cap-formats = item;
+item-value = DUMP_GUEST_MEMORY_FORMAT_ELF;
+
+/* kdump-zlib is always available */
+item-next = g_malloc0(sizeof(DumpGuestMemoryFormatList));
+item = item-next;
+item-value = DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB;
+
+/* add new item if kdump-lzo is available */
+#ifdef CONFIG_LZO
+item-next = g_malloc0(sizeof(DumpGuestMemoryFormatList));
+item = item-next;
+item-value = DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO;
+#endif
+
+/* add new item if kdump-snappy is available */
+#ifdef CONFIG_SNAPPY
+item-next = g_malloc0(sizeof(DumpGuestMemoryFormatList));
+item = item-next;
+item-value = DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY;
+#endif
+
+return cap;
+}
diff --git a/qapi-schema.json b/qapi-schema.json
index 905a8af..3eeb261 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2783,6 +2783,30 @@
 '*length': 'int', '*format': 'DumpGuestMemoryFormat' } }
 
 ##
+# @DumpGuestMemoryCapability:
+#
+# A list of the available formats for dump-guest-memory
+#
+# Since: 2.0
+##
+{ 'type': 'DumpGuestMemoryCapability',
+  'data': {
+  'formats': ['DumpGuestMemoryFormat'] } }
+
+##
+# @query-dump-guest-memory-capability:
+#
+# Returns the available formats for dump-guest-memory
+#
+# Returns:  A @DumpGuestMemoryCapability object listing available formats for
+#   dump-guest-memory
+#
+# Since: 2.0
+##
+{ 'command': 'query-dump-guest-memory-capability',
+  'returns': 'DumpGuestMemoryCapability' }
+
+##
 # @netdev_add:
 #
 # Add a network backend.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 019dde6..f38007c 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -829,6 +829,26 @@ Notes:
 EQMP
 
 {
+.name   = query-dump-guest-memory-capability,
+.args_type  = ,
+.mhandler.cmd_new = qmp_marshal_input_query_dump_guest_memory_capability,
+},
+
+SQMP
+query-dump-guest-memory-capability
+--
+
+Show available formats for 'dump-guest-memory'
+
+Example:
+
+- { execute: query-dump-guest-memory-capability }
+- { return: { formats:
+[elf, kdump-zlib, kdump-lzo, kdump-snappy] }
+
+EQMP
+
+{
 .name   = netdev_add,
 .args_type  = netdev:O,
 .mhandler.cmd_new = qmp_netdev_add,
-- 
1.7.1

[Qemu-devel] [PATCH v9 08/14] dump: add API to write dump header

2014-02-17 Thread qiaonuohan

the functions are used to write header of kdump-compressed format to vmcore.
Header of kdump-compressed format includes:
1. common header: DiskDumpHeader32 / DiskDumpHeader64
2. sub header: KdumpSubHeader32 / KdumpSubHeader64
3. extra information: only elf notes here

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|  223 +
 include/sysemu/dump.h |   96 +
 2 files changed, 319 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 3a1944e..4b2799f 100644
--- a/dump.c
+++ b/dump.c
@@ -778,6 +778,229 @@ static int buf_write_note(const void *buf, size_t size, 
void *opaque)
 return 0;
 }
 
+/* write common header, sub header and elf note to vmcore */
+static int create_header32(DumpState *s)
+{
+int ret = 0;
+DiskDumpHeader32 *dh = NULL;
+KdumpSubHeader32 *kh = NULL;
+size_t size;
+int endian = s-dump_info.d_endian;
+uint32_t block_size;
+uint32_t sub_hdr_size;
+uint32_t bitmap_blocks;
+uint32_t status = 0;
+uint64_t offset_note;
+
+/* write common header, the version of kdump-compressed format is 6th */
+size = sizeof(DiskDumpHeader32);
+dh = g_malloc0(size);
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = cpu_convert_to_target32(6, endian);
+block_size = s-page_size;
+dh-block_size = cpu_convert_to_target32(block_size, endian);
+sub_hdr_size = sizeof(struct KdumpSubHeader32) + s-note_size;
+sub_hdr_size = DIV_ROUND_UP(sub_hdr_size, block_size);
+dh-sub_hdr_size = cpu_convert_to_target32(sub_hdr_size, endian);
+/* dh-max_mapnr may be truncated, full 64bit is in kh.max_mapnr_64 */
+dh-max_mapnr = cpu_convert_to_target32(MIN(s-max_mapnr, UINT_MAX),
+endian);
+dh-nr_cpus = cpu_convert_to_target32(s-nr_cpus, endian);
+bitmap_blocks = DIV_ROUND_UP(s-len_dump_bitmap, block_size) * 2;
+dh-bitmap_blocks = cpu_convert_to_target32(bitmap_blocks, endian);
+memcpy((dh-utsname.machine), i686, 4);
+
+if (s-flag_compress  DUMP_DH_COMPRESSED_ZLIB) {
+status |= DUMP_DH_COMPRESSED_ZLIB;
+}
+#ifdef CONFIG_LZO
+if (s-flag_compress  DUMP_DH_COMPRESSED_LZO) {
+status |= DUMP_DH_COMPRESSED_LZO;
+}
+#endif
+#ifdef CONFIG_SNAPPY
+if (s-flag_compress  DUMP_DH_COMPRESSED_SNAPPY) {
+status |= DUMP_DH_COMPRESSED_SNAPPY;
+}
+#endif
+dh-status = cpu_convert_to_target32(status, endian);
+
+if (write_buffer(s-fd, 0, dh, size)  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+ret = -1;
+goto out;
+}
+
+/* write sub header */
+size = sizeof(KdumpSubHeader32);
+kh = g_malloc0(size);
+
+/* 64bit max_mapnr_64 */
+kh-max_mapnr_64 = cpu_convert_to_target64(s-max_mapnr, endian);
+kh-phys_base = cpu_convert_to_target32(PHYS_BASE, endian);
+kh-dump_level = cpu_convert_to_target32(DUMP_LEVEL, endian);
+
+offset_note = DISKDUMP_HEADER_BLOCKS * block_size + size;
+kh-offset_note = cpu_convert_to_target64(offset_note, endian);
+kh-note_size = cpu_convert_to_target32(s-note_size, endian);
+
+if (write_buffer(s-fd, DISKDUMP_HEADER_BLOCKS *
+ block_size, kh, size)  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+ret = -1;
+goto out;
+}
+
+/* write note */
+s-note_buf = g_malloc0(s-note_size);
+s-note_buf_offset = 0;
+
+/* use s-note_buf to store notes temporarily */
+if (write_elf32_notes(buf_write_note, s)  0) {
+ret = -1;
+goto out;
+}
+
+if (write_buffer(s-fd, offset_note, s-note_buf,
+ s-note_size)  0) {
+dump_error(s, dump: failed to write notes);
+ret = -1;
+goto out;
+}
+
+/* get offset of dump_bitmap */
+s-offset_dump_bitmap = (DISKDUMP_HEADER_BLOCKS + sub_hdr_size) *
+ block_size;
+
+/* get offset of page */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + sub_hdr_size + bitmap_blocks) *
+ block_size;
+
+out:
+g_free(dh);
+g_free(kh);
+g_free(s-note_buf);
+
+return ret;
+}
+
+/* write common header, sub header and elf note to vmcore */
+static int create_header64(DumpState *s)
+{
+int ret = 0;
+DiskDumpHeader64 *dh = NULL;
+KdumpSubHeader64 *kh = NULL;
+size_t size;
+int endian = s-dump_info.d_endian;
+uint32_t block_size;
+uint32_t sub_hdr_size;
+uint32_t bitmap_blocks;
+uint32_t status = 0;
+uint64_t offset_note;
+
+/* write common header, the version of kdump-compressed format is 6th */
+size = sizeof(DiskDumpHeader64);
+dh = g_malloc0(size);
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = cpu_convert_to_target32(6, endian);
+

[Qemu-devel] [PATCH v9 05/14] dump: add API to write elf notes to buffer

2014-02-17 Thread qiaonuohan

the function can be used by write_elf32_notes/write_elf64_notes to write notes
to a buffer. If fd_write_vmcore is used, write_elf32_notes/write_elf64_notes
will write elf notes to vmcore directly. Instead, if buf_write_note is used,
elf notes will be written to opaque-note_buf at first.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   19 +++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 238ffa5..2b940bd 100644
--- a/dump.c
+++ b/dump.c
@@ -76,6 +76,9 @@ typedef struct DumpState {
 int64_t begin;
 int64_t length;
 Error **errp;
+
+uint8_t *note_buf;  /* buffer for notes */
+size_t note_buf_offset; /* the writing place in note_buf */
 } DumpState;
 
 static int dump_cleanup(DumpState *s)
@@ -749,6 +752,22 @@ static int write_buffer(int fd, off_t offset, const void 
*buf, size_t size)
 return 0;
 }
 
+static int buf_write_note(const void *buf, size_t size, void *opaque)
+{
+DumpState *s = opaque;
+
+/* note_buf is not enough */
+if (s-note_buf_offset + size  s-note_size) {
+return -1;
+}
+
+memcpy(s-note_buf + s-note_buf_offset, buf, size);
+
+s-note_buf_offset += size;
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
-- 
1.7.1

[Qemu-devel] [PATCH v9 09/14] dump: add API to write dump_bitmap

2014-02-17 Thread qiaonuohan

functions are used to write 1st and 2nd dump_bitmap of kdump-compressed format,
which is used to indicate whether the corresponded page is existed in vmcore.
1st and 2nd dump_bitmap are same, because dump level is specified to 1 here.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|  164 +
 include/sysemu/dump.h |2 +
 2 files changed, 166 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 4b2799f..5755534 100644
--- a/dump.c
+++ b/dump.c
@@ -1001,6 +1001,170 @@ static int write_dump_header(DumpState *s)
 }
 }
 
+/*
+ * set dump_bitmap sequencely. the bit before last_pfn is not allowed to be
+ * rewritten, so if need to set the first bit, set last_pfn and pfn to 0.
+ * set_dump_bitmap will always leave the recently set bit un-sync. And setting
+ * (last bit + sizeof(buf) * 8) to 0 will do flushing the content in buf into
+ * vmcore, ie. synchronizing un-sync bit into vmcore.
+ */
+static int set_dump_bitmap(uint64_t last_pfn, uint64_t pfn, bool value,
+   uint8_t *buf, DumpState *s)
+{
+off_t old_offset, new_offset;
+off_t offset_bitmap1, offset_bitmap2;
+uint32_t byte, bit;
+
+/* should not set the previous place */
+assert(last_pfn = pfn);
+
+/*
+ * if the bit needed to be set is not cached in buf, flush the data in buf
+ * to vmcore firstly.
+ * making new_offset be bigger than old_offset can also sync remained data
+ * into vmcore.
+ */
+old_offset = BUFSIZE_BITMAP * (last_pfn / PFN_BUFBITMAP);
+new_offset = BUFSIZE_BITMAP * (pfn / PFN_BUFBITMAP);
+
+while (old_offset  new_offset) {
+/* calculate the offset and write dump_bitmap */
+offset_bitmap1 = s-offset_dump_bitmap + old_offset;
+if (write_buffer(s-fd, offset_bitmap1, buf,
+ BUFSIZE_BITMAP)  0) {
+return -1;
+}
+
+/* dump level 1 is chosen, so 1st and 2nd bitmap are same */
+offset_bitmap2 = s-offset_dump_bitmap + s-len_dump_bitmap +
+ old_offset;
+if (write_buffer(s-fd, offset_bitmap2, buf,
+ BUFSIZE_BITMAP)  0) {
+return -1;
+}
+
+memset(buf, 0, BUFSIZE_BITMAP);
+old_offset += BUFSIZE_BITMAP;
+}
+
+/* get the exact place of the bit in the buf, and set it */
+byte = (pfn % PFN_BUFBITMAP) / CHAR_BIT;
+bit = (pfn % PFN_BUFBITMAP) % CHAR_BIT;
+if (value) {
+buf[byte] |= 1u  bit;
+} else {
+buf[byte] = ~(1u  bit);
+}
+
+return 0;
+}
+
+/*
+ * exam every page and return the page frame number and the address of the 
page.
+ * bufptr can be NULL. note: the blocks here is supposed to reflect guest-phys
+ * blocks, so block-target_start and block-target_end should be interal
+ * multiples of the target page size.
+ */
+static bool get_next_page(GuestPhysBlock **blockptr, uint64_t *pfnptr,
+  uint8_t **bufptr, DumpState *s)
+{
+GuestPhysBlock *block = *blockptr;
+hwaddr addr;
+uint8_t *buf;
+
+/* block == NULL means the start of the iteration */
+if (!block) {
+block = QTAILQ_FIRST(s-guest_phys_blocks.head);
+*blockptr = block;
+assert(block-target_start % s-page_size == 0);
+assert(block-target_end % s-page_size == 0);
+*pfnptr = paddr_to_pfn(block-target_start, s-page_shift);
+if (bufptr) {
+*bufptr = block-host_addr;
+}
+return true;
+}
+
+*pfnptr = *pfnptr + 1;
+addr = pfn_to_paddr(*pfnptr, s-page_shift);
+
+if ((addr = block-target_start) 
+(addr + s-page_size = block-target_end)) {
+buf = block-host_addr + (addr - block-target_start);
+} else {
+/* the next page is in the next block */
+block = QTAILQ_NEXT(block, next);
+*blockptr = block;
+if (!block) {
+return false;
+}
+assert(block-target_start % s-page_size == 0);
+assert(block-target_end % s-page_size == 0);
+*pfnptr = paddr_to_pfn(block-target_start, s-page_shift);
+buf = block-host_addr;
+}
+
+if (bufptr) {
+*bufptr = buf;
+}
+
+return true;
+}
+
+static int write_dump_bitmap(DumpState *s)
+{
+int ret = 0;
+uint64_t last_pfn, pfn;
+void *dump_bitmap_buf;
+size_t num_dumpable;
+GuestPhysBlock *block_iter = NULL;
+
+/* dump_bitmap_buf is used to store dump_bitmap temporarily */
+dump_bitmap_buf = g_malloc0(BUFSIZE_BITMAP);
+
+num_dumpable = 0;
+last_pfn = 0;
+
+/*
+ * exam memory page by page, and set the bit in dump_bitmap corresponded
+ * to the existing page.
+ */
+while (get_next_page(block_iter, pfn, NULL, s)) {
+ret = set_dump_bitmap(last_pfn, pfn, true, dump_bitmap_buf, s);
+if (ret  0) {
+

[Qemu-devel] [PATCH v9 03/14] dump: add API to write header of flatten format

2014-02-17 Thread qiaonuohan

flatten format will be used when writing kdump-compressed format. The format is
also used by makedumpfile, you can refer to the following URL to get more
detailed information about flatten format of kdump-compressed format:
http://sourceforge.net/projects/makedumpfile/

The two functions here are used to write start flat header and end flat header
to vmcore, and they will be called later when flatten format is used.

struct MakedumpfileHeader stored at the head of vmcore is used to indicate the
vmcore is in flatten format.

struct MakedumpfileHeader {
char signature[16]; /* = makedumpfile */
int64_t type;   /* = 1 */
int64_t version;/* = 1 */
};

And struct MakedumpfileDataHeader, with offset and buf_size set to -1, is used
to indicate the end of vmcore in flatten format.

struct MakedumpfileDataHeader {
int64_t offset; /* = -1 */
int64_t buf_size;   /* = -1 */
};

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   42 ++
 include/sysemu/dump.h |   17 +
 2 files changed, 59 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index c9d3492..f233b3e 100644
--- a/dump.c
+++ b/dump.c
@@ -686,6 +686,48 @@ static int create_vmcore(DumpState *s)
 return 0;
 }
 
+static int write_start_flat_header(int fd)
+{
+uint8_t *buf;
+MakedumpfileHeader mh;
+int ret = 0;
+
+memset(mh, 0, sizeof(mh));
+strncpy(mh.signature, MAKEDUMPFILE_SIGNATURE,
+strlen(MAKEDUMPFILE_SIGNATURE));
+
+mh.type = cpu_to_be64(TYPE_FLAT_HEADER);
+mh.version = cpu_to_be64(VERSION_FLAT_HEADER);
+
+buf = g_malloc0(MAX_SIZE_MDF_HEADER);
+memcpy(buf, mh, sizeof(mh));
+
+size_t written_size;
+written_size = qemu_write_full(fd, buf, MAX_SIZE_MDF_HEADER);
+if (written_size != MAX_SIZE_MDF_HEADER) {
+ret = -1;
+}
+
+g_free(buf);
+return ret;
+}
+
+static int write_end_flat_header(int fd)
+{
+MakedumpfileDataHeader mdh;
+
+mdh.offset = END_FLAG_FLAT_HEADER;
+mdh.buf_size = END_FLAG_FLAT_HEADER;
+
+size_t written_size;
+written_size = qemu_write_full(fd, mdh, sizeof(mdh));
+if (written_size != sizeof(mdh)) {
+return -1;
+}
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 19fafb2..b32b390 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -14,12 +14,29 @@
 #ifndef DUMP_H
 #define DUMP_H
 
+#define MAKEDUMPFILE_SIGNATURE  makedumpfile
+#define MAX_SIZE_MDF_HEADER (4096) /* max size of makedumpfile_header 
*/
+#define TYPE_FLAT_HEADER(1)/* type of flattened format */
+#define VERSION_FLAT_HEADER (1)/* version of flattened format */
+#define END_FLAG_FLAT_HEADER(-1)
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
 int d_class;/* ELFCLASS32 or ELFCLASS64 */
 } ArchDumpInfo;
 
+typedef struct QEMU_PACKED MakedumpfileHeader {
+char signature[16]; /* = makedumpfile */
+int64_t type;
+int64_t version;
+} MakedumpfileHeader;
+
+typedef struct QEMU_PACKED MakedumpfileDataHeader {
+int64_t offset;
+int64_t buf_size;
+} MakedumpfileDataHeader;
+
 struct GuestPhysBlockList; /* memory_mapping.h */
 int cpu_get_dump_info(ArchDumpInfo *info,
   const struct GuestPhysBlockList *guest_phys_blocks);
-- 
1.7.1

[Qemu-devel] [PATCH v9 01/14] dump: const-qualify the buf of WriteCoreDumpFunction

2014-02-17 Thread qiaonuohan

WriteCoreDumpFunction is a function pointer that points to the function used to
write content in buf into core file, so buf should be const-qualify.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|2 +-
 include/qom/cpu.h |3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/dump.c b/dump.c
index 80a9116..42622de 100644
--- a/dump.c
+++ b/dump.c
@@ -99,7 +99,7 @@ static void dump_error(DumpState *s, const char *reason)
 dump_cleanup(s);
 }
 
-static int fd_write_vmcore(void *buf, size_t size, void *opaque)
+static int fd_write_vmcore(const void *buf, size_t size, void *opaque)
 {
 DumpState *s = opaque;
 size_t written_size;
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 367eda1..d734be8 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -28,7 +28,8 @@
 #include qemu/tls.h
 #include qemu/typedefs.h
 
-typedef int (*WriteCoreDumpFunction)(void *buf, size_t size, void *opaque);
+typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
+ void *opaque);
 
 /**
  * vaddr:
-- 
1.7.1

[Qemu-devel] [PATCH v9 04/14] dump: add API to write vmcore

2014-02-17 Thread qiaonuohan

Function is used to write vmcore in flatten format. In flatten format, data is
written block by block, and in front of each block, a struct
MakedumpfileDataHeader is stored there to indicate the offset and size of the
data block.

struct MakedumpfileDataHeader {
int64_t offset;
int64_t buf_size;
};

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   21 +
 1 files changed, 21 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index f233b3e..238ffa5 100644
--- a/dump.c
+++ b/dump.c
@@ -728,6 +728,27 @@ static int write_end_flat_header(int fd)
 return 0;
 }
 
+static int write_buffer(int fd, off_t offset, const void *buf, size_t size)
+{
+size_t written_size;
+MakedumpfileDataHeader mdh;
+
+mdh.offset = cpu_to_be64(offset);
+mdh.buf_size = cpu_to_be64(size);
+
+written_size = qemu_write_full(fd, mdh, sizeof(mdh));
+if (written_size != sizeof(mdh)) {
+return -1;
+}
+
+written_size = qemu_write_full(fd, buf, size);
+if (written_size != size) {
+return -1;
+}
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
-- 
1.7.1

[Qemu-devel] [PATCH v9 07/14] dump: add members to DumpState and init some of them

2014-02-17 Thread qiaonuohan

add some members to DumpState that will be used in writing vmcore in
kdump-compressed format. some of them, like page_size, will be initialized
in the patch.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   28 
 include/sysemu/dump.h |7 +++
 2 files changed, 35 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 2b940bd..3a1944e 100644
--- a/dump.c
+++ b/dump.c
@@ -79,6 +79,16 @@ typedef struct DumpState {
 
 uint8_t *note_buf;  /* buffer for notes */
 size_t note_buf_offset; /* the writing place in note_buf */
+uint32_t nr_cpus;   /* number of guest's cpu */
+size_t page_size;   /* guest's page size */
+uint32_t page_shift;/* guest's page shift */
+uint64_t max_mapnr; /* the biggest guest's phys-mem's number */
+size_t len_dump_bitmap; /* the size of the place used to store
+   dump_bitmap in vmcore */
+off_t offset_dump_bitmap;   /* offset of dump_bitmap part in vmcore */
+off_t offset_page;  /* offset of page part in vmcore */
+size_t num_dumpable;/* number of page that can be dumped */
+uint32_t flag_compress; /* indicate the compression format */
 } DumpState;
 
 static int dump_cleanup(DumpState *s)
@@ -796,6 +806,14 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static void get_max_mapnr(DumpState *s)
+{
+GuestPhysBlock *last_block;
+
+last_block = QTAILQ_LAST(s-guest_phys_blocks.head, GuestPhysBlockHead);
+s-max_mapnr = paddr_to_pfn(last_block-target_end, s-page_shift);
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
@@ -864,6 +882,16 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 qemu_get_guest_simple_memory_mapping(s-list, s-guest_phys_blocks);
 }
 
+s-nr_cpus = nr_cpus;
+s-page_size = TARGET_PAGE_SIZE;
+s-page_shift = ffs(s-page_size) - 1;
+
+get_max_mapnr(s);
+
+uint64_t tmp;
+tmp = DIV_ROUND_UP(DIV_ROUND_UP(s-max_mapnr, CHAR_BIT), s-page_size);
+s-len_dump_bitmap = tmp * s-page_size;
+
 if (s-has_filter) {
 memory_mapping_filter(s-list, s-begin, s-length);
 }
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b32b390..995bf47 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -20,6 +20,13 @@
 #define VERSION_FLAT_HEADER (1)/* version of flattened format */
 #define END_FLAG_FLAT_HEADER(-1)
 
+#define ARCH_PFN_OFFSET (0)
+
+#define paddr_to_pfn(X, page_shift) \
+(((unsigned long long)(X)  (page_shift)) - ARCH_PFN_OFFSET)
+#define pfn_to_paddr(X, page_shift) \
+(((unsigned long long)(X) + ARCH_PFN_OFFSET)  (page_shift))
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
-- 
1.7.1

[Qemu-devel] [PATCH v9 06/14] dump: add support for lzo/snappy

2014-02-17 Thread qiaonuohan

kdump-compressed format supports three compression format, zlib/lzo/snappy.
Currently, only zlib is available. This patch is used to support lzo/snappy.
'--enable-lzo/--enable-snappy' is needed to be specified with configure to make
lzo/snappy available for qemu

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 configure |   54 ++
 1 files changed, 54 insertions(+), 0 deletions(-)

diff --git a/configure b/configure
index 4648117..8631d9b 100755
--- a/configure
+++ b/configure
@@ -245,6 +245,8 @@ libusb=
 usb_redir=
 glx=
 zlib=yes
+lzo=no
+snappy=no
 guest_agent=
 guest_agent_with_vss=no
 vss_win32_sdk=
@@ -953,6 +955,10 @@ for opt do
   ;;
   --disable-zlib-test) zlib=no
   ;;
+  --enable-lzo) lzo=yes
+  ;;
+  --enable-snappy) snappy=yes
+  ;;
   --enable-guest-agent) guest_agent=yes
   ;;
   --disable-guest-agent) guest_agent=no
@@ -1242,6 +1248,8 @@ Advanced options (experts only):
   --enable-libusb  enable libusb (for usb passthrough)
   --disable-usb-redir  disable usb network redirection support
   --enable-usb-redir   enable usb network redirection support
+  --enable-lzo enable the support of lzo compression library
+  --enable-snappy  enable the support of snappy compression library
   --disable-guest-agentdisable building of the QEMU Guest Agent
   --enable-guest-agent enable building of the QEMU Guest Agent
   --with-vss-sdk=SDK-path  enable Windows VSS support in QEMU Guest Agent
@@ -1553,6 +1561,42 @@ fi
 libs_softmmu=$libs_softmmu -lz
 
 ##
+# lzo check
+
+if test $lzo != no ; then
+cat  $TMPC  EOF
+#include lzo/lzo1x.h
+int main(void) { lzo_version(); return 0; }
+EOF
+if compile_prog  -llzo2 ; then
+:
+else
+error_exit lzo check failed \
+Make sure to have the lzo libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -llzo2
+fi
+
+##
+# snappy check
+
+if test $snappy != no ; then
+cat  $TMPC  EOF
+#include snappy-c.h
+int main(void) { snappy_max_compressed_length(4096); return 0; }
+EOF
+if compile_prog  -lsnappy ; then
+:
+else
+error_exit snappy check failed \
+Make sure to have the snappy libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -lsnappy
+fi
+
+##
 # libseccomp check
 
 if test $seccomp != no ; then
@@ -3872,6 +3916,8 @@ echo libssh2 support   $libssh2
 echo TPM passthrough   $tpm_passthrough
 echo QOM debugging $qom_cast_debug
 echo vhdx  $vhdx
+echo lzo support   $lzo
+echo snappy support$snappy
 
 if test $sdl_too_old = yes; then
 echo - Your SDL version is too old - please upgrade to have SDL support
@@ -4187,6 +4233,14 @@ if test $glx = yes ; then
   echo GLX_LIBS=$glx_libs  $config_host_mak
 fi
 
+if test $lzo = yes ; then
+  echo CONFIG_LZO=y  $config_host_mak
+fi
+
+if test $snappy = yes ; then
+  echo CONFIG_SNAPPY=y  $config_host_mak
+fi
+
 if test $libiscsi = yes ; then
   echo CONFIG_LIBISCSI=y  $config_host_mak
   if test $libiscsi_version = 1.4.0; then
-- 
1.7.1

[Qemu-devel] [PATCH v9 12/14] dump: make kdump-compressed format available for 'dump-guest-memory'

2014-02-17 Thread qiaonuohan

Make monitor command 'dump-guest-memory' be able to dump in kdump-compressed
format. The command's usage:

  dump [-p] protocol [begin] [length] [format]

'format' is used to specified the format of vmcore and can be:
1. 'elf': ELF format, without compression
2. 'kdump-zlib': kdump-compressed format, with zlib-compressed
3. 'kdump-lzo': kdump-compressed format, with lzo-compressed
4. 'kdump-snappy': kdump-compressed format, with snappy-compressed
Without 'format' being set, it is same as 'elf'. And if non-elf format is
specified, paging and filter is not allowed.

Note:
  1. The kdump-compressed format is readable only with the crash utility and
 makedumpfile, and it can be smaller than the ELF format because of the
 compression support.
  2. The kdump-compressed format is the 6th edition.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c   |  131 +++---
 hmp.c|5 ++-
 qapi-schema.json |   25 ++-
 qmp-commands.hx  |7 ++-
 4 files changed, 158 insertions(+), 10 deletions(-)

diff --git a/dump.c b/dump.c
index a7b5117..fb0c896 100644
--- a/dump.c
+++ b/dump.c
@@ -1443,6 +1443,64 @@ out:
 return ret;
 }
 
+static int create_kdump_vmcore(DumpState *s)
+{
+int ret;
+
+/*
+ * the kdump-compressed format is:
+ *   File offset
+ *  +--+ 0x0
+ *  |main header (struct disk_dump_header) |
+ *  |--+ block 1
+ *  |sub header (struct kdump_sub_header)  |
+ *  |--+ block 2
+ *  |1st-dump_bitmap   |
+ *  |--+ block 2 + X blocks
+ *  |2nd-dump_bitmap   | (aligned by block)
+ *  |--+ block 2 + 2 * X blocks
+ *  |  page desc for pfn 0 (struct page_desc)  | (aligned by block)
+ *  |  page desc for pfn 1 (struct page_desc)  |
+ *  |: |
+ *  |--| (not aligned by block)
+ *  | page data (pfn 0)|
+ *  | page data (pfn 1)|
+ *  |: |
+ *  +--+
+ */
+
+ret = write_start_flat_header(s-fd);
+if (ret  0) {
+dump_error(s, dump: failed to write start flat header.\n);
+return -1;
+}
+
+ret = write_dump_header(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_dump_bitmap(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_dump_pages(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_end_flat_header(s-fd);
+if (ret  0) {
+dump_error(s, dump: failed to write end flat header.\n);
+return -1;
+}
+
+dump_completed(s);
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
@@ -1479,7 +1537,8 @@ static void get_max_mapnr(DumpState *s)
 s-max_mapnr = paddr_to_pfn(last_block-target_end, s-page_shift);
 }
 
-static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
+static int dump_init(DumpState *s, int fd, bool has_format,
+ DumpGuestMemoryFormat format, bool paging, bool 
has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
 CPUState *cpu;
@@ -1487,6 +1546,11 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 Error *err = NULL;
 int ret;
 
+/* kdump-compressed is conflict with paging and filter */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+assert(!paging  !has_filter);
+}
+
 if (runstate_is_running()) {
 vm_stop(RUN_STATE_SAVE_VM);
 s-resume = true;
@@ -1557,6 +1621,28 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 tmp = DIV_ROUND_UP(DIV_ROUND_UP(s-max_mapnr, CHAR_BIT), s-page_size);
 s-len_dump_bitmap = tmp * s-page_size;
 
+/* init for kdump-compressed format */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+switch (format) {
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB:
+s-flag_compress = DUMP_DH_COMPRESSED_ZLIB;
+break;
+
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO:
+s-flag_compress = DUMP_DH_COMPRESSED_LZO;
+break;
+
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY:
+s-flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
+break;
+
+default:
+s-flag_compress = 0;
+}
+
+return 0;
+}
+
 if (s-has_filter) {
 memory_mapping_filter(s-list, s-begin, s-length);
 }
@@ -1616,14 +1702,25 @@ cleanup:

[Qemu-devel] [PATCH v9 10/14] dump: add APIs to operate DataCache

2014-02-17 Thread qiaonuohan

DataCache is used to store data temporarily, then the data will be written to
vmcore. These functions will be called later when writing data of page to
vmcore.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   47 +++
 include/sysemu/dump.h |9 +
 2 files changed, 56 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 5755534..a7a85d3 100644
--- a/dump.c
+++ b/dump.c
@@ -1165,6 +1165,53 @@ out:
 return ret;
 }
 
+static void prepare_data_cache(DataCache *data_cache, DumpState *s,
+   off_t offset)
+{
+data_cache-fd = s-fd;
+data_cache-data_size = 0;
+data_cache-buf_size = BUFSIZE_DATA_CACHE;
+data_cache-buf = g_malloc0(BUFSIZE_DATA_CACHE);
+data_cache-offset = offset;
+}
+
+static int write_cache(DataCache *dc, const void *buf, size_t size,
+   bool flag_sync)
+{
+/*
+ * dc-buf_size should not be less than size, otherwise dc will never be
+ * enough
+ */
+assert(size = dc-buf_size);
+
+/*
+ * if flag_sync is set, synchronize data in dc-buf into vmcore.
+ * otherwise check if the space is enough for caching data in buf, if not,
+ * write the data in dc-buf to dc-fd and reset dc-buf
+ */
+if ((!flag_sync  dc-data_size + size  dc-buf_size) ||
+(flag_sync  dc-data_size  0)) {
+if (write_buffer(dc-fd, dc-offset, dc-buf, dc-data_size)  0) {
+return -1;
+}
+
+dc-offset += dc-data_size;
+dc-data_size = 0;
+}
+
+if (!flag_sync) {
+memcpy(dc-buf + dc-data_size, buf, size);
+dc-data_size += size;
+}
+
+return 0;
+}
+
+static void free_data_cache(DataCache *data_cache)
+{
+g_free(data_cache-buf);
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 6d4d0bc..92a95e4 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -41,6 +41,7 @@
 #define DISKDUMP_HEADER_BLOCKS  (1)
 #define BUFSIZE_BITMAP  (TARGET_PAGE_SIZE)
 #define PFN_BUFBITMAP   (CHAR_BIT * BUFSIZE_BITMAP)
+#define BUFSIZE_DATA_CACHE  (TARGET_PAGE_SIZE * 4)
 
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
@@ -142,6 +143,14 @@ typedef struct QEMU_PACKED KdumpSubHeader64 {
 uint64_t max_mapnr_64;  /* header_version 6 and later */
 } KdumpSubHeader64;
 
+typedef struct DataCache {
+int fd; /* fd of the file where to write the cached data */
+uint8_t *buf;   /* buffer for cached data */
+size_t buf_size;/* size of the buf */
+size_t data_size;   /* size of cached data in buf */
+off_t offset;   /* offset of the file */
+} DataCache;
+
 struct GuestPhysBlockList; /* memory_mapping.h */
 int cpu_get_dump_info(ArchDumpInfo *info,
   const struct GuestPhysBlockList *guest_phys_blocks);
-- 
1.7.1

[Qemu-devel] [PATCH v9 11/14] dump: add API to write dump pages

2014-02-17 Thread qiaonuohan

functions are used to write page to vmcore. vmcore is written page by page.
page desc is used to store the information of a page, including a page's size,
offset, compression format, etc.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|  231 +
 include/sysemu/dump.h |7 ++
 2 files changed, 238 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index a7a85d3..a7b5117 100644
--- a/dump.c
+++ b/dump.c
@@ -25,6 +25,14 @@
 #include qapi/error.h
 #include qmp-commands.h
 
+#include zlib.h
+#ifdef CONFIG_LZO
+#include lzo/lzo1x.h
+#endif
+#ifdef CONFIG_SNAPPY
+#include snappy-c.h
+#endif
+
 static uint16_t cpu_convert_to_target16(uint16_t val, int endian)
 {
 if (endian == ELFDATA2LSB) {
@@ -1212,6 +1220,229 @@ static void free_data_cache(DataCache *data_cache)
 g_free(data_cache-buf);
 }
 
+static size_t get_len_buf_out(size_t page_size, uint32_t flag_compress)
+{
+size_t len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
+size_t len_buf_out;
+
+/* init buf_out */
+len_buf_out_zlib = len_buf_out_lzo = len_buf_out_snappy = 0;
+
+/* buf size for zlib */
+len_buf_out_zlib = compressBound(page_size);
+
+/* buf size for lzo */
+#ifdef CONFIG_LZO
+if (flag_compress  DUMP_DH_COMPRESSED_LZO) {
+if (lzo_init() != LZO_E_OK) {
+/* return 0 to indicate lzo is unavailable */
+return 0;
+}
+}
+
+/*
+ * LZO will expand incompressible data by a little amount. please check the
+ * following URL to see the expansion calculation:
+ * http://www.oberhumer.com/opensource/lzo/lzofaq.php
+ */
+len_buf_out_lzo = page_size + page_size / 16 + 64 + 3;
+#endif
+
+#ifdef CONFIG_SNAPPY
+/* buf size for snappy */
+len_buf_out_snappy = snappy_max_compressed_length(page_size);
+#endif
+
+/* get the biggest that can store all kinds of compressed page */
+len_buf_out = MAX(len_buf_out_zlib,
+  MAX(len_buf_out_lzo, len_buf_out_snappy));
+
+return len_buf_out;
+}
+
+/*
+ * check if the page is all 0
+ */
+static inline bool is_zero_page(const uint8_t *buf, size_t page_size)
+{
+return buffer_is_zero(buf, page_size);
+}
+
+static int write_dump_pages(DumpState *s)
+{
+int ret = 0;
+DataCache page_desc, page_data;
+size_t len_buf_out, size_out;
+#ifdef CONFIG_LZO
+lzo_bytep wrkmem = NULL;
+#endif
+uint8_t *buf_out = NULL;
+off_t offset_desc, offset_data;
+PageDescriptor pd, pd_zero;
+uint8_t *buf;
+int endian = s-dump_info.d_endian;
+GuestPhysBlock *block_iter = NULL;
+uint64_t pfn_iter;
+
+/* get offset of page_desc and page_data in dump file */
+offset_desc = s-offset_page;
+offset_data = offset_desc + sizeof(PageDescriptor) * s-num_dumpable;
+
+prepare_data_cache(page_desc, s, offset_desc);
+prepare_data_cache(page_data, s, offset_data);
+
+/* prepare buffer to store compressed data */
+len_buf_out = get_len_buf_out(s-page_size, s-flag_compress);
+if (len_buf_out == 0) {
+dump_error(s, dump: failed to get length of output buffer.\n);
+goto out;
+}
+
+#ifdef CONFIG_LZO
+wrkmem = g_malloc(LZO1X_1_MEM_COMPRESS);
+#endif
+
+buf_out = g_malloc(len_buf_out);
+
+/*
+ * init zero page's page_desc and page_data, because every zero page
+ * uses the same page_data
+ */
+pd_zero.size = cpu_convert_to_target32(s-page_size, endian);
+pd_zero.flags = cpu_convert_to_target32(0, endian);
+pd_zero.offset = cpu_convert_to_target64(offset_data, endian);
+pd_zero.page_flags = cpu_convert_to_target64(0, endian);
+buf = g_malloc0(s-page_size);
+ret = write_cache(page_data, buf, s-page_size, false);
+g_free(buf);
+if (ret  0) {
+dump_error(s, dump: failed to write page data(zero page).\n);
+goto out;
+}
+
+offset_data += s-page_size;
+
+/*
+ * dump memory to vmcore page by page. zero page will all be resided in the
+ * first page of page section
+ */
+while (get_next_page(block_iter, pfn_iter, buf, s)) {
+/* check zero page */
+if (is_zero_page(buf, s-page_size)) {
+ret = write_cache(page_desc, pd_zero, sizeof(PageDescriptor),
+  false);
+if (ret  0) {
+dump_error(s, dump: failed to write page desc.\n);
+goto out;
+}
+} else {
+/*
+ * not zero page, then:
+ * 1. compress the page
+ * 2. write the compressed page into the cache of page_data
+ * 3. get page desc of the compressed page and write it into the
+ *cache of page_desc
+ *
+ * only one compression format will be used here, for
+ * s-flag_compress is set. But when compression fails to work,
+

[Qemu-devel] [PATCH v8 06/13] dump: add support for lzo/snappy

2014-01-27 Thread qiaonuohan

kdump-compressed format supports three compression format, zlib/lzo/snappy.
Currently, only zlib is available. This patch is used to support lzo/snappy.
'--enable-lzo/--enable-snappy' is needed to be specified with configure to make
lzo/snappy available for qemu

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 configure |   54 ++
 1 files changed, 54 insertions(+), 0 deletions(-)

diff --git a/configure b/configure
index b472694..d1b1005 100755
--- a/configure
+++ b/configure
@@ -245,6 +245,8 @@ libusb=
 usb_redir=
 glx=
 zlib=yes
+lzo=no
+snappy=no
 guest_agent=
 guest_agent_with_vss=no
 vss_win32_sdk=
@@ -948,6 +950,10 @@ for opt do
   ;;
   --disable-zlib-test) zlib=no
   ;;
+  --enable-lzo) lzo=yes
+  ;;
+  --enable-snappy) snappy=yes
+  ;;
   --enable-guest-agent) guest_agent=yes
   ;;
   --disable-guest-agent) guest_agent=no
@@ -1235,6 +1241,8 @@ Advanced options (experts only):
   --enable-libusb  enable libusb (for usb passthrough)
   --disable-usb-redir  disable usb network redirection support
   --enable-usb-redir   enable usb network redirection support
+  --enable-lzo enable the support of lzo compression library
+  --enable-snappy  enable the support of snappy compression library
   --disable-guest-agentdisable building of the QEMU Guest Agent
   --enable-guest-agent enable building of the QEMU Guest Agent
   --with-vss-sdk=SDK-path  enable Windows VSS support in QEMU Guest Agent
@@ -1539,6 +1547,42 @@ fi
 libs_softmmu=$libs_softmmu -lz
 
 ##
+# lzo check
+
+if test $lzo != no ; then
+cat  $TMPC  EOF
+#include lzo/lzo1x.h
+int main(void) { lzo_version(); return 0; }
+EOF
+if compile_prog  -llzo2 ; then
+:
+else
+error_exit lzo check failed \
+Make sure to have the lzo libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -llzo2
+fi
+
+##
+# snappy check
+
+if test $snappy != no ; then
+cat  $TMPC  EOF
+#include snappy-c.h
+int main(void) { snappy_max_compressed_length(4096); return 0; }
+EOF
+if compile_prog  -lsnappy ; then
+:
+else
+error_exit snappy check failed \
+Make sure to have the snappy libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -lsnappy
+fi
+
+##
 # libseccomp check
 
 if test $seccomp != no ; then
@@ -3843,6 +3887,8 @@ echo libssh2 support   $libssh2
 echo TPM passthrough   $tpm_passthrough
 echo QOM debugging $qom_cast_debug
 echo vhdx  $vhdx
+echo lzo support   $lzo
+echo snappy support$snappy
 
 if test $sdl_too_old = yes; then
 echo - Your SDL version is too old - please upgrade to have SDL support
@@ -4158,6 +4204,14 @@ if test $glx = yes ; then
   echo GLX_LIBS=$glx_libs  $config_host_mak
 fi
 
+if test $lzo = yes ; then
+  echo CONFIG_LZO=y  $config_host_mak
+fi
+
+if test $snappy = yes ; then
+  echo CONFIG_SNAPPY=y  $config_host_mak
+fi
+
 if test $libiscsi = yes ; then
   echo CONFIG_LIBISCSI=y  $config_host_mak
   if test $libiscsi_version = 1.4.0; then
-- 
1.7.1

[Qemu-devel] [PATCH v8 03/13] dump: add API to write header of flatten format

2014-01-27 Thread qiaonuohan

flatten format will be used when writing kdump-compressed format. The format is
also used by makedumpfile, you can refer to the following URL to get more
detailed information about flatten format of kdump-compressed format:
http://sourceforge.net/projects/makedumpfile/

The two functions here are used to write start flat header and end flat header
to vmcore, and they will be called later when flatten format is used.

struct MakedumpfileHeader stored at the head of vmcore is used to indicate the
vmcore is in flatten format.

struct MakedumpfileHeader {
char signature[16]; /* = makedumpfile */
int64_t type;   /* = 1 */
int64_t version;/* = 1 */
};

And struct MakedumpfileDataHeader, with offset and buf_size set to -1, is used
to indicate the end of vmcore in flatten format.

struct MakedumpfileDataHeader {
int64_t offset; /* = -1 */
int64_t buf_size;   /* = -1 */
};

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   42 ++
 include/sysemu/dump.h |   17 +
 2 files changed, 59 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index c9d3492..f233b3e 100644
--- a/dump.c
+++ b/dump.c
@@ -686,6 +686,48 @@ static int create_vmcore(DumpState *s)
 return 0;
 }
 
+static int write_start_flat_header(int fd)
+{
+uint8_t *buf;
+MakedumpfileHeader mh;
+int ret = 0;
+
+memset(mh, 0, sizeof(mh));
+strncpy(mh.signature, MAKEDUMPFILE_SIGNATURE,
+strlen(MAKEDUMPFILE_SIGNATURE));
+
+mh.type = cpu_to_be64(TYPE_FLAT_HEADER);
+mh.version = cpu_to_be64(VERSION_FLAT_HEADER);
+
+buf = g_malloc0(MAX_SIZE_MDF_HEADER);
+memcpy(buf, mh, sizeof(mh));
+
+size_t written_size;
+written_size = qemu_write_full(fd, buf, MAX_SIZE_MDF_HEADER);
+if (written_size != MAX_SIZE_MDF_HEADER) {
+ret = -1;
+}
+
+g_free(buf);
+return ret;
+}
+
+static int write_end_flat_header(int fd)
+{
+MakedumpfileDataHeader mdh;
+
+mdh.offset = END_FLAG_FLAT_HEADER;
+mdh.buf_size = END_FLAG_FLAT_HEADER;
+
+size_t written_size;
+written_size = qemu_write_full(fd, mdh, sizeof(mdh));
+if (written_size != sizeof(mdh)) {
+return -1;
+}
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 19fafb2..b32b390 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -14,12 +14,29 @@
 #ifndef DUMP_H
 #define DUMP_H
 
+#define MAKEDUMPFILE_SIGNATURE  makedumpfile
+#define MAX_SIZE_MDF_HEADER (4096) /* max size of makedumpfile_header 
*/
+#define TYPE_FLAT_HEADER(1)/* type of flattened format */
+#define VERSION_FLAT_HEADER (1)/* version of flattened format */
+#define END_FLAG_FLAT_HEADER(-1)
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
 int d_class;/* ELFCLASS32 or ELFCLASS64 */
 } ArchDumpInfo;
 
+typedef struct QEMU_PACKED MakedumpfileHeader {
+char signature[16]; /* = makedumpfile */
+int64_t type;
+int64_t version;
+} MakedumpfileHeader;
+
+typedef struct QEMU_PACKED MakedumpfileDataHeader {
+int64_t offset;
+int64_t buf_size;
+} MakedumpfileDataHeader;
+
 struct GuestPhysBlockList; /* memory_mapping.h */
 int cpu_get_dump_info(ArchDumpInfo *info,
   const struct GuestPhysBlockList *guest_phys_blocks);
-- 
1.7.1

[Qemu-devel] [PATCH v8 13/13] dump: add 'query-dump-guest-memory-capability' command

2014-01-27 Thread qiaonuohan

'query-dump-guest-memory-capability' is used to query whether option 'format'
is available for 'dump-guest-memory' and the available format. The output
of the command will be like:

- { execute: query-dump-guest-memory-capability }
- { return: {
format-option: optional,
capabilities: [
{available: true, format: elf},
{available: true, format: kdump-zlib},
{available: true, format: kdump-lzo},
{available: true, format: kdump-snappy}
]
}

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c   |   42 ++
 qapi-schema.json |   13 +
 qmp-commands.hx  |   31 +++
 3 files changed, 86 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 2ebbb23..8f64aab 100644
--- a/dump.c
+++ b/dump.c
@@ -1788,3 +1788,45 @@ void qmp_dump_guest_memory(bool paging, const char 
*file, bool has_begin,
 
 g_free(s);
 }
+
+DumpGuestMemoryCapability *qmp_query_dump_guest_memory_capability(Error **errp)
+{
+int i;
+DumpGuestMemoryCapabilityStatusList *item;
+DumpGuestMemoryCapability *cap =
+  g_malloc0(sizeof(DumpGuestMemoryCapability));
+
+cap-format_option = g_strdup_printf(optional);
+
+for (i = 0; i  DUMP_GUEST_MEMORY_FORMAT_MAX; i++) {
+if (cap-capabilities == NULL) {
+item = g_malloc0(sizeof(DumpGuestMemoryCapabilityStatusList));
+cap-capabilities = item;
+} else {
+item-next = 
g_malloc0(sizeof(DumpGuestMemoryCapabilityStatusList));
+item = item-next;
+}
+
+item-value = g_malloc0(sizeof(struct 
DumpGuestMemoryCapabilityStatus));
+item-value-format = i;
+
+if (i == DUMP_GUEST_MEMORY_FORMAT_ELF ||
+i == DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB) {
+item-value-available = true;
+}
+
+#ifdef CONFIG_LZO
+if (i == DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO) {
+item-value-available = true;
+}
+#endif
+
+#ifdef CONFIG_SNAPPY
+if (i == DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY) {
+item-value-available = true;
+}
+#endif
+}
+
+return cap;
+}
diff --git a/qapi-schema.json b/qapi-schema.json
index 7f62007..012c70c 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2783,6 +2783,19 @@
 '*length': 'int', '*format': 'DumpGuestMemoryFormat' } }
 
 ##
+# Since: 2.0
+##
+{ 'type':  'DumpGuestMemoryCapabilityStatus',
+  'data': { 'format': 'DumpGuestMemoryFormat', 'available': 'bool' } }
+
+{ 'type': 'DumpGuestMemoryCapability',
+  'data': {
+  'format-option': 'str',
+  'capabilities': ['DumpGuestMemoryCapabilityStatus'] } }
+
+{ 'command': 'query-dump-guest-memory-capability', 'returns': 
'DumpGuestMemoryCapability' }
+
+##
 # @netdev_add:
 #
 # Add a network backend.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 019dde6..e1b311a 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -829,6 +829,37 @@ Notes:
 EQMP
 
 {
+.name   = query-dump-guest-memory-capability,
+.args_type  = ,
+.mhandler.cmd_new = 
qmp_marshal_input_query_dump_guest_memory_capability,
+},
+
+SQMP
+query-dump-guest-memory-capability
+--
+
+Show whether option 'format' is available for 'dump-guest-memory' and the
+available formats.
+
+Example:
+
+- { execute: query-dump-guest-memory-capability }
+- { return: {
+format-option: optional,
+capabilities: [
+{available: true, format: elf},
+{available: true, format: kdump-zlib},
+{available: true, format: kdump-lzo},
+{available: true, format: kdump-snappy}
+]
+}
+
+Note: This is a light-weight introspection to let management know whether 
format
+  option is available and the supported compression formats.
+
+EQMP
+
+{
 .name   = netdev_add,
 .args_type  = netdev:O,
 .mhandler.cmd_new = qmp_netdev_add,
-- 
1.7.1

[Qemu-devel] [PATCH v8 02/13] dump: add argument to write_elfxx_notes

2014-01-27 Thread qiaonuohan

write_elf32_notes/wirte_elf64_notes use fd_write_vmcore to write elf notes to
vmcore. Adding parameter WriteCoreDumpFunction f makes it available to choose
the method of writing elf notes

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   16 
 1 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/dump.c b/dump.c
index 42622de..c9d3492 100644
--- a/dump.c
+++ b/dump.c
@@ -271,7 +271,7 @@ static inline int cpu_index(CPUState *cpu)
 return cpu-cpu_index + 1;
 }
 
-static int write_elf64_notes(DumpState *s)
+static int write_elf64_notes(WriteCoreDumpFunction f, DumpState *s)
 {
 CPUState *cpu;
 int ret;
@@ -279,7 +279,7 @@ static int write_elf64_notes(DumpState *s)
 
 CPU_FOREACH(cpu) {
 id = cpu_index(cpu);
-ret = cpu_write_elf64_note(fd_write_vmcore, cpu, id, s);
+ret = cpu_write_elf64_note(f, cpu, id, s);
 if (ret  0) {
 dump_error(s, dump: failed to write elf notes.\n);
 return -1;
@@ -287,7 +287,7 @@ static int write_elf64_notes(DumpState *s)
 }
 
 CPU_FOREACH(cpu) {
-ret = cpu_write_elf64_qemunote(fd_write_vmcore, cpu, s);
+ret = cpu_write_elf64_qemunote(f, cpu, s);
 if (ret  0) {
 dump_error(s, dump: failed to write CPU status.\n);
 return -1;
@@ -321,7 +321,7 @@ static int write_elf32_note(DumpState *s)
 return 0;
 }
 
-static int write_elf32_notes(DumpState *s)
+static int write_elf32_notes(WriteCoreDumpFunction f, DumpState *s)
 {
 CPUState *cpu;
 int ret;
@@ -329,7 +329,7 @@ static int write_elf32_notes(DumpState *s)
 
 CPU_FOREACH(cpu) {
 id = cpu_index(cpu);
-ret = cpu_write_elf32_note(fd_write_vmcore, cpu, id, s);
+ret = cpu_write_elf32_note(f, cpu, id, s);
 if (ret  0) {
 dump_error(s, dump: failed to write elf notes.\n);
 return -1;
@@ -337,7 +337,7 @@ static int write_elf32_notes(DumpState *s)
 }
 
 CPU_FOREACH(cpu) {
-ret = cpu_write_elf32_qemunote(fd_write_vmcore, cpu, s);
+ret = cpu_write_elf32_qemunote(f, cpu, s);
 if (ret  0) {
 dump_error(s, dump: failed to write CPU status.\n);
 return -1;
@@ -574,7 +574,7 @@ static int dump_begin(DumpState *s)
 }
 
 /* write notes to vmcore */
-if (write_elf64_notes(s)  0) {
+if (write_elf64_notes(fd_write_vmcore, s)  0) {
 return -1;
 }
 
@@ -597,7 +597,7 @@ static int dump_begin(DumpState *s)
 }
 
 /* write notes to vmcore */
-if (write_elf32_notes(s)  0) {
+if (write_elf32_notes(fd_write_vmcore, s)  0) {
 return -1;
 }
 }
-- 
1.7.1

[Qemu-devel] [PATCH v8 08/13] dump: add API to write dump header

2014-01-27 Thread qiaonuohan

the functions are used to write header of kdump-compressed format to vmcore.
Header of kdump-compressed format includes:
1. common header: DiskDumpHeader32 / DiskDumpHeader64
2. sub header: KdumpSubHeader32 / KdumpSubHeader64
3. extra information: only elf notes here

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|  223 +
 include/sysemu/dump.h |   96 +
 2 files changed, 319 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 3a1944e..4b2799f 100644
--- a/dump.c
+++ b/dump.c
@@ -778,6 +778,229 @@ static int buf_write_note(const void *buf, size_t size, 
void *opaque)
 return 0;
 }
 
+/* write common header, sub header and elf note to vmcore */
+static int create_header32(DumpState *s)
+{
+int ret = 0;
+DiskDumpHeader32 *dh = NULL;
+KdumpSubHeader32 *kh = NULL;
+size_t size;
+int endian = s-dump_info.d_endian;
+uint32_t block_size;
+uint32_t sub_hdr_size;
+uint32_t bitmap_blocks;
+uint32_t status = 0;
+uint64_t offset_note;
+
+/* write common header, the version of kdump-compressed format is 6th */
+size = sizeof(DiskDumpHeader32);
+dh = g_malloc0(size);
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = cpu_convert_to_target32(6, endian);
+block_size = s-page_size;
+dh-block_size = cpu_convert_to_target32(block_size, endian);
+sub_hdr_size = sizeof(struct KdumpSubHeader32) + s-note_size;
+sub_hdr_size = DIV_ROUND_UP(sub_hdr_size, block_size);
+dh-sub_hdr_size = cpu_convert_to_target32(sub_hdr_size, endian);
+/* dh-max_mapnr may be truncated, full 64bit is in kh.max_mapnr_64 */
+dh-max_mapnr = cpu_convert_to_target32(MIN(s-max_mapnr, UINT_MAX),
+endian);
+dh-nr_cpus = cpu_convert_to_target32(s-nr_cpus, endian);
+bitmap_blocks = DIV_ROUND_UP(s-len_dump_bitmap, block_size) * 2;
+dh-bitmap_blocks = cpu_convert_to_target32(bitmap_blocks, endian);
+memcpy((dh-utsname.machine), i686, 4);
+
+if (s-flag_compress  DUMP_DH_COMPRESSED_ZLIB) {
+status |= DUMP_DH_COMPRESSED_ZLIB;
+}
+#ifdef CONFIG_LZO
+if (s-flag_compress  DUMP_DH_COMPRESSED_LZO) {
+status |= DUMP_DH_COMPRESSED_LZO;
+}
+#endif
+#ifdef CONFIG_SNAPPY
+if (s-flag_compress  DUMP_DH_COMPRESSED_SNAPPY) {
+status |= DUMP_DH_COMPRESSED_SNAPPY;
+}
+#endif
+dh-status = cpu_convert_to_target32(status, endian);
+
+if (write_buffer(s-fd, 0, dh, size)  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+ret = -1;
+goto out;
+}
+
+/* write sub header */
+size = sizeof(KdumpSubHeader32);
+kh = g_malloc0(size);
+
+/* 64bit max_mapnr_64 */
+kh-max_mapnr_64 = cpu_convert_to_target64(s-max_mapnr, endian);
+kh-phys_base = cpu_convert_to_target32(PHYS_BASE, endian);
+kh-dump_level = cpu_convert_to_target32(DUMP_LEVEL, endian);
+
+offset_note = DISKDUMP_HEADER_BLOCKS * block_size + size;
+kh-offset_note = cpu_convert_to_target64(offset_note, endian);
+kh-note_size = cpu_convert_to_target32(s-note_size, endian);
+
+if (write_buffer(s-fd, DISKDUMP_HEADER_BLOCKS *
+ block_size, kh, size)  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+ret = -1;
+goto out;
+}
+
+/* write note */
+s-note_buf = g_malloc0(s-note_size);
+s-note_buf_offset = 0;
+
+/* use s-note_buf to store notes temporarily */
+if (write_elf32_notes(buf_write_note, s)  0) {
+ret = -1;
+goto out;
+}
+
+if (write_buffer(s-fd, offset_note, s-note_buf,
+ s-note_size)  0) {
+dump_error(s, dump: failed to write notes);
+ret = -1;
+goto out;
+}
+
+/* get offset of dump_bitmap */
+s-offset_dump_bitmap = (DISKDUMP_HEADER_BLOCKS + sub_hdr_size) *
+ block_size;
+
+/* get offset of page */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + sub_hdr_size + bitmap_blocks) *
+ block_size;
+
+out:
+g_free(dh);
+g_free(kh);
+g_free(s-note_buf);
+
+return ret;
+}
+
+/* write common header, sub header and elf note to vmcore */
+static int create_header64(DumpState *s)
+{
+int ret = 0;
+DiskDumpHeader64 *dh = NULL;
+KdumpSubHeader64 *kh = NULL;
+size_t size;
+int endian = s-dump_info.d_endian;
+uint32_t block_size;
+uint32_t sub_hdr_size;
+uint32_t bitmap_blocks;
+uint32_t status = 0;
+uint64_t offset_note;
+
+/* write common header, the version of kdump-compressed format is 6th */
+size = sizeof(DiskDumpHeader64);
+dh = g_malloc0(size);
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = cpu_convert_to_target32(6, endian);
+

[Qemu-devel] [PATCH v8 11/13] dump: add API to write dump pages

2014-01-27 Thread qiaonuohan

functions are used to write page to vmcore. vmcore is written page by page.
page desc is used to store the information of a page, including a page's size,
offset, compression format, etc.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|  231 +
 include/sysemu/dump.h |7 ++
 2 files changed, 238 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index a7a85d3..31be686 100644
--- a/dump.c
+++ b/dump.c
@@ -25,6 +25,14 @@
 #include qapi/error.h
 #include qmp-commands.h
 
+#include zlib.h
+#ifdef CONFIG_LZO
+#include lzo/lzo1x.h
+#endif
+#ifdef CONFIG_SNAPPY
+#include snappy-c.h
+#endif
+
 static uint16_t cpu_convert_to_target16(uint16_t val, int endian)
 {
 if (endian == ELFDATA2LSB) {
@@ -1212,6 +1220,229 @@ static void free_data_cache(DataCache *data_cache)
 g_free(data_cache-buf);
 }
 
+static size_t get_len_buf_out(size_t page_size, uint32_t flag_compress)
+{
+size_t len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
+size_t len_buf_out;
+
+/* init buf_out */
+len_buf_out_zlib = len_buf_out_lzo = len_buf_out_snappy = 0;
+
+/* buf size for zlib */
+len_buf_out_zlib = compressBound(page_size);
+
+/* buf size for lzo */
+#ifdef CONFIG_LZO
+if (flag_compress  DUMP_DH_COMPRESSED_LZO) {
+if (lzo_init() != LZO_E_OK) {
+/* return 0 to indicate lzo is unavailable */
+return 0;
+}
+}
+
+/*
+ * LZO will expand incompressible data by a little amount. please check the
+ * following URL to see the expansion calculation:
+ * http://www.oberhumer.com/opensource/lzo/lzofaq.php
+ */
+len_buf_out_lzo = page_size + page_size / 16 + 64 + 3;
+#endif
+
+#ifdef CONFIG_SNAPPY
+/* buf size for snappy */
+len_buf_out_snappy = snappy_max_compressed_length(page_size);
+#endif
+
+/* get the biggest that can store all kinds of compressed page */
+len_buf_out = MAX(len_buf_out_zlib,
+  MAX(len_buf_out_lzo, len_buf_out_snappy));
+
+return len_buf_out;
+}
+
+/*
+ * check if the page is all 0
+ */
+static inline bool is_zero_page(const uint8_t *buf, size_t page_size)
+{
+return buffer_is_zero(buf, page_size);
+}
+
+static int write_dump_pages(DumpState *s)
+{
+int ret = 0;
+DataCache page_desc, page_data;
+size_t len_buf_out, size_out;
+#ifdef CONFIG_LZO
+lzo_bytep wrkmem = NULL;
+#endif
+uint8_t *buf_out = NULL;
+off_t offset_desc, offset_data;
+PageDescriptor pd, pd_zero;
+uint8_t *buf;
+int endian = s-dump_info.d_endian;
+GuestPhysBlock *block_iter = NULL;
+uint64_t pfn_iter;
+
+/* get offset of page_desc and page_data in dump file */
+offset_desc = s-offset_page;
+offset_data = offset_desc + sizeof(PageDescriptor) * s-num_dumpable;
+
+prepare_data_cache(page_desc, s, offset_desc);
+prepare_data_cache(page_data, s, offset_data);
+
+/* prepare buffer to store compressed data */
+len_buf_out = get_len_buf_out(s-page_size, s-flag_compress);
+if (len_buf_out == 0) {
+dump_error(s, dump: failed to get length of output buffer.\n);
+goto out;
+}
+
+#ifdef CONFIG_LZO
+wrkmem = g_malloc(LZO1X_1_MEM_COMPRESS);
+#endif
+
+buf_out = g_malloc(len_buf_out);
+
+/*
+ * init zero page's page_desc and page_data, because every zero page
+ * uses the same page_data
+ */
+pd_zero.size = cpu_convert_to_target32(s-page_size, endian);
+pd_zero.flags = cpu_convert_to_target32(0, endian);
+pd_zero.offset = cpu_convert_to_target64(offset_data, endian);
+pd_zero.page_flags = cpu_convert_to_target64(0, endian);
+buf = g_malloc0(s-page_size);
+ret = write_cache(page_data, buf, s-page_size, false);
+g_free(buf);
+if (ret  0) {
+dump_error(s, dump: failed to write page data(zero page).\n);
+goto out;
+}
+
+offset_data += s-page_size;
+
+/*
+ * dump memory to vmcore page by page. zero page will all be resided in the
+ * first page of page section
+ */
+while (get_next_page(block_iter, pfn_iter, buf, s)) {
+/* check zero page */
+if (is_zero_page(buf, s-page_size)) {
+ret = write_cache(page_desc, pd_zero, sizeof(PageDescriptor),
+  false);
+if (ret  0) {
+dump_error(s, dump: failed to write page desc.\n);
+goto out;
+}
+} else {
+/*
+ * not zero page, then:
+ * 1. compress the page
+ * 2. write the compressed page into the cache of page_data
+ * 3. get page desc of the compressed page and write it into the
+ *cache of page_desc
+ *
+ * only one compression format will be used here, for
+ * s-flag_compress is set. But when compression fails to work,
+

[Qemu-devel] [PATCH v8 12/13] dump: make kdump-compressed format available for 'dump-guest-memory'

2014-01-27 Thread qiaonuohan

Make monitor command 'dump-guest-memory' be able to dump in kdump-compressed
format. The command's usage:

  dump [-p] protocol [begin] [length] [format]

'format' is used to specified the format of vmcore and can be:
1. 'elf': ELF format, without compression
2. 'kdump-zlib': kdump-compressed format, with zlib-compressed
3. 'kdump-lzo': kdump-compressed format, with lzo-compressed
4. 'kdump-snappy': kdump-compressed format, with snappy-compressed
Without 'format' being set, it is same as 'elf'. And if non-elf format is
specified, paging and filter is not allowed.

Note:
  1. The kdump-compressed format is readable only with the crash utility and
 makedumpfile, and it can be smaller than the ELF format because of the
 compression support.
  2. The kdump-compressed format is the 6th edition.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c   |  131 +++---
 hmp.c|5 ++-
 qapi-schema.json |   25 ++-
 qmp-commands.hx  |7 ++-
 4 files changed, 158 insertions(+), 10 deletions(-)

diff --git a/dump.c b/dump.c
index 31be686..2ebbb23 100644
--- a/dump.c
+++ b/dump.c
@@ -1443,6 +1443,64 @@ out:
 return ret;
 }
 
+static int create_kdump_vmcore(DumpState *s)
+{
+int ret;
+
+/*
+ * the kdump-compressed format is:
+ *   File offset
+ *  +--+ 0x0
+ *  |main header (struct disk_dump_header) |
+ *  |--+ block 1
+ *  |sub header (struct kdump_sub_header)  |
+ *  |--+ block 2
+ *  |1st-dump_bitmap   |
+ *  |--+ block 2 + X blocks
+ *  |2nd-dump_bitmap   | (aligned by block)
+ *  |--+ block 2 + 2 * X blocks
+ *  |  page desc for pfn 0 (struct page_desc)  | (aligned by block)
+ *  |  page desc for pfn 1 (struct page_desc)  |
+ *  |: |
+ *  |--| (not aligned by block)
+ *  | page data (pfn 0)|
+ *  | page data (pfn 1)|
+ *  |: |
+ *  +--+
+ */
+
+ret = write_start_flat_header(s-fd);
+if (ret  0) {
+dump_error(s, dump: failed to write start flat header.\n);
+return -1;
+}
+
+ret = write_dump_header(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_dump_bitmap(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_dump_pages(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_end_flat_header(s-fd);
+if (ret  0) {
+dump_error(s, dump: failed to write end flat header.\n);
+return -1;
+}
+
+dump_completed(s);
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
@@ -1479,7 +1537,8 @@ static void get_max_mapnr(DumpState *s)
 s-max_mapnr = paddr_to_pfn(last_block-target_end, s-page_shift);
 }
 
-static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
+static int dump_init(DumpState *s, int fd, bool has_format,
+ DumpGuestMemoryFormat format, bool paging, bool 
has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
 CPUState *cpu;
@@ -1487,6 +1546,11 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 Error *err = NULL;
 int ret;
 
+/* kdump-compressed is conflict with paging and filter */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+assert(!paging  !has_filter);
+}
+
 if (runstate_is_running()) {
 vm_stop(RUN_STATE_SAVE_VM);
 s-resume = true;
@@ -1557,6 +1621,28 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 tmp = DIV_ROUND_UP(DIV_ROUND_UP(s-max_mapnr, CHAR_BIT), s-page_size);
 s-len_dump_bitmap = tmp * s-page_size;
 
+/* init for kdump-compressed format */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+switch (format) {
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB:
+s-flag_compress = DUMP_DH_COMPRESSED_ZLIB;
+break;
+
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO:
+s-flag_compress = DUMP_DH_COMPRESSED_LZO;
+break;
+
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY:
+s-flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
+break;
+
+default:
+s-flag_compress = 0;
+}
+
+return 0;
+}
+
 if (s-has_filter) {
 memory_mapping_filter(s-list, s-begin, s-length);
 }
@@ -1616,14 +1702,25 @@ cleanup:
 }
 
 void qmp_dump_guest_memory(bool paging,

[Qemu-devel] [PATCH v8 04/13] dump: add API to write vmcore

2014-01-27 Thread qiaonuohan

Function is used to write vmcore in flatten format. In flatten format, data is
written block by block, and in front of each block, a struct
MakedumpfileDataHeader is stored there to indicate the offset and size of the
data block.

struct MakedumpfileDataHeader {
int64_t offset;
int64_t buf_size;
};

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   21 +
 1 files changed, 21 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index f233b3e..238ffa5 100644
--- a/dump.c
+++ b/dump.c
@@ -728,6 +728,27 @@ static int write_end_flat_header(int fd)
 return 0;
 }
 
+static int write_buffer(int fd, off_t offset, const void *buf, size_t size)
+{
+size_t written_size;
+MakedumpfileDataHeader mdh;
+
+mdh.offset = cpu_to_be64(offset);
+mdh.buf_size = cpu_to_be64(size);
+
+written_size = qemu_write_full(fd, mdh, sizeof(mdh));
+if (written_size != sizeof(mdh)) {
+return -1;
+}
+
+written_size = qemu_write_full(fd, buf, size);
+if (written_size != size) {
+return -1;
+}
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
-- 
1.7.1

[Qemu-devel] [PATCH v8 10/13] dump: add APIs to operate DataCache

2014-01-27 Thread qiaonuohan

DataCache is used to store data temporarily, then the data will be written to
vmcore. These functions will be called later when writing data of page to
vmcore.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   47 +++
 include/sysemu/dump.h |9 +
 2 files changed, 56 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 5755534..a7a85d3 100644
--- a/dump.c
+++ b/dump.c
@@ -1165,6 +1165,53 @@ out:
 return ret;
 }
 
+static void prepare_data_cache(DataCache *data_cache, DumpState *s,
+   off_t offset)
+{
+data_cache-fd = s-fd;
+data_cache-data_size = 0;
+data_cache-buf_size = BUFSIZE_DATA_CACHE;
+data_cache-buf = g_malloc0(BUFSIZE_DATA_CACHE);
+data_cache-offset = offset;
+}
+
+static int write_cache(DataCache *dc, const void *buf, size_t size,
+   bool flag_sync)
+{
+/*
+ * dc-buf_size should not be less than size, otherwise dc will never be
+ * enough
+ */
+assert(size = dc-buf_size);
+
+/*
+ * if flag_sync is set, synchronize data in dc-buf into vmcore.
+ * otherwise check if the space is enough for caching data in buf, if not,
+ * write the data in dc-buf to dc-fd and reset dc-buf
+ */
+if ((!flag_sync  dc-data_size + size  dc-buf_size) ||
+(flag_sync  dc-data_size  0)) {
+if (write_buffer(dc-fd, dc-offset, dc-buf, dc-data_size)  0) {
+return -1;
+}
+
+dc-offset += dc-data_size;
+dc-data_size = 0;
+}
+
+if (!flag_sync) {
+memcpy(dc-buf + dc-data_size, buf, size);
+dc-data_size += size;
+}
+
+return 0;
+}
+
+static void free_data_cache(DataCache *data_cache)
+{
+g_free(data_cache-buf);
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 6d4d0bc..92a95e4 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -41,6 +41,7 @@
 #define DISKDUMP_HEADER_BLOCKS  (1)
 #define BUFSIZE_BITMAP  (TARGET_PAGE_SIZE)
 #define PFN_BUFBITMAP   (CHAR_BIT * BUFSIZE_BITMAP)
+#define BUFSIZE_DATA_CACHE  (TARGET_PAGE_SIZE * 4)
 
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
@@ -142,6 +143,14 @@ typedef struct QEMU_PACKED KdumpSubHeader64 {
 uint64_t max_mapnr_64;  /* header_version 6 and later */
 } KdumpSubHeader64;
 
+typedef struct DataCache {
+int fd; /* fd of the file where to write the cached data */
+uint8_t *buf;   /* buffer for cached data */
+size_t buf_size;/* size of the buf */
+size_t data_size;   /* size of cached data in buf */
+off_t offset;   /* offset of the file */
+} DataCache;
+
 struct GuestPhysBlockList; /* memory_mapping.h */
 int cpu_get_dump_info(ArchDumpInfo *info,
   const struct GuestPhysBlockList *guest_phys_blocks);
-- 
1.7.1

[Qemu-devel] [PATCH v8 07/13] dump: add members to DumpState and init some of them

2014-01-27 Thread qiaonuohan

add some members to DumpState that will be used in writing vmcore in
kdump-compressed format. some of them, like page_size, will be initialized
in the patch.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c|   28 
 include/sysemu/dump.h |7 +++
 2 files changed, 35 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 2b940bd..3a1944e 100644
--- a/dump.c
+++ b/dump.c
@@ -79,6 +79,16 @@ typedef struct DumpState {
 
 uint8_t *note_buf;  /* buffer for notes */
 size_t note_buf_offset; /* the writing place in note_buf */
+uint32_t nr_cpus;   /* number of guest's cpu */
+size_t page_size;   /* guest's page size */
+uint32_t page_shift;/* guest's page shift */
+uint64_t max_mapnr; /* the biggest guest's phys-mem's number */
+size_t len_dump_bitmap; /* the size of the place used to store
+   dump_bitmap in vmcore */
+off_t offset_dump_bitmap;   /* offset of dump_bitmap part in vmcore */
+off_t offset_page;  /* offset of page part in vmcore */
+size_t num_dumpable;/* number of page that can be dumped */
+uint32_t flag_compress; /* indicate the compression format */
 } DumpState;
 
 static int dump_cleanup(DumpState *s)
@@ -796,6 +806,14 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static void get_max_mapnr(DumpState *s)
+{
+GuestPhysBlock *last_block;
+
+last_block = QTAILQ_LAST(s-guest_phys_blocks.head, GuestPhysBlockHead);
+s-max_mapnr = paddr_to_pfn(last_block-target_end, s-page_shift);
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
@@ -864,6 +882,16 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 qemu_get_guest_simple_memory_mapping(s-list, s-guest_phys_blocks);
 }
 
+s-nr_cpus = nr_cpus;
+s-page_size = TARGET_PAGE_SIZE;
+s-page_shift = ffs(s-page_size) - 1;
+
+get_max_mapnr(s);
+
+uint64_t tmp;
+tmp = DIV_ROUND_UP(DIV_ROUND_UP(s-max_mapnr, CHAR_BIT), s-page_size);
+s-len_dump_bitmap = tmp * s-page_size;
+
 if (s-has_filter) {
 memory_mapping_filter(s-list, s-begin, s-length);
 }
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b32b390..995bf47 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -20,6 +20,13 @@
 #define VERSION_FLAT_HEADER (1)/* version of flattened format */
 #define END_FLAG_FLAT_HEADER(-1)
 
+#define ARCH_PFN_OFFSET (0)
+
+#define paddr_to_pfn(X, page_shift) \
+(((unsigned long long)(X)  (page_shift)) - ARCH_PFN_OFFSET)
+#define pfn_to_paddr(X, page_shift) \
+(((unsigned long long)(X) + ARCH_PFN_OFFSET)  (page_shift))
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
-- 
1.7.1

[Qemu-devel] [PATCH v8 01/13] dump: const-qualify the buf of WriteCoreDumpFunction

2014-01-27 Thread qiaonuohan

WriteCoreDumpFunction is a function pointer that points to the function used to
write content in buf into core file, so buf should be const-qualify.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|2 +-
 include/qom/cpu.h |3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/dump.c b/dump.c
index 80a9116..42622de 100644
--- a/dump.c
+++ b/dump.c
@@ -99,7 +99,7 @@ static void dump_error(DumpState *s, const char *reason)
 dump_cleanup(s);
 }
 
-static int fd_write_vmcore(void *buf, size_t size, void *opaque)
+static int fd_write_vmcore(const void *buf, size_t size, void *opaque)
 {
 DumpState *s = opaque;
 size_t written_size;
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 7739e00..57b4164 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -28,7 +28,8 @@
 #include qemu/tls.h
 #include qemu/typedefs.h
 
-typedef int (*WriteCoreDumpFunction)(void *buf, size_t size, void *opaque);
+typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
+ void *opaque);
 
 /**
  * vaddr:
-- 
1.7.1

[Qemu-devel] [PATCH v8 05/13] dump: add API to write elf notes to buffer

2014-01-27 Thread qiaonuohan

the function can be used by write_elf32_notes/write_elf64_notes to write notes
to a buffer. If fd_write_vmcore is used, write_elf32_notes/write_elf64_notes
will write elf notes to vmcore directly. Instead, if buf_write_note is used,
elf notes will be written to opaque-note_buf at first.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   19 +++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 238ffa5..2b940bd 100644
--- a/dump.c
+++ b/dump.c
@@ -76,6 +76,9 @@ typedef struct DumpState {
 int64_t begin;
 int64_t length;
 Error **errp;
+
+uint8_t *note_buf;  /* buffer for notes */
+size_t note_buf_offset; /* the writing place in note_buf */
 } DumpState;
 
 static int dump_cleanup(DumpState *s)
@@ -749,6 +752,22 @@ static int write_buffer(int fd, off_t offset, const void 
*buf, size_t size)
 return 0;
 }
 
+static int buf_write_note(const void *buf, size_t size, void *opaque)
+{
+DumpState *s = opaque;
+
+/* note_buf is not enough */
+if (s-note_buf_offset + size  s-note_size) {
+return -1;
+}
+
+memcpy(s-note_buf + s-note_buf_offset, buf, size);
+
+s-note_buf_offset += size;
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
-- 
1.7.1

[Qemu-devel] [PATCH v8 09/13] dump: add API to write dump_bitmap

2014-01-27 Thread qiaonuohan

functions are used to write 1st and 2nd dump_bitmap of kdump-compressed format,
which is used to indicate whether the corresponded page is existed in vmcore.
1st and 2nd dump_bitmap are same, because dump level is specified to 1 here.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c|  164 +
 include/sysemu/dump.h |2 +
 2 files changed, 166 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 4b2799f..5755534 100644
--- a/dump.c
+++ b/dump.c
@@ -1001,6 +1001,170 @@ static int write_dump_header(DumpState *s)
 }
 }
 
+/*
+ * set dump_bitmap sequencely. the bit before last_pfn is not allowed to be
+ * rewritten, so if need to set the first bit, set last_pfn and pfn to 0.
+ * set_dump_bitmap will always leave the recently set bit un-sync. And setting
+ * (last bit + sizeof(buf) * 8) to 0 will do flushing the content in buf into
+ * vmcore, ie. synchronizing un-sync bit into vmcore.
+ */
+static int set_dump_bitmap(uint64_t last_pfn, uint64_t pfn, bool value,
+   uint8_t *buf, DumpState *s)
+{
+off_t old_offset, new_offset;
+off_t offset_bitmap1, offset_bitmap2;
+uint32_t byte, bit;
+
+/* should not set the previous place */
+assert(last_pfn = pfn);
+
+/*
+ * if the bit needed to be set is not cached in buf, flush the data in buf
+ * to vmcore firstly.
+ * making new_offset be bigger than old_offset can also sync remained data
+ * into vmcore.
+ */
+old_offset = BUFSIZE_BITMAP * (last_pfn / PFN_BUFBITMAP);
+new_offset = BUFSIZE_BITMAP * (pfn / PFN_BUFBITMAP);
+
+while (old_offset  new_offset) {
+/* calculate the offset and write dump_bitmap */
+offset_bitmap1 = s-offset_dump_bitmap + old_offset;
+if (write_buffer(s-fd, offset_bitmap1, buf,
+ BUFSIZE_BITMAP)  0) {
+return -1;
+}
+
+/* dump level 1 is chosen, so 1st and 2nd bitmap are same */
+offset_bitmap2 = s-offset_dump_bitmap + s-len_dump_bitmap +
+ old_offset;
+if (write_buffer(s-fd, offset_bitmap2, buf,
+ BUFSIZE_BITMAP)  0) {
+return -1;
+}
+
+memset(buf, 0, BUFSIZE_BITMAP);
+old_offset += BUFSIZE_BITMAP;
+}
+
+/* get the exact place of the bit in the buf, and set it */
+byte = (pfn % PFN_BUFBITMAP) / CHAR_BIT;
+bit = (pfn % PFN_BUFBITMAP) % CHAR_BIT;
+if (value) {
+buf[byte] |= 1u  bit;
+} else {
+buf[byte] = ~(1u  bit);
+}
+
+return 0;
+}
+
+/*
+ * exam every page and return the page frame number and the address of the 
page.
+ * bufptr can be NULL. note: the blocks here is supposed to reflect guest-phys
+ * blocks, so block-target_start and block-target_end should be interal
+ * multiples of the target page size.
+ */
+static bool get_next_page(GuestPhysBlock **blockptr, uint64_t *pfnptr,
+  uint8_t **bufptr, DumpState *s)
+{
+GuestPhysBlock *block = *blockptr;
+hwaddr addr;
+uint8_t *buf;
+
+/* block == NULL means the start of the iteration */
+if (!block) {
+block = QTAILQ_FIRST(s-guest_phys_blocks.head);
+*blockptr = block;
+assert(block-target_start % s-page_size == 0);
+assert(block-target_end % s-page_size == 0);
+*pfnptr = paddr_to_pfn(block-target_start, s-page_shift);
+if (bufptr) {
+*bufptr = block-host_addr;
+}
+return true;
+}
+
+*pfnptr = *pfnptr + 1;
+addr = pfn_to_paddr(*pfnptr, s-page_shift);
+
+if ((addr = block-target_start) 
+(addr + s-page_size = block-target_end)) {
+buf = block-host_addr + (addr - block-target_start);
+} else {
+/* the next page is in the next block */
+block = QTAILQ_NEXT(block, next);
+*blockptr = block;
+if (!block) {
+return false;
+}
+assert(block-target_start % s-page_size == 0);
+assert(block-target_end % s-page_size == 0);
+*pfnptr = paddr_to_pfn(block-target_start, s-page_shift);
+buf = block-host_addr;
+}
+
+if (bufptr) {
+*bufptr = buf;
+}
+
+return true;
+}
+
+static int write_dump_bitmap(DumpState *s)
+{
+int ret = 0;
+uint64_t last_pfn, pfn;
+void *dump_bitmap_buf;
+size_t num_dumpable;
+GuestPhysBlock *block_iter = NULL;
+
+/* dump_bitmap_buf is used to store dump_bitmap temporarily */
+dump_bitmap_buf = g_malloc0(BUFSIZE_BITMAP);
+
+num_dumpable = 0;
+last_pfn = 0;
+
+/*
+ * exam memory page by page, and set the bit in dump_bitmap corresponded
+ * to the existing page.
+ */
+while (get_next_page(block_iter, pfn, NULL, s)) {
+ret = set_dump_bitmap(last_pfn, pfn, true, dump_bitmap_buf, s);
+if (ret  0) {
+dump_error(s, dump: failed to set dump_bitmap.\n);
+

[Qemu-devel] [PATCH 10/13 v7] dump: add APIs to operate DataCache

2014-01-17 Thread qiaonuohan

DataCache is used to store data temporarily, then the data will be written to
vmcore. These functions will be called later when writing data of page to
vmcore.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   47 +++
 include/sysemu/dump.h |9 +
 2 files changed, 56 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 26a1756..fa183cb 100644
--- a/dump.c
+++ b/dump.c
@@ -1173,6 +1173,53 @@ out:
 return ret;
 }
 
+static void prepare_data_cache(DataCache *data_cache, DumpState *s,
+   off_t offset)
+{
+data_cache-fd = s-fd;
+data_cache-data_size = 0;
+data_cache-buf_size = BUFSIZE_DATA_CACHE;
+data_cache-buf = g_malloc0(BUFSIZE_DATA_CACHE);
+data_cache-offset = offset;
+}
+
+static int write_cache(DataCache *dc, const void *buf, size_t size,
+   bool flag_sync)
+{
+/*
+ * dc-buf_size should not be less than size, otherwise dc will never be
+ * enough
+ */
+assert(size = dc-buf_size);
+
+/*
+ * if flag_sync is set, synchronize data in dc-buf into vmcore.
+ * otherwise check if the space is enough for caching data in buf, if not,
+ * write the data in dc-buf to dc-fd and reset dc-buf
+ */
+if ((!flag_sync  dc-data_size + size  dc-buf_size) ||
+(flag_sync  dc-data_size  0)) {
+if (write_buffer(dc-fd, dc-offset, dc-buf, dc-data_size)  0) {
+return -1;
+}
+
+dc-offset += dc-data_size;
+dc-data_size = 0;
+}
+
+if (!flag_sync) {
+memcpy(dc-buf + dc-data_size, buf, size);
+dc-data_size += size;
+}
+
+return 0;
+}
+
+static void free_data_cache(DataCache *data_cache)
+{
+g_free(data_cache-buf);
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 6d4d0bc..92a95e4 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -41,6 +41,7 @@
 #define DISKDUMP_HEADER_BLOCKS  (1)
 #define BUFSIZE_BITMAP  (TARGET_PAGE_SIZE)
 #define PFN_BUFBITMAP   (CHAR_BIT * BUFSIZE_BITMAP)
+#define BUFSIZE_DATA_CACHE  (TARGET_PAGE_SIZE * 4)
 
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
@@ -142,6 +143,14 @@ typedef struct QEMU_PACKED KdumpSubHeader64 {
 uint64_t max_mapnr_64;  /* header_version 6 and later */
 } KdumpSubHeader64;
 
+typedef struct DataCache {
+int fd; /* fd of the file where to write the cached data */
+uint8_t *buf;   /* buffer for cached data */
+size_t buf_size;/* size of the buf */
+size_t data_size;   /* size of cached data in buf */
+off_t offset;   /* offset of the file */
+} DataCache;
+
 struct GuestPhysBlockList; /* memory_mapping.h */
 int cpu_get_dump_info(ArchDumpInfo *info,
   const struct GuestPhysBlockList *guest_phys_blocks);
-- 
1.7.1

[Qemu-devel] [PATCH 11/13 v7] dump: add API to write dump pages

2014-01-17 Thread qiaonuohan

functions are used to write page to vmcore. vmcore is written page by page.
page desc is used to store the information of a page, including a page's size,
offset, compression format, etc.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c|  229 +
 include/sysemu/dump.h |7 ++
 2 files changed, 236 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index fa183cb..bb03ef7 100644
--- a/dump.c
+++ b/dump.c
@@ -25,6 +25,14 @@
 #include qapi/error.h
 #include qmp-commands.h
 
+#include zlib.h
+#ifdef CONFIG_LZO
+#include lzo/lzo1x.h
+#endif
+#ifdef CONFIG_SNAPPY
+#include snappy-c.h
+#endif
+
 static uint16_t cpu_convert_to_target16(uint16_t val, int endian)
 {
 if (endian == ELFDATA2LSB) {
@@ -1220,6 +1228,227 @@ static void free_data_cache(DataCache *data_cache)
 g_free(data_cache-buf);
 }
 
+static size_t get_len_buf_out(size_t page_size, uint32_t flag_compress)
+{
+size_t len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
+size_t len_buf_out;
+
+/* init buf_out */
+len_buf_out_zlib = len_buf_out_lzo = len_buf_out_snappy = 0;
+
+/* buf size for zlib */
+len_buf_out_zlib = compressBound(page_size);
+
+/* buf size for lzo */
+#ifdef CONFIG_LZO
+if (flag_compress  DUMP_DH_COMPRESSED_LZO) {
+if (lzo_init() != LZO_E_OK) {
+/* return 0 to indicate lzo is unavailable */
+return 0;
+}
+}
+
+/*
+ * LZO will expand incompressible data by a little amount. please check the
+ * following URL to see the expansion calculation:
+ * http://www.oberhumer.com/opensource/lzo/lzofaq.php
+ */
+len_buf_out_lzo = page_size + page_size / 16 + 64 + 3;
+#endif
+
+#ifdef CONFIG_SNAPPY
+/* buf size for snappy */
+len_buf_out_snappy = snappy_max_compressed_length(page_size);
+#endif
+
+/* get the biggest that can store all kinds of compressed page */
+len_buf_out = MAX(len_buf_out_zlib,
+  MAX(len_buf_out_lzo, len_buf_out_snappy));
+
+return len_buf_out;
+}
+
+/*
+ * check if the page is all 0
+ */
+static inline bool is_zero_page(const uint8_t *buf, size_t page_size)
+{
+return buffer_is_zero(buf, page_size);
+}
+
+static int write_dump_pages(DumpState *s)
+{
+int ret = 0;
+DataCache page_desc, page_data;
+size_t len_buf_out, size_out;
+#ifdef CONFIG_LZO
+lzo_bytep wrkmem = NULL;
+#endif
+uint8_t *buf_out = NULL;
+off_t offset_desc, offset_data;
+PageDesc pd, pd_zero;
+uint8_t *buf;
+int endian = s-dump_info.d_endian;
+
+/* get offset of page_desc and page_data in dump file */
+offset_desc = s-offset_page;
+offset_data = offset_desc + sizeof(PageDesc) * s-num_dumpable;
+
+prepare_data_cache(page_desc, s, offset_desc);
+prepare_data_cache(page_data, s, offset_data);
+
+/* prepare buffer to store compressed data */
+len_buf_out = get_len_buf_out(s-page_size, s-flag_compress);
+if (len_buf_out == 0) {
+dump_error(s, dump: failed to get length of output buffer.\n);
+goto out;
+}
+
+#ifdef CONFIG_LZO
+wrkmem = g_malloc(LZO1X_1_MEM_COMPRESS);
+#endif
+
+buf_out = g_malloc(len_buf_out);
+
+/*
+ * init zero page's page_desc and page_data, because every zero page
+ * uses the same page_data
+ */
+pd_zero.size = cpu_convert_to_target32(s-page_size, endian);
+pd_zero.flags = cpu_convert_to_target32(0, endian);
+pd_zero.offset = cpu_convert_to_target64(offset_data, endian);
+pd_zero.page_flags = cpu_convert_to_target64(0, endian);
+buf = g_malloc0(s-page_size);
+ret = write_cache(page_data, buf, s-page_size, false);
+g_free(buf);
+if (ret  0) {
+dump_error(s, dump: failed to write page data(zero page).\n);
+goto out;
+}
+
+offset_data += s-page_size;
+
+/*
+ * dump memory to vmcore page by page. zero page will all be resided in the
+ * first page of page section
+ */
+while (get_next_page(NULL, buf, s)) {
+/* check zero page */
+if (is_zero_page(buf, s-page_size)) {
+ret = write_cache(page_desc, pd_zero, sizeof(PageDesc),
+  false);
+if (ret  0) {
+dump_error(s, dump: failed to write page desc.\n);
+goto out;
+}
+} else {
+/*
+ * not zero page, then:
+ * 1. compress the page
+ * 2. write the compressed page into the cache of page_data
+ * 3. get page desc of the compressed page and write it into the
+ *cache of page_desc
+ *
+ * only one compression format will be used here, for
+ * s-flag_compress is set. But when compression fails to work,
+ * we fall back to save in plaintext.
+ */
+ size_out = len_buf_out;
+ if ((s-flag_compress

[Qemu-devel] [PATCH 00/13 v7] Make 'dump-guest-memory' dump in kdump-compressed format

2014-01-17 Thread qiaonuohan

Hi, all

The last version is here:
http://lists.nongnu.org/archive/html/qemu-devel/2014-01/msg00209.html

Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

Compared with migration, the missing of compression feature means regression
to 'dump-guest-memory'. So we post these patches to make 'dump-guest-memory' be
able to dump guest's in kdump-compressed format. Then vmcore can be much
smaller, and easily to be delivered.

The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support. To get more detailed information about
kdump-compressed format, please refer to the following URL:
http://sourceforge.net/projects/makedumpfile/

Note, similar to 'dump-guest-memory':
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
   available by default, and option '--enable-lzo' or '--enable-snappy'
   should be specified with 'configure' to make lzo or snappy available.

Changelog:
Changes from v6 to v7:
1. support BE host
2. abandon non-flatten format to avoid using seek on vmcore
3. abandon using of very large array
4. use get_next_page to replace the iteration of guest's pages
5. abandon the support of HMP

Changes from v5 to v6:
1. add run-time check for compression format(lzo/snappy)
2. address Stefan's comments about reusing code and coding style
3. update the version of kdump-compressed format to 6th
4. resplit the patches
5. Add 'query-dump-guest-memory-capability' command

Changes from v4 to v5:
1. using flatten format to avoid using temporary files according to Stefan's
   comments
2. Address Andreas's comments about coding style

Changes from v3 to v4:
1. change to avoid conflict with Andreas's patches
2. rebase

Changes from v2 to v3:
1. Address Eric's comment

Changes from v1 to v2:
1. Address Eric  Daniel's comment: fix manner of string copy.
2. Address Eric's comment: replace reinventing new constants by using the
   ready-made ones accoring.
3. Address Andreas's comment: remove useless include.

qiaonuohan (13):
  dump: const-qualify the buf of WriteCoreDumpFunction
  dump: add argument to write_elfxx_notes
  dump: add API to write header of flatten format
  dump: add API to write vmcore
  dump: add API to write elf notes to buffer
  dump: add support for lzo/snappy
  dump: add members to DumpState and init some of them
  dump: add API to write dump header
  dump: add API to write dump_bitmap
  dump: add APIs to operate DataCache
  dump: add API to write dump pages
  dump: make kdump-compressed format available for 'dump-guest-memory'
  dump: add 'query-dump-guest-memory-capability' command

 configure |   54 +++
 dump.c|  972 -
 hmp.c |5 +-
 include/qom/cpu.h |3 +-
 include/sysemu/dump.h |  138 +++
 qapi-schema.json  |   38 ++-
 qmp-commands.hx   |   38 ++-
 7 files changed, 1228 insertions(+), 20 deletions(-)

[Qemu-devel] [PATCH 01/13 v7] dump: const-qualify the buf of WriteCoreDumpFunction

2014-01-16 Thread qiaonuohan

WriteCoreDumpFunction is a function pointer that points to the function used to
write content in buf into core file, so buf should be const-qualify.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c|2 +-
 include/qom/cpu.h |3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/dump.c b/dump.c
index 80a9116..42622de 100644
--- a/dump.c
+++ b/dump.c
@@ -99,7 +99,7 @@ static void dump_error(DumpState *s, const char *reason)
 dump_cleanup(s);
 }
 
-static int fd_write_vmcore(void *buf, size_t size, void *opaque)
+static int fd_write_vmcore(const void *buf, size_t size, void *opaque)
 {
 DumpState *s = opaque;
 size_t written_size;
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 7739e00..57b4164 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -28,7 +28,8 @@
 #include qemu/tls.h
 #include qemu/typedefs.h
 
-typedef int (*WriteCoreDumpFunction)(void *buf, size_t size, void *opaque);
+typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
+ void *opaque);
 
 /**
  * vaddr:
-- 
1.7.1

[Qemu-devel] [PATCH 07/13 v7] dump: add members to DumpState and init some of them

2014-01-16 Thread qiaonuohan

add some members to DumpState that will be used in writing vmcore in
kdump-compressed format. some of them, like page_size, will be initialized
in the patch.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c|   30 ++
 include/sysemu/dump.h |7 +++
 2 files changed, 37 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 2b940bd..bf7d31d 100644
--- a/dump.c
+++ b/dump.c
@@ -79,6 +79,16 @@ typedef struct DumpState {
 
 uint8_t *note_buf;  /* buffer for notes */
 size_t note_buf_offset; /* the writing place in note_buf */
+uint32_t nr_cpus;   /* number of guest's cpu */
+size_t page_size;   /* guest's page size */
+uint32_t page_shift;/* guest's page shift */
+uint64_t max_mapnr; /* the biggest guest's phys-mem's number */
+size_t len_dump_bitmap; /* the size of the place used to store
+   dump_bitmap in vmcore */
+off_t offset_dump_bitmap;   /* offset of dump_bitmap part in vmcore */
+off_t offset_page;  /* offset of page part in vmcore */
+size_t num_dumpable;/* number of page that can be dumped */
+uint32_t flag_compress; /* indicate the compression format */
 } DumpState;
 
 static int dump_cleanup(DumpState *s)
@@ -796,6 +806,16 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static void get_max_mapnr(DumpState *s)
+{
+MemoryMapping *memory_mapping;
+
+QTAILQ_FOREACH(memory_mapping, s-list.head, next) {
+s-max_mapnr = paddr_to_pfn(memory_mapping-phys_addr +
+memory_mapping-length, s-page_shift);
+}
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
@@ -864,6 +884,16 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 qemu_get_guest_simple_memory_mapping(s-list, s-guest_phys_blocks);
 }
 
+s-nr_cpus = nr_cpus;
+s-page_size = TARGET_PAGE_SIZE;
+s-page_shift = ffs(s-page_size) - 1;
+
+get_max_mapnr(s);
+
+uint64_t tmp;
+tmp = DIV_ROUND_UP(DIV_ROUND_UP(s-max_mapnr, CHAR_BIT), s-page_size);
+s-len_dump_bitmap = tmp * s-page_size;
+
 if (s-has_filter) {
 memory_mapping_filter(s-list, s-begin, s-length);
 }
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b32b390..995bf47 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -20,6 +20,13 @@
 #define VERSION_FLAT_HEADER (1)/* version of flattened format */
 #define END_FLAG_FLAT_HEADER(-1)
 
+#define ARCH_PFN_OFFSET (0)
+
+#define paddr_to_pfn(X, page_shift) \
+(((unsigned long long)(X)  (page_shift)) - ARCH_PFN_OFFSET)
+#define pfn_to_paddr(X, page_shift) \
+(((unsigned long long)(X) + ARCH_PFN_OFFSET)  (page_shift))
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
-- 
1.7.1

[Qemu-devel] [PATCH 06/13 v7] dump: add support for lzo/snappy

2014-01-16 Thread qiaonuohan

kdump-compressed format supports three compression format, zlib/lzo/snappy.
Currently, only zlib is available. This patch is used to support lzo/snappy.
'--enable-lzo/--enable-snappy' is needed to be specified with configure to make
lzo/snappy available for qemu

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 configure |   54 ++
 1 files changed, 54 insertions(+), 0 deletions(-)

diff --git a/configure b/configure
index 3782a6a..0bd0321 100755
--- a/configure
+++ b/configure
@@ -245,6 +245,8 @@ libusb=
 usb_redir=
 glx=
 zlib=yes
+lzo=no
+snappy=no
 guest_agent=
 guest_agent_with_vss=no
 vss_win32_sdk=
@@ -947,6 +949,10 @@ for opt do
   ;;
   --disable-zlib-test) zlib=no
   ;;
+  --enable-lzo) lzo=yes
+  ;;
+  --enable-snappy) snappy=yes
+  ;;
   --enable-guest-agent) guest_agent=yes
   ;;
   --disable-guest-agent) guest_agent=no
@@ -1234,6 +1240,8 @@ Advanced options (experts only):
   --enable-libusb  enable libusb (for usb passthrough)
   --disable-usb-redir  disable usb network redirection support
   --enable-usb-redir   enable usb network redirection support
+  --enable-lzo enable the support of lzo compression library
+  --enable-snappy  enable the support of snappy compression library
   --disable-guest-agentdisable building of the QEMU Guest Agent
   --enable-guest-agent enable building of the QEMU Guest Agent
   --with-vss-sdk=SDK-path  enable Windows VSS support in QEMU Guest Agent
@@ -1538,6 +1546,42 @@ fi
 libs_softmmu=$libs_softmmu -lz
 
 ##
+# lzo check
+
+if test $lzo != no ; then
+cat  $TMPC  EOF
+#include lzo/lzo1x.h
+int main(void) { lzo_version(); return 0; }
+EOF
+if compile_prog  -llzo2 ; then
+:
+else
+error_exit lzo check failed \
+Make sure to have the lzo libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -llzo2
+fi
+
+##
+# snappy check
+
+if test $snappy != no ; then
+cat  $TMPC  EOF
+#include snappy-c.h
+int main(void) { snappy_max_compressed_length(4096); return 0; }
+EOF
+if compile_prog  -lsnappy ; then
+:
+else
+error_exit snappy check failed \
+Make sure to have the snappy libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -lsnappy
+fi
+
+##
 # libseccomp check
 
 if test $seccomp != no ; then
@@ -3839,6 +3883,8 @@ echo libssh2 support   $libssh2
 echo TPM passthrough   $tpm_passthrough
 echo QOM debugging $qom_cast_debug
 echo vhdx  $vhdx
+echo lzo support   $lzo
+echo snappy support$snappy
 
 if test $sdl_too_old = yes; then
 echo - Your SDL version is too old - please upgrade to have SDL support
@@ -4154,6 +4200,14 @@ if test $glx = yes ; then
   echo GLX_LIBS=$glx_libs  $config_host_mak
 fi
 
+if test $lzo = yes ; then
+  echo CONFIG_LZO=y  $config_host_mak
+fi
+
+if test $snappy = yes ; then
+  echo CONFIG_SNAPPY=y  $config_host_mak
+fi
+
 if test $libiscsi = yes ; then
   echo CONFIG_LIBISCSI=y  $config_host_mak
   if test $libiscsi_version = 1.4.0; then
-- 
1.7.1

[Qemu-devel] [PATCH 05/13 v7] dump: add API to write elf notes to buffer

2014-01-16 Thread qiaonuohan

the function can be used by write_elf32_notes/write_elf64_notes to write notes
to a buffer. If fd_write_vmcore is used, write_elf32_notes/write_elf64_notes
will write elf notes to vmcore directly. Instead, if buf_write_note is used,
elf notes will be written to opaque-note_buf at first.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   19 +++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 238ffa5..2b940bd 100644
--- a/dump.c
+++ b/dump.c
@@ -76,6 +76,9 @@ typedef struct DumpState {
 int64_t begin;
 int64_t length;
 Error **errp;
+
+uint8_t *note_buf;  /* buffer for notes */
+size_t note_buf_offset; /* the writing place in note_buf */
 } DumpState;
 
 static int dump_cleanup(DumpState *s)
@@ -749,6 +752,22 @@ static int write_buffer(int fd, off_t offset, const void 
*buf, size_t size)
 return 0;
 }
 
+static int buf_write_note(const void *buf, size_t size, void *opaque)
+{
+DumpState *s = opaque;
+
+/* note_buf is not enough */
+if (s-note_buf_offset + size  s-note_size) {
+return -1;
+}
+
+memcpy(s-note_buf + s-note_buf_offset, buf, size);
+
+s-note_buf_offset += size;
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
-- 
1.7.1

[Qemu-devel] [PATCH 13/13 v7] dump: add 'query-dump-guest-memory-capability' command

2014-01-16 Thread qiaonuohan

'query-dump-guest-memory-capability' is used to query whether option 'format'
is available for 'dump-guest-memory' and the available format. The output
of the command will be like:

- { execute: query-dump-guest-memory-capability }
- { return: {
format-option: optional,
capabilities: [
{available: true, format: elf},
{available: true, format: kdump-zlib},
{available: true, format: kdump-lzo},
{available: true, format: kdump-snappy}
]
}

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c   |   42 ++
 qapi-schema.json |   13 +
 qmp-commands.hx  |   31 +++
 3 files changed, 86 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index dbf4bb6..c288cd3 100644
--- a/dump.c
+++ b/dump.c
@@ -1794,3 +1794,45 @@ void qmp_dump_guest_memory(bool paging, const char 
*file, bool has_begin,
 
 g_free(s);
 }
+
+DumpGuestMemoryCapability *qmp_query_dump_guest_memory_capability(Error **errp)
+{
+int i;
+DumpGuestMemoryCapabilityStatusList *item;
+DumpGuestMemoryCapability *cap =
+  g_malloc0(sizeof(DumpGuestMemoryCapability));
+
+cap-format_option = g_strdup_printf(optional);
+
+for (i = 0; i  DUMP_GUEST_MEMORY_FORMAT_MAX; i++) {
+if (cap-capabilities == NULL) {
+item = g_malloc0(sizeof(DumpGuestMemoryCapabilityStatusList));
+cap-capabilities = item;
+} else {
+item-next = 
g_malloc0(sizeof(DumpGuestMemoryCapabilityStatusList));
+item = item-next;
+}
+
+item-value = g_malloc0(sizeof(struct 
DumpGuestMemoryCapabilityStatus));
+item-value-format = i;
+
+if (i == DUMP_GUEST_MEMORY_FORMAT_ELF || i ==
+DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB) {
+item-value-available = true;
+}
+
+#ifdef CONFIG_LZO
+if (i == DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO) {
+item-value-available = true;
+}
+#endif
+
+#ifdef CONFIG_SNAPPY
+if (i == DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY) {
+item-value-available = true;
+}
+#endif
+}
+
+return cap;
+}
diff --git a/qapi-schema.json b/qapi-schema.json
index 52df894..3085294 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2744,6 +2744,19 @@
 '*length': 'int', '*format': 'DumpGuestMemoryFormat' } }
 
 ##
+# Since: 2.0
+##
+{ 'type':  'DumpGuestMemoryCapabilityStatus',
+  'data': { 'format': 'DumpGuestMemoryFormat', 'available': 'bool' } }
+
+{ 'type': 'DumpGuestMemoryCapability',
+  'data': {
+  'format-option': 'str',
+  'capabilities': ['DumpGuestMemoryCapabilityStatus'] } }
+
+{ 'command': 'query-dump-guest-memory-capability', 'returns': 
'DumpGuestMemoryCapability' }
+
+##
 # @netdev_add:
 #
 # Add a network backend.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 9158f22..ad51de0 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -829,6 +829,37 @@ Notes:
 EQMP
 
 {
+.name   = query-dump-guest-memory-capability,
+.args_type  = ,
+.mhandler.cmd_new = 
qmp_marshal_input_query_dump_guest_memory_capability,
+},
+
+SQMP
+query-dump-guest-memory-capability
+--
+
+Show whether option 'format' is available for 'dump-guest-memory' and the
+available format.
+
+Example:
+
+- { execute: query-dump-guest-memory-capability }
+- { return: {
+format-option: optional,
+capabilities: [
+{available: true, format: elf},
+{available: true, format: kdump-zlib},
+{available: true, format: kdump-lzo},
+{available: true, format: kdump-snappy}
+]
+}
+
+Note: This is a light-weight introspection to let management know whether 
format
+  option is available and the supported compression format.
+
+EQMP
+
+{
 .name   = netdev_add,
 .args_type  = netdev:O,
 .mhandler.cmd_new = qmp_netdev_add,
-- 
1.7.1

[Qemu-devel] [PATCH 02/13 v7] dump: add argument to write_elfxx_notes

2014-01-16 Thread qiaonuohan

write_elf32_notes/wirte_elf64_notes use fd_write_vmcore to write elf notes to
vmcore. Adding parameter WriteCoreDumpFunction f makes it available to choose
the method of writing elf notes

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   16 
 1 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/dump.c b/dump.c
index 42622de..c9d3492 100644
--- a/dump.c
+++ b/dump.c
@@ -271,7 +271,7 @@ static inline int cpu_index(CPUState *cpu)
 return cpu-cpu_index + 1;
 }
 
-static int write_elf64_notes(DumpState *s)
+static int write_elf64_notes(WriteCoreDumpFunction f, DumpState *s)
 {
 CPUState *cpu;
 int ret;
@@ -279,7 +279,7 @@ static int write_elf64_notes(DumpState *s)
 
 CPU_FOREACH(cpu) {
 id = cpu_index(cpu);
-ret = cpu_write_elf64_note(fd_write_vmcore, cpu, id, s);
+ret = cpu_write_elf64_note(f, cpu, id, s);
 if (ret  0) {
 dump_error(s, dump: failed to write elf notes.\n);
 return -1;
@@ -287,7 +287,7 @@ static int write_elf64_notes(DumpState *s)
 }
 
 CPU_FOREACH(cpu) {
-ret = cpu_write_elf64_qemunote(fd_write_vmcore, cpu, s);
+ret = cpu_write_elf64_qemunote(f, cpu, s);
 if (ret  0) {
 dump_error(s, dump: failed to write CPU status.\n);
 return -1;
@@ -321,7 +321,7 @@ static int write_elf32_note(DumpState *s)
 return 0;
 }
 
-static int write_elf32_notes(DumpState *s)
+static int write_elf32_notes(WriteCoreDumpFunction f, DumpState *s)
 {
 CPUState *cpu;
 int ret;
@@ -329,7 +329,7 @@ static int write_elf32_notes(DumpState *s)
 
 CPU_FOREACH(cpu) {
 id = cpu_index(cpu);
-ret = cpu_write_elf32_note(fd_write_vmcore, cpu, id, s);
+ret = cpu_write_elf32_note(f, cpu, id, s);
 if (ret  0) {
 dump_error(s, dump: failed to write elf notes.\n);
 return -1;
@@ -337,7 +337,7 @@ static int write_elf32_notes(DumpState *s)
 }
 
 CPU_FOREACH(cpu) {
-ret = cpu_write_elf32_qemunote(fd_write_vmcore, cpu, s);
+ret = cpu_write_elf32_qemunote(f, cpu, s);
 if (ret  0) {
 dump_error(s, dump: failed to write CPU status.\n);
 return -1;
@@ -574,7 +574,7 @@ static int dump_begin(DumpState *s)
 }
 
 /* write notes to vmcore */
-if (write_elf64_notes(s)  0) {
+if (write_elf64_notes(fd_write_vmcore, s)  0) {
 return -1;
 }
 
@@ -597,7 +597,7 @@ static int dump_begin(DumpState *s)
 }
 
 /* write notes to vmcore */
-if (write_elf32_notes(s)  0) {
+if (write_elf32_notes(fd_write_vmcore, s)  0) {
 return -1;
 }
 }
-- 
1.7.1

[Qemu-devel] [PATCH 03/13 v7] dump: add API to write header of flatten format

2014-01-16 Thread qiaonuohan

flatten format will be used when writing kdump-compressed format. The format is
also used by makedumpfile, you can refer to the following URL to get more
detailed information about flatten format of kdump-compressed format:
http://sourceforge.net/projects/makedumpfile/

The two functions here are used to write start flat header and end flat header
to vmcore, and they will be called later when flatten format is used.

struct MakedumpfileHeader stored at the head of vmcore is used to indicate the
vmcore is in flatten format.

struct MakedumpfileHeader {
char signature[16]; /* = makedumpfile */
int64_t type;   /* = 1 */
int64_t version;/* = 1 */
};

And struct MakedumpfileDataHeader, with offset and buf_size set to -1, is used
to indicate the end of vmcore in flatten format.

struct MakedumpfileDataHeader {
int64_t offset; /* = -1 */
int64_t buf_size;   /* = -1 */
};

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|   42 ++
 include/sysemu/dump.h |   17 +
 2 files changed, 59 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index c9d3492..f233b3e 100644
--- a/dump.c
+++ b/dump.c
@@ -686,6 +686,48 @@ static int create_vmcore(DumpState *s)
 return 0;
 }
 
+static int write_start_flat_header(int fd)
+{
+uint8_t *buf;
+MakedumpfileHeader mh;
+int ret = 0;
+
+memset(mh, 0, sizeof(mh));
+strncpy(mh.signature, MAKEDUMPFILE_SIGNATURE,
+strlen(MAKEDUMPFILE_SIGNATURE));
+
+mh.type = cpu_to_be64(TYPE_FLAT_HEADER);
+mh.version = cpu_to_be64(VERSION_FLAT_HEADER);
+
+buf = g_malloc0(MAX_SIZE_MDF_HEADER);
+memcpy(buf, mh, sizeof(mh));
+
+size_t written_size;
+written_size = qemu_write_full(fd, buf, MAX_SIZE_MDF_HEADER);
+if (written_size != MAX_SIZE_MDF_HEADER) {
+ret = -1;
+}
+
+g_free(buf);
+return ret;
+}
+
+static int write_end_flat_header(int fd)
+{
+MakedumpfileDataHeader mdh;
+
+mdh.offset = END_FLAG_FLAT_HEADER;
+mdh.buf_size = END_FLAG_FLAT_HEADER;
+
+size_t written_size;
+written_size = qemu_write_full(fd, mdh, sizeof(mdh));
+if (written_size != sizeof(mdh)) {
+return -1;
+}
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 19fafb2..b32b390 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -14,12 +14,29 @@
 #ifndef DUMP_H
 #define DUMP_H
 
+#define MAKEDUMPFILE_SIGNATURE  makedumpfile
+#define MAX_SIZE_MDF_HEADER (4096) /* max size of makedumpfile_header 
*/
+#define TYPE_FLAT_HEADER(1)/* type of flattened format */
+#define VERSION_FLAT_HEADER (1)/* version of flattened format */
+#define END_FLAG_FLAT_HEADER(-1)
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
 int d_class;/* ELFCLASS32 or ELFCLASS64 */
 } ArchDumpInfo;
 
+typedef struct QEMU_PACKED MakedumpfileHeader {
+char signature[16]; /* = makedumpfile */
+int64_t type;
+int64_t version;
+} MakedumpfileHeader;
+
+typedef struct QEMU_PACKED MakedumpfileDataHeader {
+int64_t offset;
+int64_t buf_size;
+} MakedumpfileDataHeader;
+
 struct GuestPhysBlockList; /* memory_mapping.h */
 int cpu_get_dump_info(ArchDumpInfo *info,
   const struct GuestPhysBlockList *guest_phys_blocks);
-- 
1.7.1

[Qemu-devel] [PATCH 08/13 v7] dump: add API to write dump header

2014-01-16 Thread qiaonuohan

the functions are used to write header of kdump-compressed format to vmcore.
Header of kdump-compressed format includes:
1. common header: DiskDumpHeader32 / DiskDumpHeader64
2. sub header: KdumpSubHeader32 / KdumpSubHeader64
3. extra information: only elf notes here

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c|  223 +
 include/sysemu/dump.h |   96 +
 2 files changed, 319 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index bf7d31d..a7fdc3f 100644
--- a/dump.c
+++ b/dump.c
@@ -778,6 +778,229 @@ static int buf_write_note(const void *buf, size_t size, 
void *opaque)
 return 0;
 }
 
+/* write common header, sub header and elf note to vmcore */
+static int create_header32(DumpState *s)
+{
+int ret = 0;
+DiskDumpHeader32 *dh = NULL;
+KdumpSubHeader32 *kh = NULL;
+size_t size;
+int endian = s-dump_info.d_endian;
+uint32_t block_size;
+uint32_t sub_hdr_size;
+uint32_t bitmap_blocks;
+uint32_t status = 0;
+uint64_t offset_note;
+
+/* write common header, the version of kdump-compressed format is 6th */
+size = sizeof(DiskDumpHeader32);
+dh = g_malloc0(size);
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = cpu_convert_to_target32(6, endian);
+block_size = s-page_size;
+dh-block_size = cpu_convert_to_target32(block_size, endian);
+sub_hdr_size = sizeof(struct KdumpSubHeader32) + s-note_size;
+sub_hdr_size = DIV_ROUND_UP(sub_hdr_size, block_size);
+dh-sub_hdr_size = cpu_convert_to_target32(sub_hdr_size, endian);
+/* dh-max_mapnr may be truncated, full 64bit is in kh.max_mapnr_64 */
+dh-max_mapnr = cpu_convert_to_target32(MIN(s-max_mapnr, UINT_MAX),
+endian);
+dh-nr_cpus = cpu_convert_to_target32(s-nr_cpus, endian);
+bitmap_blocks = DIV_ROUND_UP(s-len_dump_bitmap, block_size) * 2;
+dh-bitmap_blocks = cpu_convert_to_target32(bitmap_blocks, endian);
+memcpy((dh-utsname.machine), i686, 4);
+
+if (s-flag_compress  DUMP_DH_COMPRESSED_ZLIB) {
+status |= DUMP_DH_COMPRESSED_ZLIB;
+}
+#ifdef CONFIG_LZO
+if (s-flag_compress  DUMP_DH_COMPRESSED_LZO) {
+status |= DUMP_DH_COMPRESSED_LZO;
+}
+#endif
+#ifdef CONFIG_SNAPPY
+if (s-flag_compress  DUMP_DH_COMPRESSED_SNAPPY) {
+status |= DUMP_DH_COMPRESSED_SNAPPY;
+}
+#endif
+dh-status = cpu_convert_to_target32(status, endian);
+
+if (write_buffer(s-fd, 0, dh, size)  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+ret = -1;
+goto out;
+}
+
+/* write sub header */
+size = sizeof(KdumpSubHeader32);
+kh = g_malloc0(size);
+
+/* 64bit max_mapnr_64 */
+kh-max_mapnr_64 = cpu_convert_to_target64(s-max_mapnr, endian);
+kh-phys_base = cpu_convert_to_target32(PHYS_BASE, endian);
+kh-dump_level = cpu_convert_to_target32(DUMP_LEVEL, endian);
+
+offset_note = DISKDUMP_HEADER_BLOCKS * block_size + size;
+kh-offset_note = cpu_convert_to_target64(offset_note, endian);
+kh-note_size = cpu_convert_to_target32(s-note_size, endian);
+
+if (write_buffer(s-fd, DISKDUMP_HEADER_BLOCKS *
+ block_size, kh, size)  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+ret = -1;
+goto out;
+}
+
+/* write note */
+s-note_buf = g_malloc0(s-note_size);
+s-note_buf_offset = 0;
+
+/* use s-note_buf to store notes temporarily */
+if (write_elf32_notes(buf_write_note, s)  0) {
+ret = -1;
+goto out;
+}
+
+if (write_buffer(s-fd, offset_note, s-note_buf,
+ s-note_size)  0) {
+dump_error(s, dump: failed to write notes);
+ret = -1;
+goto out;
+}
+
+/* get offset of dump_bitmap */
+s-offset_dump_bitmap = (DISKDUMP_HEADER_BLOCKS + sub_hdr_size) *
+ block_size;
+
+/* get offset of page */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + sub_hdr_size + bitmap_blocks) *
+ block_size;
+
+out:
+g_free(dh);
+g_free(kh);
+g_free(s-note_buf);
+
+return ret;
+}
+
+/* write common header, sub header and elf note to vmcore */
+static int create_header64(DumpState *s)
+{
+int ret = 0;
+DiskDumpHeader64 *dh = NULL;
+KdumpSubHeader64 *kh = NULL;
+size_t size;
+int endian = s-dump_info.d_endian;
+uint32_t block_size;
+uint32_t sub_hdr_size;
+uint32_t bitmap_blocks;
+uint32_t status = 0;
+uint64_t offset_note;
+
+/* write common header, the version of kdump-compressed format is 6th */
+size = sizeof(DiskDumpHeader64);
+dh = g_malloc0(size);
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = cpu_convert_to_target32(6, endian);
+

[Qemu-devel] [PATCH 12/13 v7] dump: make kdump-compressed format available for 'dump-guest-memory'

2014-01-16 Thread qiaonuohan

Make monitor command 'dump-guest-memory' be able to dump in kdump-compressed
format. The command's usage:

  dump [-p] protocol [begin] [length] [format]

'format' is used to specified the format of vmcore and can be:
1. 'elf': ELF format, without compression
2. 'kdump-zlib': kdump-compressed format, with zlib-compressed
3. 'kdump-lzo': kdump-compressed format, with lzo-compressed
4. 'kdump-snappy': kdump-compressed format, with snappy-compressed
Without 'format' being set, it is same as 'elf'. And if non-elf format is
specified, paging and filter is not allowed.

Note:
  1. The kdump-compressed format is readable only with the crash utility and
 makedumpfile, and it can be smaller than the ELF format because of the
 compression support.
  2. The kdump-compressed format is the 6th edition.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 dump.c   |  129 +++---
 hmp.c|5 ++-
 qapi-schema.json |   25 ++-
 qmp-commands.hx  |7 ++-
 4 files changed, 156 insertions(+), 10 deletions(-)

diff --git a/dump.c b/dump.c
index bb03ef7..dbf4bb6 100644
--- a/dump.c
+++ b/dump.c
@@ -1449,6 +1449,64 @@ out:
 return ret;
 }
 
+static int create_kdump_vmcore(DumpState *s)
+{
+int ret;
+
+/*
+ * the kdump-compressed format is:
+ *   File offset
+ *  +--+ 0x0
+ *  |main header (struct disk_dump_header) |
+ *  |--+ block 1
+ *  |sub header (struct kdump_sub_header)  |
+ *  |--+ block 2
+ *  |1st-dump_bitmap   |
+ *  |--+ block 2 + X blocks
+ *  |2nd-dump_bitmap   | (aligned by block)
+ *  |--+ block 2 + 2 * X blocks
+ *  |  page desc for pfn 0 (struct page_desc)  | (aligned by block)
+ *  |  page desc for pfn 1 (struct page_desc)  |
+ *  |: |
+ *  |--| (not aligned by block)
+ *  | page data (pfn 0)|
+ *  | page data (pfn 1)|
+ *  |: |
+ *  +--+
+ */
+
+ret = write_start_flat_header(s-fd);
+if (ret  0) {
+dump_error(s, dump: failed to write start flat header.\n);
+return -1;
+}
+
+ret = write_dump_header(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_dump_bitmap(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_dump_pages(s);
+if (ret  0) {
+return -1;
+}
+
+ret = write_end_flat_header(s-fd);
+if (ret  0) {
+dump_error(s, dump: failed to write end flat header.\n);
+return -1;
+}
+
+dump_completed(s);
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
@@ -1487,7 +1545,8 @@ static void get_max_mapnr(DumpState *s)
 }
 }
 
-static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
+static int dump_init(DumpState *s, int fd, bool has_format,
+ DumpGuestMemoryFormat format, bool paging, bool 
has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
 CPUState *cpu;
@@ -1495,6 +1554,11 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 Error *err = NULL;
 int ret;
 
+/* kdump-compressed is conflict with paging and filter */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+assert(!paging  !has_filter);
+}
+
 if (runstate_is_running()) {
 vm_stop(RUN_STATE_SAVE_VM);
 s-resume = true;
@@ -1565,6 +1629,28 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 tmp = DIV_ROUND_UP(DIV_ROUND_UP(s-max_mapnr, CHAR_BIT), s-page_size);
 s-len_dump_bitmap = tmp * s-page_size;
 
+/* init for kdump-compressed format */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+switch (format) {
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_ZLIB:
+s-flag_compress = DUMP_DH_COMPRESSED_ZLIB;
+break;
+
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_LZO:
+s-flag_compress = DUMP_DH_COMPRESSED_LZO;
+break;
+
+case DUMP_GUEST_MEMORY_FORMAT_KDUMP_SNAPPY:
+s-flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
+break;
+
+default:
+s-flag_compress = 0;
+}
+
+return 0;
+}
+
 if (s-has_filter) {
 memory_mapping_filter(s-list, s-begin, s-length);
 }
@@ -1624,14 +1710,25 @@ cleanup:
 }
 
 void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
-

[Qemu-devel] [PATCH 04/13 v7] dump: add API to write vmcore

2014-01-16 Thread qiaonuohan

Function is used to write vmcore in flatten format. In flatten format, data is
written block by block, and in front of each block, a struct
MakedumpfileDataHeader is stored there to indicate the offset and size of the
data block.

struct MakedumpfileDataHeader {
int64_t offset;
int64_t buf_size;
};

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Laszlo Ersek ler...@redhat.com
---
 dump.c |   21 +
 1 files changed, 21 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index f233b3e..238ffa5 100644
--- a/dump.c
+++ b/dump.c
@@ -728,6 +728,27 @@ static int write_end_flat_header(int fd)
 return 0;
 }
 
+static int write_buffer(int fd, off_t offset, const void *buf, size_t size)
+{
+size_t written_size;
+MakedumpfileDataHeader mdh;
+
+mdh.offset = cpu_to_be64(offset);
+mdh.buf_size = cpu_to_be64(size);
+
+written_size = qemu_write_full(fd, mdh, sizeof(mdh));
+if (written_size != sizeof(mdh)) {
+return -1;
+}
+
+written_size = qemu_write_full(fd, buf, size);
+if (written_size != size) {
+return -1;
+}
+
+return 0;
+}
+
 static ram_addr_t get_start_block(DumpState *s)
 {
 GuestPhysBlock *block;
-- 
1.7.1

[Qemu-devel] [PATCH v4 0/9] Make 'dump-guest-memory' dump in kdump-compressed format

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Hi, all

The last version is here:
http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg02280.html

Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

For migration, 'dump-guest-memory' is supposed to support compression feature.
Because of the regression, the missing of compression feature, we post these
patches to make 'dump-guest-memory' be able to dump guest's in kdump-compressed
format. Then vmcore can be much smaller, and easily be delivered.

The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support.

Note, similar to 'dump-guest-memory':
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
   available by default, and option '--enable-lzo' or '--enable-snappy'
   should be used with configure to make lzo or snappy available.

Changelog:
Changes from v3 to v4:
1. change to avoid conflict with Andreas's patches
2. rebase

Changes from v2 to v3:
1. Address Eric's comment

Changes from v1 to v2:
1. Address Eric  Daniel's comment: fix manner of string copy.
2. Address Eric's comment: replace reinventing new constants by using the
   ready-made ones accoring.
3. Address Andreas's comment: remove useless include.


Qiao Nuohan (9):
  dump: Add API to manipulate dump_bitmap
  dump: Add API to manipulate cache_data
  dump: Move struct definition into dump_memroy.h
  dump: Add API to create header of vmcore
  dump: Add API to create data of dump bitmap
  dump: Add API to create page
  dump: Add API to free memory used by creating header, bitmap and page
  dump: Add API to write header, bitmap and page into vmcore
  dump: Make kdump-compressed format available for 'dump-guest-memory'

 Makefile.target  |1 +
 cache_data.c |  121 ++
 configure|   50 +++
 dump.c   |  866 --
 dump_bitmap.c|  171 +
 hmp-commands.hx  |   12 +-
 hmp.c|   23 +-
 include/cache_data.h |   56 +++
 include/dump_bitmap.h|   60 +++
 include/sysemu/dump_memory.h |  178 +
 qapi-schema.json |   22 +-
 qmp-commands.hx  |6 +-
 12 files changed, 1526 insertions(+), 40 deletions(-)
 create mode 100644 cache_data.c
 create mode 100644 dump_bitmap.c
 create mode 100644 include/cache_data.h
 create mode 100644 include/dump_bitmap.h
 create mode 100644 include/sysemu/dump_memory.h

[Qemu-devel] [PATCH v4 3/9] dump: Move struct definition into dump_memroy.h

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Move definition of struct DumpState into include/sysemu/dump_memory.h.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c   |   22 +-
 include/sysemu/dump_memory.h |   40 
 2 files changed, 41 insertions(+), 21 deletions(-)
 create mode 100644 include/sysemu/dump_memory.h

diff --git a/dump.c b/dump.c
index c0d3da5..9ac66be 100644
--- a/dump.c
+++ b/dump.c
@@ -15,10 +15,9 @@
 #include elf.h
 #include cpu.h
 #include exec/cpu-all.h
-#include exec/hwaddr.h
 #include monitor/monitor.h
 #include sysemu/kvm.h
-#include sysemu/dump.h
+#include sysemu/dump_memory.h
 #include sysemu/sysemu.h
 #include sysemu/memory_mapping.h
 #include qapi/error.h
@@ -57,25 +56,6 @@ static uint64_t cpu_convert_to_target64(uint64_t val, int 
endian)
 return val;
 }
 
-typedef struct DumpState {
-ArchDumpInfo dump_info;
-MemoryMappingList list;
-uint16_t phdr_num;
-uint32_t sh_info;
-bool have_section;
-bool resume;
-size_t note_size;
-hwaddr memory_offset;
-int fd;
-
-RAMBlock *block;
-ram_addr_t start;
-bool has_filter;
-int64_t begin;
-int64_t length;
-Error **errp;
-} DumpState;
-
 static int dump_cleanup(DumpState *s)
 {
 int ret = 0;
diff --git a/include/sysemu/dump_memory.h b/include/sysemu/dump_memory.h
new file mode 100644
index 000..ce22c05
--- /dev/null
+++ b/include/sysemu/dump_memory.h
@@ -0,0 +1,40 @@
+/*
+ * QEMU dump memory
+ *
+ * Copyright (C) 2013 FUJITSU LIMITED
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef DUMP_MEMORY_H
+#define DUMP_MEMORY_H
+
+#include exec/cpu-all.h
+#include sysemu/memory_mapping.h
+#include sysemu/dump.h
+
+typedef struct DumpState {
+ArchDumpInfo dump_info;
+MemoryMappingList list;
+uint16_t phdr_num;
+uint32_t sh_info;
+bool have_section;
+bool resume;
+size_t note_size;
+hwaddr memory_offset;
+int fd;
+
+RAMBlock *block;
+ram_addr_t start;
+bool has_filter;
+int64_t begin;
+int64_t length;
+Error **errp;
+} DumpState;
+
+#endif
-- 
1.7.1

[Qemu-devel] [PATCH v4 1/9] dump: Add API to manipulate dump_bitmap

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Struct dump_bitmap is associated with a tmp file which is used to store bitmap
in kdump-compressed format temporarily. The following patch will use these
functions to gather data of bitmap and cache them into tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 Makefile.target   |1 +
 dump_bitmap.c |  171 +
 include/dump_bitmap.h |   60 +
 3 files changed, 232 insertions(+), 0 deletions(-)
 create mode 100644 dump_bitmap.c
 create mode 100644 include/dump_bitmap.h

diff --git a/Makefile.target b/Makefile.target
index ce4391f..8e557f9 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -110,6 +110,7 @@ obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
+obj-y += dump_bitmap.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
diff --git a/dump_bitmap.c b/dump_bitmap.c
new file mode 100644
index 000..f35f39d
--- /dev/null
+++ b/dump_bitmap.c
@@ -0,0 +1,171 @@
+/*
+ * QEMU dump bitmap
+ *
+ * Copyright (C) 2013 FUJITSU LIMITED
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include qemu-common.h
+#include dump_bitmap.h
+
+int init_dump_bitmap(struct dump_bitmap *db, const char *filename)
+{
+int fd;
+char *tmpname;
+
+/* init the tmp file */
+tmpname = getenv(TMPDIR);
+if (!tmpname) {
+tmpname = (char *)P_tmpdir;
+}
+
+db-file_name = (char *)g_strdup_printf(%s/%s, tmpname, filename);
+
+fd = mkstemp(db-file_name);
+if (fd  0) {
+return -1;
+}
+
+db-fd = fd;
+unlink(db-file_name);
+
+/* num_block should be set to -1, for nothing is store in buf now */
+db-num_block = -1;
+memset(db-buf, 0, BUFSIZE_BITMAP);
+/* the tmp file starts to write at the very beginning */
+db-offset = 0;
+
+return 0;
+}
+
+int clear_dump_bitmap(struct dump_bitmap *db, size_t len_db)
+{
+int ret;
+char buf[BUFSIZE_BITMAP];
+off_t offset_bit;
+
+/* use buf to clear the tmp file block by block */
+memset(buf, 0, sizeof(buf));
+
+ret = lseek(db-fd, db-offset, SEEK_SET);
+if (ret  0) {
+return -1;
+}
+
+offset_bit = 0;
+while (offset_bit  len_db) {
+if (write(db-fd, buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+
+offset_bit += BUFSIZE_BITMAP;
+}
+
+return 0;
+}
+
+int set_dump_bitmap(struct dump_bitmap *db, unsigned long long pfn, int val)
+{
+int byte, bit;
+off_t old_offset, new_offset;
+old_offset = db-offset + BUFSIZE_BITMAP * db-num_block;
+new_offset = db-offset + BUFSIZE_BITMAP * (pfn / PFN_BUFBITMAP);
+
+/*
+ * if the block needed to be set is not same as the one cached in buf,
+ * write the block back to the tmp file, then cache new block to the buf
+ */
+if (0 = db-num_block  old_offset != new_offset) {
+if (lseek(db-fd, old_offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(db-fd, db-buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+}
+
+if (old_offset != new_offset) {
+if (lseek(db-fd, new_offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (read(db-fd, db-buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+
+db-num_block = pfn / PFN_BUFBITMAP;
+}
+
+/* get the exact place of the bit in the buf, set it */
+byte = (pfn % PFN_BUFBITMAP)  3;
+bit  = (pfn % PFN_BUFBITMAP)  7;
+if (val) {
+db-buf[byte] |= 1bit;
+} else {
+db-buf[byte] = ~(1bit);
+}
+
+return 0;
+}
+
+int sync_dump_bitmap(struct dump_bitmap *db)
+{
+off_t offset;
+offset = db-offset + BUFSIZE_BITMAP * db-num_block;
+
+/* db has not been used yet */
+if (db-num_block  0) {
+return 0;
+}
+
+if (lseek(db-fd, offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(db-fd, db-buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+
+return 0;
+}
+
+/*
+ * check if the bit is set to 1
+ */
+static inline int is_on(char *bitmap, int i)
+{
+return bitmap[i3]  (1  (i  7));
+}
+
+int is_bit_set(struct dump_bitmap *db, unsigned long long pfn)
+{
+off_t offset;
+
+/* cache the block pfn belongs to, then check the block */
+if (pfn == 0 || db-num_block != pfn/PFN_BUFBITMAP) {
+offset = db-offset + BUFSIZE_BITMAP * (pfn/PFN_BUFBITMAP);
+lseek(db-fd, offset, SEEK_SET);
+read(db-fd, db-buf, BUFSIZE_BITMAP);
+db-num_block = pfn/PFN_BUFBITMAP;
+}
+
+return is_on(db-buf,

[Qemu-devel] [PATCH v4 5/9] dump: Add API to create data of dump bitmap

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Functions in this patch are used to gather data of 1st and 2nd dump bitmap in
kdump-compressed format. The following patch will use these functions to gather
data of dump bitmap, then cache them into tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c   |   98 ++
 include/sysemu/dump_memory.h |   11 +
 2 files changed, 109 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 3b9d4ca..dc214b7 100644
--- a/dump.c
+++ b/dump.c
@@ -788,6 +788,104 @@ static int create_header(DumpState *s)
 return create_header64(s);
 }
 
+/*
+ * create two tmpfile and save 1st and 2nd bitmap separately
+ */
+static int prepare_dump_bitmap(DumpState *s)
+{
+int ret;
+struct dump_bitmap *db1;
+struct dump_bitmap *db2;
+
+db1 = g_malloc0(sizeof(struct dump_bitmap));
+
+db2 = g_malloc0(sizeof(struct dump_bitmap));
+
+ret = init_dump_bitmap(db1, FILENAME_BITMAP1);
+if (ret  0) {
+dump_error(s, dump: failed to init db1.);
+return -1;
+}
+s-dump_bitmap1 = db1;
+
+ret = init_dump_bitmap(db2, FILENAME_BITMAP2);
+if (ret  0) {
+dump_error(s, dump: failed to init db1.);
+return -1;
+}
+s-dump_bitmap2 = db2;
+
+return 0;
+}
+
+static int create_dump_bitmap(DumpState *s)
+{
+int ret;
+unsigned long long num_dumpable;
+MemoryMapping *memory_mapping;
+unsigned long long pfn_start, pfn_end, pfn;
+
+ret = prepare_dump_bitmap(s);
+if (ret  0) {
+dump_error(s, dump: failed to prepare dump_bitmap.\n);
+return -1;
+}
+
+ret = clear_dump_bitmap(s-dump_bitmap1, s-len_dump_bitmap / 2);
+if (ret  0) {
+dump_error(s, dump: failed to clear dump_bitmap1.\n);
+return -1;
+}
+
+ret = clear_dump_bitmap(s-dump_bitmap2, s-len_dump_bitmap / 2);
+if (ret  0) {
+dump_error(s, dump: failed to clear dump_bitmap2.\n);
+return -1;
+}
+
+/* write dump bitmap to tmp files */
+num_dumpable = 0;
+
+QTAILQ_FOREACH(memory_mapping, s-list.head, next) {
+pfn_start = paddr_to_pfn(memory_mapping-phys_addr, s-page_shift);
+pfn_end = paddr_to_pfn(memory_mapping-phys_addr +
+memory_mapping-length, s-page_shift);
+
+for (pfn = pfn_start; pfn  pfn_end; pfn++) {
+ret = set_dump_bitmap(s-dump_bitmap1, pfn, 1);
+if (ret  0) {
+dump_error(s, dump: failed to set dump_bitmap1.\n);
+return -1;
+}
+/* set dump_bitmap2, same as dump_bitmap1 */
+ret = set_dump_bitmap(s-dump_bitmap2, pfn, 1);
+if (ret  0) {
+dump_error(s, dump: failed to set dump_bitmap2.\n);
+return -1;
+}
+num_dumpable++;
+}
+}
+
+/* write cached data to tmp files */
+ret = sync_dump_bitmap(s-dump_bitmap1);
+if (ret  0) {
+dump_error(s, dump: failed to sync dump_bitmap1.\n);
+return -1;
+}
+
+ret = sync_dump_bitmap(s-dump_bitmap2);
+if (ret  0) {
+dump_error(s, dump: failed to sync dump_bitmap2.\n);
+return -1;
+}
+
+/* get the number of dumpable page */
+s-num_dumpable = num_dumpable;
+
+return 0;
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
diff --git a/include/sysemu/dump_memory.h b/include/sysemu/dump_memory.h
index 56e0f40..2b60af6 100644
--- a/include/sysemu/dump_memory.h
+++ b/include/sysemu/dump_memory.h
@@ -18,13 +18,20 @@
 #include sysemu/memory_mapping.h
 #include sysemu/dump.h
 
+#include dump_bitmap.h
+
 #define KDUMP_SIGNATURE KDUMP   
 #define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
 #define DISKDUMP_HEADER_BLOCKS  (1)
 #define PHYS_BASE   (0)
 #define DUMP_LEVEL  (1)
+#define ARCH_PFN_OFFSET (0)
+#define FILENAME_BITMAP1kdump_bitmap1_XX
+#define FILENAME_BITMAP2kdump_bitmap2_XX
 
 #define divideup(x, y)  (((x) + ((y) - 1)) / (y))
+#define paddr_to_pfn(X, page_shift) \
+(((unsigned long long)(X)  (page_shift)) - ARCH_PFN_OFFSET)
 
 struct new_utsname {
 char sysname[65];
@@ -118,14 +125,18 @@ typedef struct DumpState {
 Error **errp;
 
 int page_size;
+int page_shift;
 unsigned long long max_mapnr;
 int nr_cpus;
 void *dh;
 void *kh;
+unsigned long long num_dumpable;
 off_t offset_sub_header;
 
 off_t offset_dump_bitmap;
 unsigned long len_dump_bitmap;
+struct dump_bitmap *dump_bitmap1;
+struct dump_bitmap *dump_bitmap2;
 
 off_t offset_page;
 } DumpState;
-- 
1.7.1

[Qemu-devel] [PATCH v4 8/9] dump: Add API to write header, bitmap and page into vmcore

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

The following patch will use these functions to write cached data into vmcore.
Header is cached in DumpState, and bitmap and page are cached in tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c   |  259 ++
 include/sysemu/dump_memory.h |1 +
 2 files changed, 260 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 91a2384..17686ad 100644
--- a/dump.c
+++ b/dump.c
@@ -76,6 +76,36 @@ static void dump_error(DumpState *s, const char *reason)
 dump_cleanup(s);
 }
 
+/*
+ * write 'size' of 'c' to dump file
+ */
+static int fill_space(size_t size, int c, void *opaque)
+{
+DumpState *s = opaque;
+char tmpbuf[TMP_BUF_SIZE];
+size_t fill_size;
+size_t written_size;
+
+memset(tmpbuf, c, TMP_BUF_SIZE);
+
+while (size) {
+if (size  TMP_BUF_SIZE) {
+fill_size = TMP_BUF_SIZE;
+} else {
+fill_size = size;
+}
+
+size -= fill_size;
+
+written_size = qemu_write_full(s-fd, tmpbuf, fill_size);
+if (written_size != fill_size) {
+return -1;
+}
+}
+
+return 0;
+}
+
 static int fd_write_vmcore(void *buf, size_t size, void *opaque)
 {
 DumpState *s = opaque;
@@ -681,6 +711,235 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static int write_dump_header(DumpState *s)
+{
+int ret;
+void *dh;
+void *kh;
+
+/* write common header */
+dh = s-dh;
+
+if (s-dump_info.d_machine == EM_386) {
+ret = fd_write_vmcore(dh, sizeof(struct disk_dump_header32), s);
+if (ret  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+return -1;
+}
+} else {
+ret = fd_write_vmcore(dh, sizeof(struct disk_dump_header64), s);
+if (ret  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+return -1;
+}
+}
+
+/* fill gap between command header and sub header */
+ret = fill_space(s-offset_sub_header, 0, s);
+if (ret  0) {
+dump_error(s, dump: failed to fill the space between header and\
+sub header.\n);
+return -1;
+}
+
+/* write sub header */
+kh = s-kh;
+
+if (s-dump_info.d_machine == EM_386) {
+ret = fd_write_vmcore(kh, sizeof(struct kdump_sub_header32), s);
+if (ret  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+return -1;
+}
+} else {
+ret = fd_write_vmcore(kh, sizeof(struct kdump_sub_header64), s);
+if (ret  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+return -1;
+}
+}
+
+/* write note */
+if (s-dump_info.d_class == ELFCLASS64) {
+if (write_elf64_notes(s)  0) {
+return -1;
+}
+} else {
+if (write_elf32_notes(s)  0) {
+return -1;
+}
+   }
+
+return 0;
+}
+
+static int write_dump_bitmap(DumpState *s)
+{
+struct cache_data bm;
+long buf_size;
+int ret;
+int no_bitmap;
+
+/* fill gap between header and dump_bitmap */
+ret = fill_space(s-offset_dump_bitmap, 0, s);
+if (ret  0) {
+dump_error(s, dump: failed to fill the space between header and\
+dump_bitmap.\n);
+goto out;
+}
+
+bm.buf = g_malloc0(TMP_BUF_SIZE);
+
+/* write dump_bitmap1 */
+bm.fd = s-dump_bitmap1-fd;
+no_bitmap = 1;
+
+again:
+buf_size = s-len_dump_bitmap / 2;
+bm.offset = 0;
+
+while (buf_size  0) {
+if (buf_size = TMP_BUF_SIZE) {
+bm.cache_size = TMP_BUF_SIZE;
+} else {
+bm.cache_size = buf_size;
+}
+
+ret = read_cache(bm);
+if (ret  0) {
+goto out;
+}
+
+ret = fd_write_vmcore(bm.buf, bm.cache_size, s);
+if (ret  0) {
+goto out;
+}
+
+buf_size -= bm.cache_size;
+}
+
+/* switch to dump_bitmap2 */
+if (no_bitmap == 1) {
+no_bitmap = 2;
+bm.fd = s-dump_bitmap2-fd;
+goto again;
+}
+
+return 0;
+
+out:
+if (bm.buf) {
+g_free(bm.buf);
+}
+
+return -1;
+}
+
+static int write_dump_pages(DumpState *s)
+{
+struct cache_data page;
+unsigned long long total_size;
+int is_page_desc;
+int ret;
+
+page.buf = g_malloc0(TMP_BUF_SIZE);
+
+/* write page_desc */
+is_page_desc = 1;
+total_size = s-page_desc_size;
+page.fd = s-page_desc-fd;
+page.offset = s-page_desc-offset;
+
+again:
+while (total_size  0) {
+if (total_size  TMP_BUF_SIZE) {
+page.cache_size = TMP_BUF_SIZE;
+} else {
+page.cache_size = total_size;
+}
+
+ret = read_cache(page);
+if (ret  0)

[Qemu-devel] [PATCH v4 7/9] dump: Add API to free memory used by creating header, bitmap and page

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

When calling create_header, create_dump_bitmap and create_pages, some memory
spaces are allocated. The following patch will use this function to free these
memory.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c |   19 +++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 405eeb1..91a2384 100644
--- a/dump.c
+++ b/dump.c
@@ -1261,6 +1261,25 @@ cleanup:
 return -1;
 }
 
+static void clean_state(DumpState *s)
+{
+if (s-dh) {
+g_free(s-dh);
+}
+
+if (s-kh) {
+g_free(s-kh);
+}
+
+free_dump_bitmap(s-dump_bitmap1);
+
+free_dump_bitmap(s-dump_bitmap2);
+
+free_cache_data(s-page_desc);
+
+free_cache_data(s-page_data);
+}
+
 void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
int64_t begin, bool has_length, int64_t length,
Error **errp)
-- 
1.7.1

[Qemu-devel] [PATCH v4 4/9] dump: Add API to create header of vmcore

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Functions in this patch are used to gather data of header and sub header in
kdump-compressed format. The following patch will use these functions to gather
data of header, then cache them into struct DumpState.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c   |  107 ++
 include/sysemu/dump_memory.h |   93 
 2 files changed, 200 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 9ac66be..3b9d4ca 100644
--- a/dump.c
+++ b/dump.c
@@ -681,6 +681,113 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static int create_header32(DumpState *s)
+{
+struct  disk_dump_header32 *dh;
+struct  kdump_sub_header32 *kh;
+
+/* create common header, the version of kdump-compressed format is 5th */
+dh = g_malloc0(sizeof(struct disk_dump_header32));
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = 5;
+dh-block_size = s-page_size;
+dh-sub_hdr_size = sizeof(struct kdump_sub_header32) + s-note_size;
+dh-sub_hdr_size = divideup(dh-sub_hdr_size, dh-block_size);
+dh-max_mapnr = s-max_mapnr;
+dh-nr_cpus = s-nr_cpus;
+dh-bitmap_blocks = divideup(s-len_dump_bitmap, s-page_size);
+
+memcpy((dh-utsname.machine), i686, 4);
+
+s-dh = dh;
+
+/* create sub header */
+kh = g_malloc0(sizeof(struct kdump_sub_header32));
+
+kh-phys_base = PHYS_BASE;
+kh-dump_level = DUMP_LEVEL;
+
+kh-offset_note = DISKDUMP_HEADER_BLOCKS * dh-block_size +
+sizeof(struct kdump_sub_header32);
+kh-note_size = s-note_size;
+
+s-kh = kh;
+
+/* get gap between header and sub header */
+s-offset_sub_header = DISKDUMP_HEADER_BLOCKS * dh-block_size -
+sizeof(struct disk_dump_header32);
+
+/* get gap between header and dump_bitmap */
+s-offset_dump_bitmap = dh-sub_hdr_size * dh-block_size -
+(sizeof(struct kdump_sub_header32) + s-note_size);
+
+/* get offset of page desc */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + dh-sub_hdr_size +
+dh-bitmap_blocks) * dh-block_size;
+
+return 0;
+}
+
+static int create_header64(DumpState *s)
+{
+struct  disk_dump_header64 *dh;
+struct  kdump_sub_header64 *kh;
+
+/* create common header, the version of kdump-compressed format is 5th */
+dh = g_malloc0(sizeof(struct disk_dump_header64));
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = 5;
+dh-block_size = s-page_size;
+dh-sub_hdr_size = sizeof(struct kdump_sub_header64) + s-note_size;
+dh-sub_hdr_size = divideup(dh-sub_hdr_size, dh-block_size);
+dh-max_mapnr = s-max_mapnr;
+dh-nr_cpus = s-nr_cpus;
+dh-bitmap_blocks = divideup(s-len_dump_bitmap, s-page_size);
+
+memcpy((dh-utsname.machine), x86_64, 6);
+
+s-dh = dh;
+
+/* create sub header */
+kh = g_malloc0(sizeof(struct kdump_sub_header64));
+
+kh-phys_base = PHYS_BASE;
+kh-dump_level = DUMP_LEVEL;
+
+kh-offset_note = DISKDUMP_HEADER_BLOCKS * dh-block_size +
+sizeof(struct kdump_sub_header64);
+kh-note_size = s-note_size;
+
+s-kh = kh;
+
+/* get gap between header and sub header */
+s-offset_sub_header = DISKDUMP_HEADER_BLOCKS * dh-block_size -
+sizeof(struct disk_dump_header64);
+
+/* get gap between header and dump_bitmap */
+s-offset_dump_bitmap = dh-sub_hdr_size * dh-block_size -
+(sizeof(struct kdump_sub_header64) + s-note_size);
+
+/* get offset of page desc */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + dh-sub_hdr_size +
+dh-bitmap_blocks) * dh-block_size;
+
+return 0;
+}
+
+/*
+ * gather data of header and sub header
+ */
+static int create_header(DumpState *s)
+{
+if (s-dump_info.d_machine == EM_386)
+return create_header32(s);
+else
+return create_header64(s);
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
diff --git a/include/sysemu/dump_memory.h b/include/sysemu/dump_memory.h
index ce22c05..56e0f40 100644
--- a/include/sysemu/dump_memory.h
+++ b/include/sysemu/dump_memory.h
@@ -18,6 +18,87 @@
 #include sysemu/memory_mapping.h
 #include sysemu/dump.h
 
+#define KDUMP_SIGNATURE KDUMP   
+#define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
+#define DISKDUMP_HEADER_BLOCKS  (1)
+#define PHYS_BASE   (0)
+#define DUMP_LEVEL  (1)
+
+#define divideup(x, y)  (((x) + ((y) - 1)) / (y))
+
+struct new_utsname {
+char sysname[65];
+char nodename[65];
+char release[65];
+char

[Qemu-devel] [PATCH v4 2/9] dump: Add API to manipulate cache_data

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Struct cache_data is associated with a tmp file which is used to store page
desc and page data in kdump-compressed format temporarily. The following patch
will use these function to gather data of page and cache them in tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 Makefile.target  |2 +-
 cache_data.c |  121 ++
 include/cache_data.h |   56 +++
 3 files changed, 178 insertions(+), 1 deletions(-)
 create mode 100644 cache_data.c
 create mode 100644 include/cache_data.h

diff --git a/Makefile.target b/Makefile.target
index 8e557f9..298ec84 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -110,7 +110,7 @@ obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-y += dump_bitmap.o
+obj-y += dump_bitmap.o cache_data.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
diff --git a/cache_data.c b/cache_data.c
new file mode 100644
index 000..6e91538
--- /dev/null
+++ b/cache_data.c
@@ -0,0 +1,121 @@
+/*
+ * QEMU cache data
+ *
+ * Copyright (C) 2013 FUJITSU LIMITED
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include qemu-common.h
+#include cache_data.h
+
+int init_cache_data(struct cache_data *cd, const char *filename)
+{
+int fd;
+char *tmpname;
+
+/* init the tmp file */
+tmpname = getenv(TMPDIR);
+if (!tmpname) {
+tmpname = (char *)P_tmpdir;
+}
+
+cd-file_name = (char *)g_strdup_printf(%s/%s, tmpname, filename);
+
+fd = mkstemp(cd-file_name);
+if (fd  0) {
+return -1;
+}
+
+cd-fd = fd;
+unlink(cd-file_name);
+
+/* init buf */
+cd-buf_size = BUFSIZE_CACHE_DATA;
+cd-cache_size = 0;
+cd-buf = g_malloc0(BUFSIZE_CACHE_DATA);
+
+cd-offset = 0;
+
+return 0;
+}
+
+int write_cache(struct cache_data *cd, void *buf, size_t size)
+{
+/*
+ * check if the space is enough to cache data, if not write cached
+ * data to the tmp file
+ */
+if (cd-cache_size + size  cd-buf_size) {
+if (lseek(cd-fd, cd-offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(cd-fd, cd-buf, cd-cache_size) != cd-cache_size) {
+return -1;
+}
+
+cd-offset += cd-cache_size;
+cd-cache_size = 0;
+}
+
+memcpy(cd-buf + cd-cache_size, buf, size);
+cd-cache_size += size;
+
+return 0;
+}
+
+int sync_cache(struct cache_data *cd)
+{
+/* no data is cached in cache_data */
+if (cd-cache_size == 0) {
+return 0;
+}
+
+if (lseek(cd-fd, cd-offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(cd-fd, cd-buf, cd-cache_size) != cd-cache_size) {
+return -1;
+}
+
+cd-offset += cd-cache_size;
+
+return 0;
+}
+
+int read_cache(struct cache_data *cd)
+{
+if (lseek(cd-fd, cd-offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (read(cd-fd, cd-buf, cd-cache_size) != cd-cache_size) {
+return -1;
+}
+
+cd-offset += cd-cache_size;
+
+return 0;
+}
+
+void free_cache_data(struct cache_data *cd)
+{
+if (cd) {
+if (cd-file_name) {
+g_free(cd-file_name);
+}
+
+if (cd-buf) {
+g_free(cd-buf);
+}
+
+g_free(cd);
+}
+}
diff --git a/include/cache_data.h b/include/cache_data.h
new file mode 100644
index 000..18e8680
--- /dev/null
+++ b/include/cache_data.h
@@ -0,0 +1,56 @@
+/*
+ * QEMU cache data
+ *
+ * Copyright (C) 2013 FUJITSU LIMITED
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef CACHE_DATA_H
+#define CACHE_DATA_H
+
+#define BUFSIZE_CACHE_DATA  (4096 * 4)
+
+struct cache_data {
+int fd; /* fd of the tmp file used to store cache data */
+char *file_name;/* name of the tmp file */
+char *buf;  /* used to cache data */
+size_t buf_size;/* size of the buf */
+size_t cache_size;  /* size of cached data in buf */
+off_t offset;   /* offset of the tmp file */
+};
+
+/*
+ * create a tmp file used to store cache data, then init the buf
+ */
+int init_cache_data(struct cache_data *cd, const char *filename);
+
+/*
+ * write data to the tmp file, the data may first be cached in the buf of
+ * cache_data
+ */
+int write_cache(struct cache_data *cd, void *buf, size_t size);
+
+/*
+ * when cache_data is caching data in the buf, sync_cache is needed to write 
the
+ * data back to tmp file
+ */
+int sync_cache(struct

[Qemu-devel] [PATCH v4 6/9] dump: Add API to create page

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Functions in this patch are used to gather data of page desc and page data in
kdump-compressed format. The following patch will use these functions to gather
data of page, then cache them into tmp files

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c   |  259 ++
 include/sysemu/dump_memory.h |   32 +
 2 files changed, 291 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index dc214b7..405eeb1 100644
--- a/dump.c
+++ b/dump.c
@@ -886,6 +886,265 @@ static int create_dump_bitmap(DumpState *s)
 return 0;
 }
 
+/*
+ * create two tmpfile and save page_desc and page_data
+ */
+static int prepare_pages(DumpState *s)
+{
+int ret;
+struct cache_data *page_desc;
+struct cache_data *page_data;
+
+page_desc = g_malloc0(sizeof(struct cache_data));
+
+page_data = g_malloc0(sizeof(struct cache_data));
+
+ret = init_cache_data(page_desc, FILENAME_PAGE_DESC);
+if (ret  0) {
+dump_error(s, dump: failed to init page_desc.\n);
+return -1;
+}
+s-page_desc = page_desc;
+
+ret = init_cache_data(page_data, FILENAME_PAGE_DATA);
+if (ret  0) {
+dump_error(s, dump: failed to init page_desc.\n);
+return -1;
+}
+s-page_data = page_data;
+
+return 0;
+}
+
+/*
+ * memory should be read page by page, or it may exceed the boundary and
+ * fail to read
+ */
+static int readmem(void *bufptr, ram_addr_t addr, size_t size, DumpState *s)
+{
+RAMBlock *block;
+
+block = s-block;
+
+while (block) {
+if ((addr = block-offset) 
+(addr + size = block-offset + block-length)) {
+memcpy(bufptr, block-host + (addr - block-offset), size);
+return 0;
+} else {
+block = QTAILQ_NEXT(block, next);
+}
+}
+
+return -1;
+}
+
+/*
+ * check if the page is all 0
+ */
+static inline int is_zero_page(unsigned char *buf, long page_size)
+{
+size_t i;
+
+for (i = 0; i  page_size; i++) {
+if (buf[i]) {
+return 0;
+}
+}
+
+return 1;
+}
+
+static int create_pages(DumpState *s)
+{
+int ret;
+unsigned long long pfn;
+unsigned char buf[s-page_size];
+unsigned char *buf_out = NULL;
+unsigned long len_buf_out;
+unsigned long size_out;
+int zero_page;
+struct page_desc pd, pd_zero;
+off_t offset_desc, offset_data;
+unsigned long len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
+
+ret = 0;
+
+ret = prepare_pages(s);
+if (ret  0) {
+dump_error(s, dump: failed to prepare pages.\n);
+goto out;
+}
+
+/* init buf_out */
+len_buf_out_zlib = len_buf_out_lzo = len_buf_out_snappy = 0;
+
+/* buf size for zlib */
+len_buf_out_zlib = compressBound(s-page_size);
+
+/* buf size for lzo */
+#ifdef CONFIG_LZO
+if (s-flag_compress  DUMP_DH_COMPRESSED_LZO) {
+if (lzo_init() != LZO_E_OK) {
+ret = -1;
+dump_error(s, dump: failed to use lzo compression);
+goto out;
+}
+}
+
+lzo_bytep wrkmem;
+
+wrkmem = g_malloc(LZO1X_1_MEM_COMPRESS);
+
+len_buf_out_lzo = s-page_size + s-page_size / 16 + 64 + 3;
+#endif
+
+/* buf size for snappy */
+#ifdef CONFIG_SNAPPY
+len_buf_out_snappy = snappy_max_compressed_length(s-page_size);
+#endif
+
+/* get the biggest that can store all kinds of compressed page */
+len_buf_out = MAX(len_buf_out_zlib,
+MAX(len_buf_out_lzo, len_buf_out_snappy));
+
+buf_out = g_malloc(len_buf_out);
+
+/* get offset of page_desc and page_data in dump file */
+offset_desc = s-offset_page;
+offset_data = offset_desc + sizeof(page_desc_t) * s-num_dumpable;
+
+/*
+ * init zero page's page_desc and page_data, and all zero pages
+ * will use the same page_data
+ */
+pd_zero.size = s-page_size;
+pd_zero.flags = 0;
+pd_zero.offset = offset_data;
+pd_zero.page_flags = 0;
+memset(buf, 0, pd_zero.size);
+write_cache(s-page_data, buf, pd_zero.size);
+offset_data += pd_zero.size;
+
+for (pfn = 0; pfn  s-max_mapnr; pfn++) {
+/* check whether the page is dumpable */
+if (!is_bit_set(s-dump_bitmap2, pfn)) {
+continue;
+}
+
+memset(buf, 0, s-page_size);
+ret = readmem(buf, pfn_to_paddr(pfn, s-page_shift), s-page_size, s);
+if (ret  0) {
+dump_error(s, dump: failed to read memory );
+goto out;
+}
+
+/* check zero page */
+zero_page = is_zero_page(buf, s-page_size);
+if (zero_page) {
+write_cache(s-page_desc, pd_zero, sizeof(page_desc_t));
+} else {
+/*
+ * not zero page, then:
+ * 1. compress the page
+ * 2. write the compressed page into the cache of

[Qemu-devel] [PATCH v4 9/9] dump: Make kdump-compressed format available for 'dump-guest-memory'

2013-05-27 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Make monitor command 'dump-guest-memory' be able to dump in kdump-compressed
format. The command's usage:

  dump [-p] protocol [begin] [length] [format]

'format' is used to specified the format of vmcore and can be:
1. 'elf': ELF format, without compression
2. 'zlib': kdump-compressed format, with zlib-compressed
3. 'lzo': kdump-compressed format, with lzo-compressed
4. 'snappy': kdump-compressed format, with snappy-compressed
And without 'format' being set, it is same as 'elf'.

Note:
  1. The kdump-compressed format is readable only with the crash utility, and
 it can be smaller than the ELF format because of the compression support.
  2. The kdump-compressed format is the 5th edition.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Signed-off-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 configure|   50 
 dump.c   |  102 +-
 hmp-commands.hx  |   12 +++--
 hmp.c|   23 +-
 include/sysemu/dump_memory.h |1 +
 qapi-schema.json |   22 +-
 qmp-commands.hx  |6 ++-
 7 files changed, 197 insertions(+), 19 deletions(-)

diff --git a/configure b/configure
index eb74510..5f459a5 100755
--- a/configure
+++ b/configure
@@ -230,6 +230,8 @@ libusb=
 usb_redir=
 glx=
 zlib=yes
+lzo=no
+snappy=no
 guest_agent=yes
 want_tools=yes
 libiscsi=
@@ -902,6 +904,10 @@ for opt do
   ;;
   --disable-zlib-test) zlib=no
   ;;
+  --enable-lzo) lzo=yes
+  ;;
+  --enable-snappy) snappy=yes
+  ;;
   --enable-guest-agent) guest_agent=yes
   ;;
   --disable-guest-agent) guest_agent=no
@@ -1499,6 +1505,42 @@ fi
 libs_softmmu=$libs_softmmu -lz
 
 ##
+# lzo check
+
+if test $lzo != no ; then
+cat  $TMPC  EOF
+#include lzo/lzo1x.h
+int main(void) { lzo_version(); return 0; }
+EOF
+if compile_prog  -llzo2 ; then
+:
+else
+error_exit lzo check failed \
+Make sure to have the lzo libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -llzo2
+fi
+
+##
+# snappy check
+
+if test $snappy != no ; then
+cat  $TMPC  EOF
+#include snappy-c.h
+int main(void) { snappy_max_compressed_length(4096); return 0; }
+EOF
+if compile_prog  -lsnappy ; then
+:
+else
+error_exit snappy check failed \
+Make sure to have the snappy libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -lsnappy
+fi
+
+##
 # libseccomp check
 
 if test $seccomp != no ; then
@@ -3901,6 +3943,14 @@ if test $glx = yes ; then
   echo GLX_LIBS=$glx_libs  $config_host_mak
 fi
 
+if test $lzo = yes ; then
+  echo CONFIG_LZO=y  $config_host_mak
+fi
+
+if test $snappy = yes ; then
+  echo CONFIG_SNAPPY=y  $config_host_mak
+fi
+
 if test $libiscsi = yes ; then
   echo CONFIG_LIBISCSI=y  $config_host_mak
 fi
diff --git a/dump.c b/dump.c
index 17686ad..274249f 100644
--- a/dump.c
+++ b/dump.c
@@ -1036,6 +1036,16 @@ static int create_header64(DumpState *s)
 return 0;
 }
 
+static void get_max_mapnr(DumpState *s)
+{
+MemoryMapping *memory_mapping;
+
+QTAILQ_FOREACH(memory_mapping, s-list.head, next) {
+s-max_mapnr = paddr_to_pfn(memory_mapping-phys_addr +
+memory_mapping-length, s-page_shift);
+}
+}
+
 /*
  * gather data of header and sub header
  */
@@ -1404,12 +1414,14 @@ out:
 return ret;
 }
 
-static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
- int64_t begin, int64_t length, Error **errp)
+static int dump_init(DumpState *s, int fd, bool has_format,
+DumpGuestMemoryFormat format, bool paging, bool has_filter,
+int64_t begin, int64_t length, Error **errp)
 {
 CPUArchState *env;
 int nr_cpus;
 int ret;
+unsigned long tmp;
 
 if (runstate_is_running()) {
 vm_stop(RUN_STATE_SAVE_VM);
@@ -1464,6 +1476,56 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 qemu_get_guest_simple_memory_mapping(s-list);
 }
 
+/* init for kdump-compressed format */
+if (has_format  format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+switch (format) {
+case DUMP_GUEST_MEMORY_FORMAT_ZLIB:
+s-flag_compress = DUMP_DH_COMPRESSED_ZLIB;
+break;
+case DUMP_GUEST_MEMORY_FORMAT_LZO:
+s-flag_compress = DUMP_DH_COMPRESSED_LZO;
+break;
+case DUMP_GUEST_MEMORY_FORMAT_SNAPPY:
+s-flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
+break;
+default:
+s-flag_compress = 0;
+}
+
+s-nr_cpus = nr_cpus;
+s-page_size = PAGE_SIZE;
+s-page_shift = ffs(s-page_size) - 1;
+
+get_max_mapnr(s);
+
+tmp = divideup(divideup(s-max_mapnr,

Re: [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

2013-05-12 Thread qiaonuohan

Sorry, the cc list is wrong, I will resend the mails.
 -Original Message-
 From: qiaonuo...@cn.fujitsu.com [mailto:qiaonuo...@cn.fujitsu.com]
 Sent: Wednesday, May 08, 2013 9:54 AM
 To: qemu-devel@nongnu.org
 Cc:
 ebl...@redhat.com.berra...@redhat.com.afaer...@suse.de.d.hatayama@jp.fujitsu.c
 om.zhan...@cn.fujitsu.com; Qiao Nuohan
 Subject: [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in
 kdump-compressed format

 From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

 Hi, all

 The last version is here:
 http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg00803.html

 Command 'dump-guest-memory' was introduced to dump guest's memory. But the
 vmcore's format is only elf32 or elf64. The message is here:
 http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

 For migration, 'dump-guest-memory' is supposed to support compression feature.
 Because of the regression, the missing of compression feature, we post these
 patches to make 'dump-guest-memory' be able to dump guest's in kdump-
 compressed
 format. Then vmcore can be much smaller, and easily be delivered.

 The kdump-compressed format is *linux specific* *linux standard* crash dump
 format used in kdump framework. The kdump-compressed format is readable only
 with the crash utility, and it can be smaller than the ELF format because of
 the compression support.

 Note, similar to 'dump-guest-memory':
 1. The guest should be x86 or x86_64. The other arch is not supported now.
 2. If the OS is in the second kernel, gdb may not work well, and crash can
work by specifying '--machdep phys_addr=xxx' in the command line. The
reason is that the second kernel will update the page table, and we can
not get the page table for the first kernel.
 3. The cpu's state is stored in QEMU note.
 4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
available by default, and option '--enable-lzo' or '--enable-snappy'
should be used with configure to make lzo or snappy available.

 Changelog:
 Changes from v1 to v2:
 1. Address Eric  Daniel's comment: fix manner of  string copy.
 2. Address Eric's comment: replace reinventing new constants by using the
ready-made ones accoring.
 3. Address Andreas's comment: remove useless include.

 Qiao Nuohan (9):
   Add API to manipulate dump_bitmap
   Add API to manipulate cache_data
   Move includes and struct definition to dump.h
   Add API to create header of vmcore
   Add API to create data of dump bitmap
   Add API to create page
   Add API to free buf used by creating header, bitmap and page
   Add API to write header, bitmap and page into vmcore
   Make monitor command 'dump-guest-memory' dump in kdump-compressed
 format

  Makefile.target   |2 +-
  cache_data.c  |  121 +++
  configure |   50 +++
  dump.c|  877 ++--
 -
  dump_bitmap.c |  171 ++
  hmp-commands.hx   |   12 +-
  hmp.c |   23 ++-
  include/cache_data.h  |   56 
  include/dump_bitmap.h |   60 
  include/sysemu/dump.h |  176 ++
  qapi-schema.json  |   23 ++-
  qmp-commands.hx   |6 +-
  12 files changed, 1527 insertions(+), 50 deletions(-)
  create mode 100644 cache_data.c
  create mode 100644 dump_bitmap.c
  create mode 100644 include/cache_data.h
  create mode 100644 include/dump_bitmap.h

[Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Hi, all

The last version is here:
http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg00803.html

Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

For migration, 'dump-guest-memory' is supposed to support compression feature.
Because of the regression, the missing of compression feature, we post these
patches to make 'dump-guest-memory' be able to dump guest's in kdump-compressed
format. Then vmcore can be much smaller, and easily be delivered.

The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support.

Note, similar to 'dump-guest-memory':
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
   available by default, and option '--enable-lzo' or '--enable-snappy'
   should be used with configure to make lzo or snappy available.

Changelog:
Changes from v1 to v2:
1. Address Eric  Daniel's comment: fix manner of  string copy.
2. Address Eric's comment: replace reinventing new constants by using the
   ready-made ones accoring.
3. Address Andreas's comment: remove useless include.

Qiao Nuohan (9):
  Add API to manipulate dump_bitmap
  Add API to manipulate cache_data
  Move includes and struct definition to dump.h
  Add API to create header of vmcore
  Add API to create data of dump bitmap
  Add API to create page
  Add API to free buf used by creating header, bitmap and page
  Add API to write header, bitmap and page into vmcore
  Make monitor command 'dump-guest-memory' dump in kdump-compressed
format

 Makefile.target   |2 +-
 cache_data.c  |  121 +++
 configure |   50 +++
 dump.c|  877 ++---
 dump_bitmap.c |  171 ++
 hmp-commands.hx   |   12 +-
 hmp.c |   23 ++-
 include/cache_data.h  |   56 
 include/dump_bitmap.h |   60 
 include/sysemu/dump.h |  176 ++
 qapi-schema.json  |   23 ++-
 qmp-commands.hx   |6 +-
 12 files changed, 1527 insertions(+), 50 deletions(-)
 create mode 100644 cache_data.c
 create mode 100644 dump_bitmap.c
 create mode 100644 include/cache_data.h
 create mode 100644 include/dump_bitmap.h

[Qemu-devel] [PATCH 2/9 v2] Add API to manipulate cache_data

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Struct dump_bitmap is associated with a tmp file, and the tmp file can be used
to save data of page desc and page data in kdump-compressed format temporarily.
The following patch will use these function to get the data of page desc and
page data and cache them in tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 Makefile.target  |2 +-
 cache_data.c |  121 ++
 include/cache_data.h |   56 +++
 3 files changed, 178 insertions(+), 1 deletions(-)
 create mode 100644 cache_data.c
 create mode 100644 include/cache_data.h

diff --git a/Makefile.target b/Makefile.target
index 00d4f13..b579aff 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -112,7 +112,7 @@ obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
-obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o
+obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o cache_data.o
 obj-$(CONFIG_NO_GET_MEMORY_MAPPING) += memory_mapping-stub.o
 obj-$(CONFIG_NO_CORE_DUMP) += dump-stub.o
 LIBS+=$(libs_softmmu)
diff --git a/cache_data.c b/cache_data.c
new file mode 100644
index 000..469ed94
--- /dev/null
+++ b/cache_data.c
@@ -0,0 +1,121 @@
+/*
+ * QEMU cache data
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include qemu-common.h
+#include cache_data.h
+
+int init_cache_data(struct cache_data *cd, const char *filename)
+{
+int fd;
+char *tmpname;
+
+/* init the tmp file */
+tmpname = getenv(TMPDIR);
+if (!tmpname) {
+tmpname = (char *)P_tmpdir;
+}
+
+cd-file_name = (char *)g_strdup_printf(%s/%s, tmpname, filename);
+
+fd = mkstemp(cd-file_name);
+if (fd  0) {
+return -1;
+}
+
+cd-fd = fd;
+unlink(cd-file_name);
+
+/* init buf */
+cd-buf_size = BUFSIZE_CACHE_DATA;
+cd-cache_size = 0;
+cd-buf = g_malloc0(BUFSIZE_CACHE_DATA);
+
+cd-offset = 0;
+
+return 0;
+}
+
+int write_cache(struct cache_data *cd, void *buf, size_t size)
+{
+/*
+ * check if the space is enough to cache data, if not write cached
+ * data to the tmp file
+ */
+if (cd-cache_size + size  cd-buf_size) {
+if (lseek(cd-fd, cd-offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(cd-fd, cd-buf, cd-cache_size) != cd-cache_size) {
+return -1;
+}
+
+cd-offset += cd-cache_size;
+cd-cache_size = 0;
+}
+
+memcpy(cd-buf + cd-cache_size, buf, size);
+cd-cache_size += size;
+
+return 0;
+}
+
+int sync_cache(struct cache_data *cd)
+{
+/* no data is cached in cache_data */
+if (cd-cache_size == 0) {
+return 0;
+}
+
+if (lseek(cd-fd, cd-offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(cd-fd, cd-buf, cd-cache_size) != cd-cache_size) {
+return -1;
+}
+
+cd-offset += cd-cache_size;
+
+return 0;
+}
+
+int read_cache(struct cache_data *cd)
+{
+if (lseek(cd-fd, cd-offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (read(cd-fd, cd-buf, cd-cache_size) != cd-cache_size) {
+return -1;
+}
+
+cd-offset += cd-cache_size;
+
+return 0;
+}
+
+void free_cache_data(struct cache_data *cd)
+{
+if (cd) {
+if (cd-file_name) {
+g_free(cd-file_name);
+}
+
+if (cd-buf) {
+g_free(cd-buf);
+}
+
+g_free(cd);
+}
+}
diff --git a/include/cache_data.h b/include/cache_data.h
new file mode 100644
index 000..cbce60e
--- /dev/null
+++ b/include/cache_data.h
@@ -0,0 +1,56 @@
+/*
+ * QEMU cache data
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef CACHE_DATA_H
+#define CACHE_DATA_H
+
+#define BUFSIZE_CACHE_DATA  (4096 * 4)
+
+struct cache_data {
+int fd; /* fd of the tmp file used to store cache data */
+char *file_name;/* name of the tmp file */
+char *buf;  /* used to cache data */
+size_t buf_size;/* size of the buf */
+size_t cache_size;  /* size of cached data in buf */
+off_t offset;   /* offset of the tmp file */
+};
+
+/*
+ * create a tmp file used to store cache data, then init the buf
+ */
+int init_cache_data(struct cache_data *cd, const char *filename);
+
+/*
+ * write data to the tmp file, the data may first be cached in the buf of
+ * cache_data
+ */
+int write_cache(struct cache_data *cd, void *buf, size_t

[Qemu-devel] [PATCH 3/9 v2] Move includes and struct definition to dump.h

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Move includes and definition of struct DumpState into include/sysemu/dump.h.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c|   29 -
 include/sysemu/dump.h |   30 ++
 2 files changed, 30 insertions(+), 29 deletions(-)

diff --git a/dump.c b/dump.c
index c0d3da5..705c978 100644
--- a/dump.c
+++ b/dump.c
@@ -12,17 +12,7 @@
  */
 
 #include qemu-common.h
-#include elf.h
-#include cpu.h
-#include exec/cpu-all.h
-#include exec/hwaddr.h
-#include monitor/monitor.h
-#include sysemu/kvm.h
 #include sysemu/dump.h
-#include sysemu/sysemu.h
-#include sysemu/memory_mapping.h
-#include qapi/error.h
-#include qmp-commands.h
 
 static uint16_t cpu_convert_to_target16(uint16_t val, int endian)
 {
@@ -57,25 +47,6 @@ static uint64_t cpu_convert_to_target64(uint64_t val, int 
endian)
 return val;
 }
 
-typedef struct DumpState {
-ArchDumpInfo dump_info;
-MemoryMappingList list;
-uint16_t phdr_num;
-uint32_t sh_info;
-bool have_section;
-bool resume;
-size_t note_size;
-hwaddr memory_offset;
-int fd;
-
-RAMBlock *block;
-ram_addr_t start;
-bool has_filter;
-int64_t begin;
-int64_t length;
-Error **errp;
-} DumpState;
-
 static int dump_cleanup(DumpState *s)
 {
 int ret = 0;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b8c770f..b41469a 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -14,12 +14,42 @@
 #ifndef DUMP_H
 #define DUMP_H
 
+#include elf.h
+#include cpu.h
+#include exec/cpu-all.h
+#include exec/hwaddr.h
+#include monitor/monitor.h
+#include sysemu/kvm.h
+#include sysemu/sysemu.h
+#include sysemu/memory_mapping.h
+#include qapi/error.h
+#include qmp-commands.h
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or ELFDATA2MSB */
 int d_class;/* ELFCLASS32 or ELFCLASS64 */
 } ArchDumpInfo;
 
+typedef struct DumpState {
+ArchDumpInfo dump_info;
+MemoryMappingList list;
+uint16_t phdr_num;
+uint32_t sh_info;
+bool have_section;
+bool resume;
+size_t note_size;
+hwaddr memory_offset;
+int fd;
+
+RAMBlock *block;
+ram_addr_t start;
+bool has_filter;
+int64_t begin;
+int64_t length;
+Error **errp;
+} DumpState;
+
 int cpu_get_dump_info(ArchDumpInfo *info);
 ssize_t cpu_get_note_size(int class, int machine, int nr_cpus);
 
-- 
1.7.1

[Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Struct dump_bitmap is associated with a tmp file, and the tmp file can be used
to save data of bitmap in kdump-compressed format temporarily.
The following patch will use these functions to get the data of bitmap and cache
them into tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 Makefile.target   |2 +-
 dump_bitmap.c |  171 +
 include/dump_bitmap.h |   60 +
 3 files changed, 232 insertions(+), 1 deletions(-)
 create mode 100644 dump_bitmap.c
 create mode 100644 include/dump_bitmap.h

diff --git a/Makefile.target b/Makefile.target
index ce4391f..00d4f13 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -112,7 +112,7 @@ obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
-obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
+obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o
 obj-$(CONFIG_NO_GET_MEMORY_MAPPING) += memory_mapping-stub.o
 obj-$(CONFIG_NO_CORE_DUMP) += dump-stub.o
 LIBS+=$(libs_softmmu)
diff --git a/dump_bitmap.c b/dump_bitmap.c
new file mode 100644
index 000..d2f6c9f
--- /dev/null
+++ b/dump_bitmap.c
@@ -0,0 +1,171 @@
+/*
+ * QEMU dump bitmap
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan qiaonuo...@cn.fujitsu.com
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include qemu-common.h
+#include dump_bitmap.h
+
+int init_dump_bitmap(struct dump_bitmap *db, const char *filename)
+{
+int fd;
+char *tmpname;
+
+/* init the tmp file */
+tmpname = getenv(TMPDIR);
+if (!tmpname) {
+tmpname = (char *)P_tmpdir;
+}
+
+db-file_name = (char *)g_strdup_printf(%s/%s, tmpname, filename);
+
+fd = mkstemp(db-file_name);
+if (fd  0) {
+return -1;
+}
+
+db-fd = fd;
+unlink(db-file_name);
+
+/* num_block should be set to -1, for nothing is store in buf now */
+db-num_block = -1;
+memset(db-buf, 0, BUFSIZE_BITMAP);
+/* the tmp file starts to write at the very beginning */
+db-offset = 0;
+
+return 0;
+}
+
+int clear_dump_bitmap(struct dump_bitmap *db, size_t len_db)
+{
+int ret;
+char buf[BUFSIZE_BITMAP];
+off_t offset_bit;
+
+/* use buf to clear the tmp file block by block */
+memset(buf, 0, sizeof(buf));
+
+ret = lseek(db-fd, db-offset, SEEK_SET);
+if (ret  0) {
+return -1;
+}
+
+offset_bit = 0;
+while (offset_bit  len_db) {
+if (write(db-fd, buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+
+offset_bit += BUFSIZE_BITMAP;
+}
+
+return 0;
+}
+
+int set_dump_bitmap(struct dump_bitmap *db, unsigned long long pfn, int val)
+{
+int byte, bit;
+off_t old_offset, new_offset;
+old_offset = db-offset + BUFSIZE_BITMAP * db-num_block;
+new_offset = db-offset + BUFSIZE_BITMAP * (pfn / PFN_BUFBITMAP);
+
+/*
+ * if the block needed to be set is not same as the one cached in buf,
+ * write the block back to the tmp file, then cache new block to the buf
+ */
+if (0 = db-num_block  old_offset != new_offset) {
+if (lseek(db-fd, old_offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(db-fd, db-buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+}
+
+if (old_offset != new_offset) {
+if (lseek(db-fd, new_offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (read(db-fd, db-buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+
+db-num_block = pfn / PFN_BUFBITMAP;
+}
+
+/* get the exact place of the bit in the buf, set it */
+byte = (pfn % PFN_BUFBITMAP)  3;
+bit  = (pfn % PFN_BUFBITMAP)  7;
+if (val) {
+db-buf[byte] |= 1bit;
+} else {
+db-buf[byte] = ~(1bit);
+}
+
+return 0;
+}
+
+int sync_dump_bitmap(struct dump_bitmap *db)
+{
+off_t offset;
+offset = db-offset + BUFSIZE_BITMAP * db-num_block;
+
+/* db has not been used yet */
+if (db-num_block  0) {
+return 0;
+}
+
+if (lseek(db-fd, offset, SEEK_SET)  0) {
+return -1;
+}
+
+if (write(db-fd, db-buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+return -1;
+}
+
+return 0;
+}
+
+/*
+ * check if the bit is set to 1
+ */
+static inline int is_on(char *bitmap, int i)
+{
+return bitmap[i3]  (1  (i  7));
+}
+
+int is_bit_set(struct dump_bitmap *db, unsigned long long pfn)
+{
+off_t offset;
+
+/* cache the block pfn belongs to, then check the block */
+if (pfn == 0 || db-num_block != pfn/PFN_BUFBITMAP) {
+offset = db-offset + BUFSIZE_BITMAP * (pfn/PFN_BUFBITMAP);
+lseek(db-fd, offset,

[Qemu-devel] [PATCH 6/9 v2] Add API to create page

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Add API to get data of page desc and page data and save them into tmp files.
The following patch will use these functions to gather data of page desc and
page data, then write them into vmcore

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c|  259 +
 include/sysemu/dump.h |   32 ++
 2 files changed, 291 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 998d71e..ebfb190 100644
--- a/dump.c
+++ b/dump.c
@@ -877,6 +877,265 @@ static int create_dump_bitmap(DumpState *s)
 return 0;
 }
 
+/*
+ * create two tmpfile and save page_desc and page_data
+ */
+static int prepare_pages(DumpState *s)
+{
+int ret;
+struct cache_data *page_desc;
+struct cache_data *page_data;
+
+page_desc = g_malloc0(sizeof(struct cache_data));
+
+page_data = g_malloc0(sizeof(struct cache_data));
+
+ret = init_cache_data(page_desc, FILENAME_PAGE_DESC);
+if (ret  0) {
+dump_error(s, dump: failed to init page_desc.\n);
+return -1;
+}
+s-page_desc = page_desc;
+
+ret = init_cache_data(page_data, FILENAME_PAGE_DATA);
+if (ret  0) {
+dump_error(s, dump: failed to init page_desc.\n);
+return -1;
+}
+s-page_data = page_data;
+
+return 0;
+}
+
+/*
+ * memory should be read page by page, or it may exceed the boundary and
+ * fail to read
+ */
+static int readmem(void *bufptr, ram_addr_t addr, size_t size, DumpState *s)
+{
+RAMBlock *block;
+
+block = s-block;
+
+while (block) {
+if ((addr = block-offset) 
+(addr + size = block-offset + block-length)) {
+memcpy(bufptr, block-host + (addr - block-offset), size);
+return 0;
+} else {
+block = QTAILQ_NEXT(block, next);
+}
+}
+
+return -1;
+}
+
+/*
+ * check if the page is all 0
+ */
+static inline int is_zero_page(unsigned char *buf, long page_size)
+{
+size_t i;
+
+for (i = 0; i  page_size; i++) {
+if (buf[i]) {
+return 0;
+}
+}
+
+return 1;
+}
+
+static int create_pages(DumpState *s)
+{
+int ret;
+unsigned long long pfn;
+unsigned char buf[s-page_size];
+unsigned char *buf_out = NULL;
+unsigned long len_buf_out;
+unsigned long size_out;
+int zero_page;
+struct page_desc pd, pd_zero;
+off_t offset_desc, offset_data;
+unsigned long len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
+
+ret = 0;
+
+ret = prepare_pages(s);
+if (ret  0) {
+dump_error(s, dump: failed to prepare pages.\n);
+goto out;
+}
+
+/* init buf_out */
+len_buf_out_zlib = len_buf_out_lzo = len_buf_out_snappy = 0;
+
+/* buf size for zlib */
+len_buf_out_zlib = compressBound(s-page_size);
+
+/* buf size for lzo */
+#ifdef CONFIG_LZO
+if (s-flag_compress  DUMP_DH_COMPRESSED_LZO) {
+if (lzo_init() != LZO_E_OK) {
+ret = -1;
+dump_error(s, dump: failed to use lzo compression);
+goto out;
+}
+}
+
+lzo_bytep wrkmem;
+
+wrkmem = g_malloc(LZO1X_1_MEM_COMPRESS);
+
+len_buf_out_lzo = s-page_size + s-page_size / 16 + 64 + 3;
+#endif
+
+/* buf size for snappy */
+#ifdef CONFIG_SNAPPY
+len_buf_out_snappy = snappy_max_compressed_length(s-page_size);
+#endif
+
+/* get the biggest that can store all kinds of compressed page */
+len_buf_out = MAX(len_buf_out_zlib,
+MAX(len_buf_out_lzo, len_buf_out_snappy));
+
+buf_out = g_malloc(len_buf_out);
+
+/* get offset of page_desc and page_data in dump file */
+offset_desc = s-offset_page;
+offset_data = offset_desc + sizeof(page_desc_t) * s-num_dumpable;
+
+/*
+ * init zero page's page_desc and page_data, and all zero pages
+ * will use the same page_data
+ */
+pd_zero.size = s-page_size;
+pd_zero.flags = 0;
+pd_zero.offset = offset_data;
+pd_zero.page_flags = 0;
+memset(buf, 0, pd_zero.size);
+write_cache(s-page_data, buf, pd_zero.size);
+offset_data += pd_zero.size;
+
+for (pfn = 0; pfn  s-max_mapnr; pfn++) {
+/* check whether the page is dumpable */
+if (!is_bit_set(s-dump_bitmap2, pfn)) {
+continue;
+}
+
+memset(buf, 0, s-page_size);
+ret = readmem(buf, pfn_to_paddr(pfn, s-page_shift), s-page_size, s);
+if (ret  0) {
+dump_error(s, dump: failed to read memory );
+goto out;
+}
+
+/* check zero page */
+zero_page = is_zero_page(buf, s-page_size);
+if (zero_page) {
+write_cache(s-page_desc, pd_zero, sizeof(page_desc_t));
+} else {
+/*
+ * not zero page, then:
+ * 1. compress the page
+ * 2. write the compressed page into the cache of page_data
+

[Qemu-devel] [PATCH 8/9 v2] Add API to write header, bitmap and page into vmcore

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

The following patch will use these functions to write cached data into vmcore.
Header is cached in DumpState, and bitmap and page are cached in tmp files.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c|  259 +
 include/sysemu/dump.h |1 +
 2 files changed, 260 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 75ef032..c04e2ae 100644
--- a/dump.c
+++ b/dump.c
@@ -67,6 +67,36 @@ static void dump_error(DumpState *s, const char *reason)
 dump_cleanup(s);
 }
 
+/*
+ * write 'size' of 'c' to dump file
+ */
+static int fill_space(size_t size, int c, void *opaque)
+{
+DumpState *s = opaque;
+char tmpbuf[TMP_BUF_SIZE];
+size_t fill_size;
+size_t written_size;
+
+memset(tmpbuf, c, TMP_BUF_SIZE);
+
+while (size) {
+if (size  TMP_BUF_SIZE) {
+fill_size = TMP_BUF_SIZE;
+} else {
+fill_size = size;
+}
+
+size -= fill_size;
+
+written_size = qemu_write_full(s-fd, tmpbuf, fill_size);
+if (written_size != fill_size) {
+return -1;
+}
+}
+
+return 0;
+}
+
 static int fd_write_vmcore(void *buf, size_t size, void *opaque)
 {
 DumpState *s = opaque;
@@ -672,6 +702,235 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static int write_dump_header(DumpState *s)
+{
+int ret;
+void *dh;
+void *kh;
+
+/* write common header */
+dh = s-dh;
+
+if (s-dump_info.d_machine == EM_386) {
+ret = fd_write_vmcore(dh, sizeof(struct disk_dump_header32), s);
+if (ret  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+return -1;
+}
+} else {
+ret = fd_write_vmcore(dh, sizeof(struct disk_dump_header64), s);
+if (ret  0) {
+dump_error(s, dump: failed to write disk dump header.\n);
+return -1;
+}
+}
+
+/* fill gap between command header and sub header */
+ret = fill_space(s-offset_sub_header, 0, s);
+if (ret  0) {
+dump_error(s, dump: failed to fill the space between header and\
+sub header.\n);
+return -1;
+}
+
+/* write sub header */
+kh = s-kh;
+
+if (s-dump_info.d_machine == EM_386) {
+ret = fd_write_vmcore(kh, sizeof(struct kdump_sub_header32), s);
+if (ret  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+return -1;
+}
+} else {
+ret = fd_write_vmcore(kh, sizeof(struct kdump_sub_header64), s);
+if (ret  0) {
+dump_error(s, dump: failed to write kdump sub header.\n);
+return -1;
+}
+}
+
+/* write note */
+if (s-dump_info.d_class == ELFCLASS64) {
+if (write_elf64_notes(s)  0) {
+return -1;
+}
+} else {
+if (write_elf32_notes(s)  0) {
+return -1;
+}
+   }
+
+return 0;
+}
+
+static int write_dump_bitmap(DumpState *s)
+{
+struct cache_data bm;
+long buf_size;
+int ret;
+int no_bitmap;
+
+/* fill gap between header and dump_bitmap */
+ret = fill_space(s-offset_dump_bitmap, 0, s);
+if (ret  0) {
+dump_error(s, dump: failed to fill the space between header and\
+dump_bitmap.\n);
+goto out;
+}
+
+bm.buf = g_malloc0(TMP_BUF_SIZE);
+
+/* write dump_bitmap1 */
+bm.fd = s-dump_bitmap1-fd;
+no_bitmap = 1;
+
+again:
+buf_size = s-len_dump_bitmap / 2;
+bm.offset = 0;
+
+while (buf_size  0) {
+if (buf_size = TMP_BUF_SIZE) {
+bm.cache_size = TMP_BUF_SIZE;
+} else {
+bm.cache_size = buf_size;
+}
+
+ret = read_cache(bm);
+if (ret  0) {
+goto out;
+}
+
+ret = fd_write_vmcore(bm.buf, bm.cache_size, s);
+if (ret  0) {
+goto out;
+}
+
+buf_size -= bm.cache_size;
+}
+
+/* switch to dump_bitmap2 */
+if (no_bitmap == 1) {
+no_bitmap = 2;
+bm.fd = s-dump_bitmap2-fd;
+goto again;
+}
+
+return 0;
+
+out:
+if (bm.buf) {
+g_free(bm.buf);
+}
+
+return -1;
+}
+
+static int write_dump_pages(DumpState *s)
+{
+struct cache_data page;
+unsigned long long total_size;
+int is_page_desc;
+int ret;
+
+page.buf = g_malloc0(TMP_BUF_SIZE);
+
+/* write page_desc */
+is_page_desc = 1;
+total_size = s-page_desc_size;
+page.fd = s-page_desc-fd;
+page.offset = s-page_desc-offset;
+
+again:
+while (total_size  0) {
+if (total_size  TMP_BUF_SIZE) {
+page.cache_size = TMP_BUF_SIZE;
+} else {
+page.cache_size = total_size;
+}
+
+ret = read_cache(page);
+if (ret  0) {
+

[Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Make monitor command 'dump-guest-memory' dump in kdump-compressed format.
The command's usage:
  dump [-p] protocol [begin] [length] [format]
'format' is used to specified the format of vmcore and can be:
1. 'elf': ELF format, without compression
2. 'zlib': kdump-compressed format, with zlib-compressed
3. 'lzo': kdump-compressed format, with lzo-compressed
4. 'snappy': kdump-compressed format, with snappy-compressed
And without 'format' being set, vmcore will be in ELF format.

Note:
  1. The kdump-compressed format is readable only with the crash utility, and it
 can be smaller than the ELF format because of the compression support.
  2. The kdump-compressed format is the 5th edition.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Signed-off-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 configure |   50 +++
 dump.c|  106 +++-
 hmp-commands.hx   |   12 --
 hmp.c |   23 ++-
 include/sysemu/dump.h |8 
 qapi-schema.json  |   23 ++-
 qmp-commands.hx   |6 ++-
 7 files changed, 208 insertions(+), 20 deletions(-)

diff --git a/configure b/configure
index 9439f1c..b89beea 100755
--- a/configure
+++ b/configure
@@ -229,6 +229,8 @@ libusb=
 usb_redir=
 glx=
 zlib=yes
+lzo=no
+snappy=no
 guest_agent=yes
 want_tools=yes
 libiscsi=
@@ -898,6 +900,10 @@ for opt do
   ;;
   --disable-zlib-test) zlib=no
   ;;
+  --enable-lzo) lzo=yes
+  ;;
+  --enable-snappy) snappy=yes
+  ;;
   --enable-guest-agent) guest_agent=yes
   ;;
   --disable-guest-agent) guest_agent=no
@@ -1494,6 +1500,42 @@ fi
 libs_softmmu=$libs_softmmu -lz
 
 ##
+# lzo check
+
+if test $lzo != no ; then
+cat  $TMPC  EOF
+#include lzo/lzo1x.h
+int main(void) { lzo_version(); return 0; }
+EOF
+if compile_prog  -llzo2 ; then
+:
+else
+error_exit lzo check failed \
+Make sure to have the lzo libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -llzo2
+fi
+
+##
+# snappy check
+
+if test $snappy != no ; then
+cat  $TMPC  EOF
+#include snappy-c.h
+int main(void) { snappy_max_compressed_length(4096); return 0; }
+EOF
+if compile_prog  -lsnappy ; then
+:
+else
+error_exit snappy check failed \
+Make sure to have the snappy libs and headers installed.
+fi
+
+libs_softmmu=$libs_softmmu -lsnappy
+fi
+
+##
 # libseccomp check
 
 if test $seccomp != no ; then
@@ -3891,6 +3933,14 @@ if test $glx = yes ; then
   echo GLX_LIBS=$glx_libs  $config_host_mak
 fi
 
+if test $lzo = yes ; then
+  echo CONFIG_LZO=y  $config_host_mak
+fi
+
+if test $snappy = yes ; then
+  echo CONFIG_SNAPPY=y  $config_host_mak
+fi
+
 if test $libiscsi = yes ; then
   echo CONFIG_LIBISCSI=y  $config_host_mak
 fi
diff --git a/dump.c b/dump.c
index c04e2ae..74aa727 100644
--- a/dump.c
+++ b/dump.c
@@ -1027,6 +1027,16 @@ static int create_header64(DumpState *s)
 return 0;
 }
 
+static void get_max_mapnr(DumpState *s)
+{
+MemoryMapping *memory_mapping;
+
+QTAILQ_FOREACH(memory_mapping, s-list.head, next) {
+s-max_mapnr = paddr_to_pfn(memory_mapping-phys_addr +
+memory_mapping-length, s-page_shift);
+}
+}
+
 /*
  * gather data of header and sub header
  */
@@ -1395,12 +1405,14 @@ out:
 return ret;
 }
 
-static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
- int64_t begin, int64_t length, Error **errp)
+static int dump_init(DumpState *s, int fd, bool compress_format,
+DumpGuestMemoryFormat format, bool paging,
+bool has_filter, int64_t begin, int64_t length, Error 
**errp)
 {
 CPUArchState *env;
 int nr_cpus;
 int ret;
+unsigned long tmp;
 
 if (runstate_is_running()) {
 vm_stop(RUN_STATE_SAVE_VM);
@@ -1455,6 +1467,56 @@ static int dump_init(DumpState *s, int fd, bool paging, 
bool has_filter,
 qemu_get_guest_simple_memory_mapping(s-list);
 }
 
+/* init for kdump-compressed format */
+if (compress_format) {
+switch (format) {
+case DUMP_GUEST_MEMORY_FORMAT_ZLIB:
+s-flag_compress = DUMP_DH_COMPRESSED_ZLIB;
+break;
+case DUMP_GUEST_MEMORY_FORMAT_LZO:
+s-flag_compress = DUMP_DH_COMPRESSED_LZO;
+break;
+case DUMP_GUEST_MEMORY_FORMAT_SNAPPY:
+s-flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
+break;
+default:
+s-flag_compress = 0;
+}
+
+s-nr_cpus = nr_cpus;
+s-page_size = PAGE_SIZE;
+s-page_shift = ffs(s-page_size) - 1;
+
+get_max_mapnr(s);
+
+tmp = divideup(divideup(s-max_mapnr, CHAR_BIT), s-page_size);
+s-len_dump_bitmap = tmp *

[Qemu-devel] [PATCH 5/9 v2] Add API to create data of dump bitmap

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Add API to get data of the 1st and 2nd dump bitmap and save them into tmp files.
The following patch will use these functions to gather data of dump bitmap,
then write them into vmcore.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c|   98 +
 include/sysemu/dump.h |   10 +
 2 files changed, 108 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index bdebb33..998d71e 100644
--- a/dump.c
+++ b/dump.c
@@ -779,6 +779,104 @@ static int create_header(DumpState *s)
 return create_header64(s);
 }
 
+/*
+ * create two tmpfile and save 1st and 2nd bitmap separately
+ */
+static int prepare_dump_bitmap(DumpState *s)
+{
+int ret;
+struct dump_bitmap *db1;
+struct dump_bitmap *db2;
+
+db1 = g_malloc0(sizeof(struct dump_bitmap));
+
+db2 = g_malloc0(sizeof(struct dump_bitmap));
+
+ret = init_dump_bitmap(db1, FILENAME_BITMAP1);
+if (ret  0) {
+dump_error(s, dump: failed to init db1.);
+return -1;
+}
+s-dump_bitmap1 = db1;
+
+ret = init_dump_bitmap(db2, FILENAME_BITMAP2);
+if (ret  0) {
+dump_error(s, dump: failed to init db1.);
+return -1;
+}
+s-dump_bitmap2 = db2;
+
+return 0;
+}
+
+static int create_dump_bitmap(DumpState *s)
+{
+int ret;
+unsigned long long num_dumpable;
+MemoryMapping *memory_mapping;
+unsigned long long pfn_start, pfn_end, pfn;
+
+ret = prepare_dump_bitmap(s);
+if (ret  0) {
+dump_error(s, dump: failed to prepare dump_bitmap.\n);
+return -1;
+}
+
+ret = clear_dump_bitmap(s-dump_bitmap1, s-len_dump_bitmap / 2);
+if (ret  0) {
+dump_error(s, dump: failed to clear dump_bitmap1.\n);
+return -1;
+}
+
+ret = clear_dump_bitmap(s-dump_bitmap2, s-len_dump_bitmap / 2);
+if (ret  0) {
+dump_error(s, dump: failed to clear dump_bitmap2.\n);
+return -1;
+}
+
+/* write dump bitmap to tmp files */
+num_dumpable = 0;
+
+QTAILQ_FOREACH(memory_mapping, s-list.head, next) {
+pfn_start = paddr_to_pfn(memory_mapping-phys_addr, s-page_shift);
+pfn_end = paddr_to_pfn(memory_mapping-phys_addr +
+memory_mapping-length, s-page_shift);
+
+for (pfn = pfn_start; pfn  pfn_end; pfn++) {
+ret = set_dump_bitmap(s-dump_bitmap1, pfn, 1);
+if (ret  0) {
+dump_error(s, dump: failed to set dump_bitmap1.\n);
+return -1;
+}
+/* set dump_bitmap2, same as dump_bitmap1 */
+ret = set_dump_bitmap(s-dump_bitmap2, pfn, 1);
+if (ret  0) {
+dump_error(s, dump: failed to set dump_bitmap2.\n);
+return -1;
+}
+num_dumpable++;
+}
+}
+
+/* write cached data to tmp files */
+ret = sync_dump_bitmap(s-dump_bitmap1);
+if (ret  0) {
+dump_error(s, dump: failed to sync dump_bitmap1.\n);
+return -1;
+}
+
+ret = sync_dump_bitmap(s-dump_bitmap2);
+if (ret  0) {
+dump_error(s, dump: failed to sync dump_bitmap2.\n);
+return -1;
+}
+
+/* get the number of dumpable page */
+s-num_dumpable = num_dumpable;
+
+return 0;
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b7846c9..93c35bc 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -24,6 +24,7 @@
 #include sysemu/memory_mapping.h
 #include qapi/error.h
 #include qmp-commands.h
+#include dump_bitmap.h
 
 #include sys/utsname.h
 
@@ -32,8 +33,13 @@
 #define DISKDUMP_HEADER_BLOCKS  (1)
 #define PHYS_BASE   (0)
 #define DUMP_LEVEL  (1)
+#define ARCH_PFN_OFFSET (0)
+#define FILENAME_BITMAP1kdump_bitmap1_XX
+#define FILENAME_BITMAP2kdump_bitmap2_XX
 
 #define divideup(x, y)  (((x) + ((y) - 1)) / (y))
+#define paddr_to_pfn(X, page_shift) \
+(((unsigned long long)(X)  (page_shift)) - ARCH_PFN_OFFSET)
 
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
@@ -133,14 +139,18 @@ typedef struct DumpState {
 Error **errp;
 
 int page_size;
+int page_shift;
 unsigned long long max_mapnr;
 int nr_cpus;
 void *dh;
 void *kh;
+unsigned long long num_dumpable;
 off_t offset_sub_header;
 
 off_t offset_dump_bitmap;
 unsigned long len_dump_bitmap;
+struct dump_bitmap *dump_bitmap1;
+struct dump_bitmap *dump_bitmap2;
 
 off_t offset_page;
 } DumpState;
-- 
1.7.1

[Qemu-devel] [PATCH 4/9 v2] Add API to create header of vmcore

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

Add API to create header, sub header of vmcore in kdump-compressed format.

The data is store in struct DumpState.
The following patch will use this function to gather data of header, then
write them into vmcore.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c|  107 +
 include/sysemu/dump.h |   95 +++
 2 files changed, 202 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index 705c978..bdebb33 100644
--- a/dump.c
+++ b/dump.c
@@ -672,6 +672,113 @@ static ram_addr_t get_start_block(DumpState *s)
 return -1;
 }
 
+static int create_header32(DumpState *s)
+{
+struct  disk_dump_header32 *dh;
+struct  kdump_sub_header32 *kh;
+
+/* create common header, the version of kdump-compressed format is 5th */
+dh = g_malloc0(sizeof(struct disk_dump_header32));
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = 5;
+dh-block_size = s-page_size;
+dh-sub_hdr_size = sizeof(struct kdump_sub_header32) + s-note_size;
+dh-sub_hdr_size = divideup(dh-sub_hdr_size, dh-block_size);
+dh-max_mapnr = s-max_mapnr;
+dh-nr_cpus = s-nr_cpus;
+dh-bitmap_blocks = divideup(s-len_dump_bitmap, s-page_size);
+
+memcpy((dh-utsname.machine), i686, 4);
+
+s-dh = dh;
+
+/* create sub header */
+kh = g_malloc0(sizeof(struct kdump_sub_header32));
+
+kh-phys_base = PHYS_BASE;
+kh-dump_level = DUMP_LEVEL;
+
+kh-offset_note = DISKDUMP_HEADER_BLOCKS * dh-block_size +
+sizeof(struct kdump_sub_header32);
+kh-note_size = s-note_size;
+
+s-kh = kh;
+
+/* get gap between header and sub header */
+s-offset_sub_header = DISKDUMP_HEADER_BLOCKS * dh-block_size -
+sizeof(struct disk_dump_header32);
+
+/* get gap between header and dump_bitmap */
+s-offset_dump_bitmap = dh-sub_hdr_size * dh-block_size -
+(sizeof(struct kdump_sub_header32) + s-note_size);
+
+/* get offset of page desc */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + dh-sub_hdr_size +
+dh-bitmap_blocks) * dh-block_size;
+
+return 0;
+}
+
+static int create_header64(DumpState *s)
+{
+struct  disk_dump_header64 *dh;
+struct  kdump_sub_header64 *kh;
+
+/* create common header, the version of kdump-compressed format is 5th */
+dh = g_malloc0(sizeof(struct disk_dump_header64));
+
+strncpy(dh-signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+dh-header_version = 5;
+dh-block_size = s-page_size;
+dh-sub_hdr_size = sizeof(struct kdump_sub_header64) + s-note_size;
+dh-sub_hdr_size = divideup(dh-sub_hdr_size, dh-block_size);
+dh-max_mapnr = s-max_mapnr;
+dh-nr_cpus = s-nr_cpus;
+dh-bitmap_blocks = divideup(s-len_dump_bitmap, s-page_size);
+
+memcpy((dh-utsname.machine), x86_64, 6);
+
+s-dh = dh;
+
+/* create sub header */
+kh = g_malloc0(sizeof(struct kdump_sub_header64));
+
+kh-phys_base = PHYS_BASE;
+kh-dump_level = DUMP_LEVEL;
+
+kh-offset_note = DISKDUMP_HEADER_BLOCKS * dh-block_size +
+sizeof(struct kdump_sub_header64);
+kh-note_size = s-note_size;
+
+s-kh = kh;
+
+/* get gap between header and sub header */
+s-offset_sub_header = DISKDUMP_HEADER_BLOCKS * dh-block_size -
+sizeof(struct disk_dump_header64);
+
+/* get gap between header and dump_bitmap */
+s-offset_dump_bitmap = dh-sub_hdr_size * dh-block_size -
+(sizeof(struct kdump_sub_header64) + s-note_size);
+
+/* get offset of page desc */
+s-offset_page = (DISKDUMP_HEADER_BLOCKS + dh-sub_hdr_size +
+dh-bitmap_blocks) * dh-block_size;
+
+return 0;
+}
+
+/*
+ * gather data of header and sub header
+ */
+static int create_header(DumpState *s)
+{
+if (s-dump_info.d_machine == EM_386)
+return create_header32(s);
+else
+return create_header64(s);
+}
+
 static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
  int64_t begin, int64_t length, Error **errp)
 {
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b41469a..b7846c9 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -25,12 +25,95 @@
 #include qapi/error.h
 #include qmp-commands.h
 
+#include sys/utsname.h
+
+#define KDUMP_SIGNATURE KDUMP   
+#define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
+#define DISKDUMP_HEADER_BLOCKS  (1)
+#define PHYS_BASE   (0)
+#define DUMP_LEVEL  (1)
+
+#define divideup(x, y)  (((x) + ((y) - 1)) / (y))
+
 typedef struct ArchDumpInfo {
 int d_machine;  /* Architecture */
 int d_endian;   /* ELFDATA2LSB or

[Qemu-devel] [PATCH 7/9 v2] Add API to free buf used by creating header, bitmap and page

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

When calling create_header, create_dump_bitmap and create_pages, some memory 
spaces
are allocated. The following patch will use this function to free these spaces.

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
---
 dump.c |   19 +++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/dump.c b/dump.c
index ebfb190..75ef032 100644
--- a/dump.c
+++ b/dump.c
@@ -1252,6 +1252,25 @@ cleanup:
 return -1;
 }
 
+static void clean_state(DumpState *s)
+{
+if (s-dh) {
+g_free(s-dh);
+}
+
+if (s-kh) {
+g_free(s-kh);
+}
+
+free_dump_bitmap(s-dump_bitmap1);
+
+free_dump_bitmap(s-dump_bitmap2);
+
+free_cache_data(s-page_desc);
+
+free_cache_data(s-page_data);
+}
+
 void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
int64_t begin, bool has_length, int64_t length,
Error **errp)
-- 
1.7.1

[Qemu-devel] [PATCH] Remove twice include of qemu-common.h

2013-05-12 Thread qiaonuohan

From: Qiao Nuohan qiaonuo...@cn.fujitsu.com

This patch is used to remove twice include of qemu-common.h in
block/win32-aio.c

Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
---
 block/win32-aio.c |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/block/win32-aio.c b/block/win32-aio.c
index 5d0fbbf..fcb7c75 100644
--- a/block/win32-aio.c
+++ b/block/win32-aio.c
@@ -25,7 +25,6 @@
 #include qemu/timer.h
 #include block/block_int.h
 #include qemu/module.h
-#include qemu-common.h
 #include block/aio.h
 #include raw-aio.h
 #include qemu/event_notifier.h
-- 
1.7.1

Re: [Qemu-devel] [PATCH 1/9] Add API to manipulate dump_bitmap

2013-05-08 Thread qiaonuohan

 -Original Message-
 From: Eric Blake [mailto:ebl...@redhat.com]
 Sent: Wednesday, May 08, 2013 12:14 AM
 To: Qiao Nuohan
 Cc: qemu-devel@nongnu.org
 Subject: Re: [Qemu-devel] [PATCH 1/9] Add API to manipulate dump_bitmap

 On 05/07/2013 01:16 AM, Qiao Nuohan wrote:
  Struct dump_bitmap is associated with a tmp file, and the tmp file can be
 used
  to save data of bitmap in kdump-compressed format temporarily.
  The following patch will use these functions to get the data of bitmap and
 cache
  them into tmp files.

  Signed-off-by: Qiao Nuohan qiaonuo...@cn.fujitsu.com
  Reviewed-by: Zhang Xiaohe zhan...@cn.fujitsu.com
  ---

  +db-file_name = (char *)g_malloc(strlen(filename) + strlen(tmpname) +
 1);
  +
  +strcpy(db-file_name, tmpname);
  +strcat(db-file_name, /);
  +strcat(db-file_name, filename);

 Off-by-one buffer overflow, since you forgot space for the NUL byte.  We
 use C, not C++, so you don't need to cast the result of g_malloc().

I will fix it as Daniel suggested.

  +++ b/include/dump_bitmap.h
  @@ -0,0 +1,57 @@
  +/*
  + * QEMU dump bitmap
  + *
  + * Copyright Fujitsu, Corp. 2013
  + *
  + * Authors:
  + * Qiao Nuohan qiaonuo...@cn.fujitsu.com
  + *
  + * This work is licensed under the terms of the GNU GPL, version 2 or 
  later.
  + * See the COPYING file in the top-level directory.
  + *
  + */
  +

 No double-inclusion guard?

  +#define TMP_DIR /tmp

 Why not reuse P_tmpdir from stdio.h instead of reinventing a new name
 for this constant?

  +#define BITPERBYTE  (8)

 Why not use CHAR_BIT from limits.h instead of reinventing a new name
 for this constant?

  +#define BUFSIZE_BITMAP  (4096)
  +#define PFN_BUFBITMAP   (BITPERBYTE * BUFSIZE_BITMAP)
  +
  +struct dump_bitmap {
  +int fd; /* fd of the tmp file used to store dump
 bitmap */
  +int no_block;   /* number of block cached in buf */

 Trailing whitespace.  Run your patch series through scripts/checkpatch.pl.

 The name no_block sounds like there aren't any blocks.  You probably
 want the name num_block instead.

Got it.

Thanks for your comments.

 --
 Eric Blake   eblake redhat com+1-919-301-3266
 Libvirt virtualization library http://libvirt.org

Re: [Qemu-devel] [PATCH 9/9] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

2013-05-08 Thread qiaonuohan

 -Original Message-
 From: Eric Blake [mailto:ebl...@redhat.com]
 Sent: Wednesday, May 08, 2013 1:13 AM
 To: Qiao Nuohan
 Cc: qemu-devel@nongnu.org; Zhang Xiaohe
 Subject: Re: [Qemu-devel] [PATCH 9/9] Make monitor command 'dump-guest-memory'
 dump in kdump-compressed format

  +.params = [-p] filename [flags] [begin] [length],
   .help   = dump guest memory to file
  +  \n\t\t\t flags: the type of compression

 That documentation does nothing for me.  What types are valid?

  +++ b/qapi-schema.json
  @@ -2410,6 +2410,8 @@
   #2. fd: the protocol starts with fd:, and the following
 string
   #   is the fd's name.
   #
  +# @flags: #optional if specified, the format of dump file.
  +#

 Missing a (since 1.6) tag to declare that it was added after the fact.
  We probably also ought to solve the introspection issue before adding
 this feature, so that QMP clients like libvirt know when this optional
 parameter is available for use.

Got it.

   # @begin: #optional if specified, the starting physical address.
   #
   # @length: #optional if specified, the memory size, in bytes. If you don't
  @@ -2421,8 +2423,8 @@
   # Since: 1.2
   ##
   { 'command': 'dump-guest-memory',
  -  'data': { 'paging': 'bool', 'protocol': 'str', '*begin': 'int',
  -'*length': 'int' } }
  +  'data': { 'paging': 'bool', 'protocol': 'str', '*flags': 'int',
  +'*begin': 'int', '*length': 'int' } }

 EW - this is a LOUSY interface. Don't make '*flags' an 'int', and
 for that matter, don't name it 'flags' (that implies that we can
 bitwise-or multiple compressions together, but it really doesn't make
 sense to do both lzo and snappy at the same time - compression really
 only makes sense as a single format at a time).  Name it for what it
 represents (compression type), and provide an enum that lists the valid
 types.  Something like:

 { 'enum': 'DumpGuestMemoryFormat',
   'data': [ 'uncompressed', 'zlib', 'lzo', 'snappy' ] }

 { 'command': 'dump-guest-memory',
   'data': { '*format': 'DumpGuestMemoryFormat', ... }}

Thanks for your suggestion. I will fix it like:

{ 'enum': 'DumpCompressionFormat',
  'data': [ 'zlib', 'lzo', 'snappy' ] }

For zlib is treated as the default compression format, and
'uncompressed' won't be an option.

 --
 Eric Blake   eblake redhat com+1-919-301-3266
 Libvirt virtualization library http://libvirt.org

72 matches

Mail list logo