Re: JDK-8036003: Add variable not to separate debug information.
Hi, * Yasumasa Suenaga yasue...@gmail.com [2014-04-04 10:56]: I've succeeded to make binaries which are contained debuginfo as following: http://mail.openjdk.java.net/pipermail/build-dev/2014-March/012037.html $ make images STRIP_POLICY=no_strip POST_STRIP_CMD= I guess that we should run make above options to avoid this issue in currently. Thanks. I have verified that this works. Sorry about the delay in getting it pushed; I haven't had the time due to the security update and now rawhide being broken due to a GCC 4.9 bump. Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681
Re: JDK-8036003: Add variable not to separate debug information.
On 5/04/2014 12:55 AM, Yasumasa Suenaga wrote: Hi all, This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315551d634b7467f28788d90595b467ea5eb I updated OpenJDK8 to java-1.8.0-openjdk-1.8.0.0-0.34.b132.fc20 . However, debuginfo files are not contained ELF sections for debugging. (I checked libjvm.so.debug and libnio.so.debug with readelf -S) According to SPEC file of OpenJDK8, following options are passed to make: - make \ SCTP_WERROR= \ DEBUG_BINARIES=true \ FULL_DEBUG_SYMBOLS=0 \ STRIP_POLICY=none \ ALT_OBJCOPY=/does_not_exist \ LOG=trace \ all - I ran grep with DEBUG_BINARIES in jdk makefiles, however I could not find it. At least, DEBUG_BINARIES does not affect to jdk sources, and also does not affect to hotspot sources. DEBUG_BINARIES is used in the hotspot makefiles. ./linux/makefiles/gcc.make ifeq ($(DEBUG_BINARIES), true) CFLAGS += -g else David - I've succeeded to make binaries which are contained debuginfo as following: http://mail.openjdk.java.net/pipermail/build-dev/2014-March/012037.html $ make images STRIP_POLICY=no_strip POST_STRIP_CMD= I guess that we should run make above options to avoid this issue in currently. Thanks, Yasumasa On 03/04/2014 04:01 AM, Omair Majid wrote: * Andrew Haley a...@redhat.com [2014-03-03 04:43]: On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote: For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. That's a bug in Fedora's build. We should fix it. This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315551d634b7467f28788d90595b467ea5eb Thanks, Omair
Re: JDK-8036003: Add variable not to separate debug information.
Hi all, This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315551d634b7467f28788d90595b467ea5eb I updated OpenJDK8 to java-1.8.0-openjdk-1.8.0.0-0.34.b132.fc20 . However, debuginfo files are not contained ELF sections for debugging. (I checked libjvm.so.debug and libnio.so.debug with readelf -S) According to SPEC file of OpenJDK8, following options are passed to make: - make \ SCTP_WERROR= \ DEBUG_BINARIES=true \ FULL_DEBUG_SYMBOLS=0 \ STRIP_POLICY=none \ ALT_OBJCOPY=/does_not_exist \ LOG=trace \ all - I ran grep with DEBUG_BINARIES in jdk makefiles, however I could not find it. At least, DEBUG_BINARIES does not affect to jdk sources, and also does not affect to hotspot sources. I've succeeded to make binaries which are contained debuginfo as following: http://mail.openjdk.java.net/pipermail/build-dev/2014-March/012037.html $ make images STRIP_POLICY=no_strip POST_STRIP_CMD= I guess that we should run make above options to avoid this issue in currently. Thanks, Yasumasa On 03/04/2014 04:01 AM, Omair Majid wrote: * Andrew Haley a...@redhat.com [2014-03-03 04:43]: On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote: For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. That's a bug in Fedora's build. We should fix it. This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315551d634b7467f28788d90595b467ea5eb Thanks, Omair
Re: JDK-8036003: Add variable not to separate debug information.
Magnus, Not, we are not doing it now. But we should consider doing it in a future and therefore keep it in mind when designing option to choose debug symbol mode. -Dmitry On 2014-03-24 15:18, Magnus Ihse Bursie wrote: On 2014-03-21 10:36, Dmitry Samersoff wrote: (c) Compression mode: 1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file Is 2 something we're doing? I couldn't find any references to it in the code. Or is it something we're planning to do? /Magnus -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources.
Re: JDK-8036003: Add variable not to separate debug information.
On 2014-03-21 10:36, Dmitry Samersoff wrote: (c) Compression mode: 1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file Is 2 something we're doing? I couldn't find any references to it in the code. Or is it something we're planning to do? /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
I don't think this quite works as there are other variations not captured here. Rather than zipped it should just be external. Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between internal and external because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). But they are not independent on each other! Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items. And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3. And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version. And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works. Can you give a more concrete example of the variations of your aspects that you think my suggested solution would not capture? /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
On 21/03/2014 6:14 PM, Magnus Ihse Bursie wrote: I don't think this quite works as there are other variations not captured here. Rather than zipped it should just be external. Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between internal and external because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). But they are not independent on each other! Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items. Yes I just said that. :) And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3. But you can vary what is kept internally independent of what is made external. And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version. zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged. And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works. Umm we fully strip all our binaries in embedded. Can you give a more concrete example of the variations of your aspects that you think my suggested solution would not capture? I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants. If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped. Multiple inter-connected choices, not just three (or four if you add zipped) David /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
David, In practice, we don't have to much to keep internally. There are no reason to copy out some of .debug_* sections but keep other ones. So we have a matrix: (a) Strip mode: 1. full 2. keep symbols 3. keep symbols and .debug_* (b) Copy mode: 1. Copy all to ext file 2. Copy none to ext file (c) Compression mode: 1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file Where *a2 + b2 + c2* have perfect sense, *a1 + b1 + c3* have no sense etc. -Dmitry Refs: 1. https://sourceware.org/bugzilla/show_bug.cgi?id=11819 http://gcc.gnu.org/ml/gcc-patches/2013-04/msg01780.html -Dmitry On 2014-03-21 12:57, David Holmes wrote: On 21/03/2014 6:14 PM, Magnus Ihse Bursie wrote: I don't think this quite works as there are other variations not captured here. Rather than zipped it should just be external. Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between internal and external because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). But they are not independent on each other! Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items. Yes I just said that. :) And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3. But you can vary what is kept internally independent of what is made external. And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version. zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged. And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works. Umm we fully strip all our binaries in embedded. Can you give a more concrete example of the variations of your aspects that you think my suggested solution would not capture? I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants. If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped. Multiple inter-connected choices, not just three (or four if you add zipped) David /Magnus -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the source code.
Re: JDK-8036003: Add variable not to separate debug information.
On 21/03/2014 7:36 PM, Dmitry Samersoff wrote: David, In practice, we don't have to much to keep internally. There are no reason to copy out some of .debug_* sections but keep other ones. I'm not familiar with exactly what the different strip options do but the point is this is covered by the strip-policy. So we have a matrix: (a) Strip mode: 1. full 2. keep symbols 3. keep symbols and .debug_* (b) Copy mode: 1. Copy all to ext file 2. Copy none to ext file (c) Compression mode: 1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file Where *a2 + b2 + c2* have perfect sense, So now your compression mode may apply to either the external file or the original? That's even more permutations. *a1 + b1 + c3* have no sense etc. Why does full strip + copy-all + zip make no sense? It is exactly what we do with embedded builds. (Naturally you have to copy before you strip) David - -Dmitry Refs: 1. https://sourceware.org/bugzilla/show_bug.cgi?id=11819 http://gcc.gnu.org/ml/gcc-patches/2013-04/msg01780.html -Dmitry On 2014-03-21 12:57, David Holmes wrote: On 21/03/2014 6:14 PM, Magnus Ihse Bursie wrote: I don't think this quite works as there are other variations not captured here. Rather than zipped it should just be external. Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between internal and external because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). But they are not independent on each other! Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items. Yes I just said that. :) And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3. But you can vary what is kept internally independent of what is made external. And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version. zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged. And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works. Umm we fully strip all our binaries in embedded. Can you give a more concrete example of the variations of your aspects that you think my suggested solution would not capture? I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants. If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped. Multiple inter-connected choices, not just three (or four if you add zipped) David /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
David, My point was that we don't need in fine grained selection of elf sections on strip - three options are enough. Why does full strip + copy-all + zip make no sense? It is exactly what we do with embedded builds. (Naturally you have to copy before you strip) Sorry! it should be: full strip + copy-none + zip -Dmitry On 2014-03-21 15:13, David Holmes wrote: On 21/03/2014 7:36 PM, Dmitry Samersoff wrote: David, In practice, we don't have to much to keep internally. There are no reason to copy out some of .debug_* sections but keep other ones. I'm not familiar with exactly what the different strip options do but the point is this is covered by the strip-policy. So we have a matrix: (a) Strip mode: 1. full 2. keep symbols 3. keep symbols and .debug_* (b) Copy mode: 1. Copy all to ext file 2. Copy none to ext file (c) Compression mode: 1. none 2. per-section compression, SHF_GNU_COMPRESSED [1] 3. zip entire file Where *a2 + b2 + c2* have perfect sense, So now your compression mode may apply to either the external file or the original? That's even more permutations. *a1 + b1 + c3* have no sense etc. Why does full strip + copy-all + zip make no sense? It is exactly what we do with embedded builds. (Naturally you have to copy before you strip) David - -Dmitry Refs: 1. https://sourceware.org/bugzilla/show_bug.cgi?id=11819 http://gcc.gnu.org/ml/gcc-patches/2013-04/msg01780.html -Dmitry On 2014-03-21 12:57, David Holmes wrote: On 21/03/2014 6:14 PM, Magnus Ihse Bursie wrote: I don't think this quite works as there are other variations not captured here. Rather than zipped it should just be external. Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping th *Subject:* Fwd: Re: Compute sizes of classes loat is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between internal and external because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). But they are not independent on each other! Unless you generate debug symbols, you can't strip them, nor save them elsewhere. So generating debug symbols (your item #1) is a prerequisite for the rest of your items. Yes I just said that. :) And while, technically, you can save symbols externally, and at the same time keep them in the binary, that does not make sense. So, in a practical sense, you are going to do #2 if you are going to do #3. But you can vary what is kept internally independent of what is made external. And you can't zip external symbols unless you create a non-zipped version. And if you zip them, it does not make sense to keep the non-zipped version. zip vs non-zip is just an issue of disk space. It is not a fundamental configuration choice, just a variation on how external symbols are packaged. And yes, we do not strip all debug information when creating external debug info. But there seems to be no real use case (not even for IcedTea, as it turned out) to want a completely stripped binary, I don't think that's worth discussing much. That's just part of how the external debuginfo scheme works. Umm we fully strip all our binaries in embedded. Can you give a more concrete example of the variations of your aspects that you think my suggested solution would not capture? I think I already have. There aren't tree simple choices here, there are three aspects, each of which could have different variants. If I could draw a flow chart easily I would but basically if you generate debug symbols you can then select what symbols are kept internally (the strip policy) and what are put externally (objcopy options), then for the external symbol file you can choose zipped or unzipped. Multiple inter-connected choices, not just three (or four if you add zipped) David /Magnus -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the source code.
Re: JDK-8036003: Add variable not to separate debug information.
On 2014-03-19 15:57, Andrew Hughes wrote: I think that's something we must fix ourselves. What we really need from OpenJDK is a way to build with complete debuginfo in both binaries and jarfiles. This was my intent with #2. The jstack/jmap issue needs to be fixed by stripping less debuginfo post-build on libjvm.so. Ok, that sounds great. We provide a way to keep the debuginfo in the binaries, and then you can do your own magic, and everyone is happy. :) I'll re-target the original bug (JDK-8036003) so that it is about implementing such debuginfo handling instead. /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
On 18/03/2014 6:59 PM, Magnus Ihse Bursie wrote: On 2014-03-18 02:19, Andrew Hughes wrote: Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build. I'm completely thinking along your lines here. I think we should focus on the end result, not the specific implementations details. I've been thinking about exposing a configure option like this: --with-debug-symbols=none (for #1) --with-debug-symbols=internal (for #2) --with-debug-symbols=zipped (for #3) I don't think this quite works as there are other variations not captured here. Rather than zipped it should just be external. Whether the debuginfo files are zipped or not is then an additional build time option. Additionally we still have to be able to control the degree of stripping that is carried out. It doesn't make sense to me to try and enumerate all possible combinations as top-level configure options. Further, as Dan was saying, this doesn't capture the overlap between internal and external because we still leave some symbols internal even when creating the debuginfo file. So I don't think this proposed categorization works. We still have three aspects: - generating symbols into the object files - stripping symbols from the final binaries - saving symbols into an external form Each of which can potentally be varied (of course if you don't generate symbols in the obj files stripping and saving are non issues). David - but perhaps with better names. I also imagine there might be a use case for yet another option, #4. Debuggin information stripped from the binaries, but not zipped expressed like this --with-debug-symbols=external And then this would be the only user interface needed, and the rest of the current set of not-really-clear flags can be hidden inside the configure script. For compatibility reasons, the default would be --with-debug-symbols=zipped (which is called FDS here), just as we currently do, but distributions would just need to add a --with-debug-symbols=internal to get what they want. /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
On 03/18/2014 06:22 PM, Andrew Hughes wrote: The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Mmm, but maybe that will break things when debuginfo isn't installed. In fact, I already know that it has broken some things. So it's not ideal. Andrew.
Re: JDK-8036003: Add variable not to separate debug information.
On 2014-03-18 19:25, Andrew Haley wrote: On 03/18/2014 06:22 PM, Andrew Hughes wrote: The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Mmm, but maybe that will break things when debuginfo isn't installed. In fact, I already know that it has broken some things. So it's not ideal. Which case is it that will break? #3 or #2? What would an ideal solution be for you? /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
On 03/19/2014 01:51 PM, Magnus Ihse Bursie wrote: On 2014-03-18 19:25, Andrew Haley wrote: On 03/18/2014 06:22 PM, Andrew Hughes wrote: The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Mmm, but maybe that will break things when debuginfo isn't installed. In fact, I already know that it has broken some things. So it's not ideal. Which case is it that will break? #3 or #2? What would an ideal solution be for you? I think the problem was that jstack and jmap didn't work until the debuginfo package was installed. This tells me that the debuginfo stripping in our Linux build system is too aggressive for this purpose. I think that's something we must fix ourselves. What we really need from OpenJDK is a way to build with complete debuginfo in both binaries and jarfiles. Andrew.
Re: JDK-8036003: Add variable not to separate debug information.
- Original Message - On 03/19/2014 01:51 PM, Magnus Ihse Bursie wrote: On 2014-03-18 19:25, Andrew Haley wrote: On 03/18/2014 06:22 PM, Andrew Hughes wrote: The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Mmm, but maybe that will break things when debuginfo isn't installed. In fact, I already know that it has broken some things. So it's not ideal. Which case is it that will break? #3 or #2? What would an ideal solution be for you? I think the problem was that jstack and jmap didn't work until the debuginfo package was installed. This tells me that the debuginfo stripping in our Linux build system is too aggressive for this purpose. Yes, https://bugzilla.redhat.com/show_bug.cgi?id=1010786 The distro builds strip too much and remove the needed symtab from libjvm.so. I think that's something we must fix ourselves. What we really need from OpenJDK is a way to build with complete debuginfo in both binaries and jarfiles. This was my intent with #2. The jstack/jmap issue needs to be fixed by stripping less debuginfo post-build on libjvm.so. Andrew. -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07
Re: JDK-8036003: Add variable not to separate debug information.
On 2014-03-18 02:19, Andrew Hughes wrote: Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build. I'm completely thinking along your lines here. I think we should focus on the end result, not the specific implementations details. I've been thinking about exposing a configure option like this: --with-debug-symbols=none (for #1) --with-debug-symbols=internal (for #2) --with-debug-symbols=zipped (for #3) but perhaps with better names. I also imagine there might be a use case for yet another option, #4. Debuggin information stripped from the binaries, but not zipped expressed like this --with-debug-symbols=external And then this would be the only user interface needed, and the rest of the current set of not-really-clear flags can be hidden inside the configure script. For compatibility reasons, the default would be --with-debug-symbols=zipped (which is called FDS here), just as we currently do, but distributions would just need to add a --with-debug-symbols=internal to get what they want. /Magnus
Re: JDK-8036003: Add variable not to separate debug information.
- Original Message - On 2014-03-18 02:19, Andrew Hughes wrote: Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build. I'm completely thinking along your lines here. I think we should focus on the end result, not the specific implementations details. I've been thinking about exposing a configure option like this: --with-debug-symbols=none (for #1) --with-debug-symbols=internal (for #2) --with-debug-symbols=zipped (for #3) but perhaps with better names. I also imagine there might be a use case for yet another option, #4. Debuggin information stripped from the binaries, but not zipped expressed like this --with-debug-symbols=external And then this would be the only user interface needed, and the rest of the current set of not-really-clear flags can be hidden inside the configure script. For compatibility reasons, the default would be --with-debug-symbols=zipped (which is called FDS here), just as we currently do, but distributions would just need to add a --with-debug-symbols=internal to get what they want. /Magnus That sounds perfect. We already have to change numerous defaults; one more isn't an issue. The problem at the moment is that, at least in 7, I've had to patch the Makefiles to get what we want (i.e. making STRIP_POLICY=no_strip not create duplicate debuginfo in zipped debuginfo files) and that should be avoided if possible. -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07
Re: JDK-8036003: Add variable not to separate debug information.
On 3/17/14 7:19 PM, Andrew Hughes wrote: - Original Message - On 3/3/14 2:49 PM, Omair Majid wrote: * David Holmes david.hol...@oracle.com [2014-02-28 18:48]: There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short... At least to me, Full Debug Symbols suggests #1 (i.e. that symbols are generated in the binaries), not #2. That's why it sounded so odd to me when you suggested turning it off, when we discussed this before. It's also not clear why you'd want a situation where #3 would be turned off, but not #2, as you end up with two copies of the debug symbols. That's the problem I believe we have with our builds; we can turn the stripping off, but then we end up with duplicate debug information. Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. The way Oracle does it is complete debug info in the separate files and partial debug info left in the original binaries so you get symbol names in stack traces for those cases where the full debug info bundles are not available. I'm not clear whether we need a 4th alternative or if #3 covers this case. Dan It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build.
Re: JDK-8036003: Add variable not to separate debug information.
- Original Message - On 3/17/14 7:19 PM, Andrew Hughes wrote: - Original Message - On 3/3/14 2:49 PM, Omair Majid wrote: * David Holmes david.hol...@oracle.com [2014-02-28 18:48]: There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short... At least to me, Full Debug Symbols suggests #1 (i.e. that symbols are generated in the binaries), not #2. That's why it sounded so odd to me when you suggested turning it off, when we discussed this before. It's also not clear why you'd want a situation where #3 would be turned off, but not #2, as you end up with two copies of the debug symbols. That's the problem I believe we have with our builds; we can turn the stripping off, but then we end up with duplicate debug information. Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. The way Oracle does it is complete debug info in the separate files and partial debug info left in the original binaries so you get symbol names in stack traces for those cases where the full debug info bundles are not available. I'm not clear whether we need a 4th alternative or if #3 covers this case. The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Dan It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build. -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07
Re: JDK-8036003: Add variable not to separate debug information.
On 3/18/14 12:22 PM, Andrew Hughes wrote: - Original Message - On 3/17/14 7:19 PM, Andrew Hughes wrote: - Original Message - On 3/3/14 2:49 PM, Omair Majid wrote: * David Holmes david.hol...@oracle.com [2014-02-28 18:48]: There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short... At least to me, Full Debug Symbols suggests #1 (i.e. that symbols are generated in the binaries), not #2. That's why it sounded so odd to me when you suggested turning it off, when we discussed this before. It's also not clear why you'd want a situation where #3 would be turned off, but not #2, as you end up with two copies of the debug symbols. That's the problem I believe we have with our builds; we can turn the stripping off, but then we end up with duplicate debug information. Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. The way Oracle does it is complete debug info in the separate files and partial debug info left in the original binaries so you get symbol names in stack traces for those cases where the full debug info bundles are not available. I'm not clear whether we need a 4th alternative or if #3 covers this case. The intent was for #3 to cover this case (i.e. whatever Oracle does now) and for #2 to be what the GNU/Linux distributions want (i.e. binaries with all debuginfo generated and left intact so they can do their own stripping). Very cool. Thanks for the clarification. Dan Dan It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build.
Re: JDK-8036003: Add variable not to separate debug information.
- Original Message - On 3/3/14 2:49 PM, Omair Majid wrote: * David Holmes david.hol...@oracle.com [2014-02-28 18:48]: There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short... At least to me, Full Debug Symbols suggests #1 (i.e. that symbols are generated in the binaries), not #2. That's why it sounded so odd to me when you suggested turning it off, when we discussed this before. It's also not clear why you'd want a situation where #3 would be turned off, but not #2, as you end up with two copies of the debug symbols. That's the problem I believe we have with our builds; we can turn the stripping off, but then we end up with duplicate debug information. Do we need more than just the following three alternatives? #1. No debugging information at all. #2. Debugging information left in the original binaries. #3. Debugging information stripped from the binaries and zipped in separate files. It sounds to me like Oracle want #3, while distros want #2 and I imagine some end users may just want #1 for a faster, smaller build. -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07
Re: JDK-8036003: Add variable not to separate debug information.
On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote: For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. That's a bug in Fedora's build. We should fix it. Andrew.
Re: JDK-8036003: Add variable not to separate debug information.
On 03/01/2014 11:08 PM, Mike Duigou wrote: Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. Exactly. OpenJDK generating separate debuginfo files is very inconvenient for RPM builds. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. The OpenJDK build can't have any idea of what debuginfo files the distro runtime requires, and indeed this may change. There is more than one mechanism for finding debuginfo. Separating debuginfo is not something that we want the OpenJDK build to do, and the most useful thing that we could have is a switch to turn all of OpenJDK's stripping and separate debuginfo off. I expect the same applies to all distros. Andrew.
Re: JDK-8036003: Add variable not to separate debug information.
Hi David, Moving forward I think we would need to expose the strip-policy via a configure option and have that pass STRIP_POLICY through to both hotspot and the Images target. I want you to do so :-) Additionally, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) I think that FDS should be separated 1 from 2. That is more useful for building OpenJDK. That said, as per the above I think you can achieve what you want by enabling FDS but set STRIP_POLICY to none, and set POST_STRIP_CMD to empty. I tried: $ make images STRIP_POLICY=no_strip POST_STRIP_CMD= I was able to get binaries which is included debug information. Thanks, Yasumasa On 2014/03/03 13:39, David Holmes wrote: Hi, On 2/03/2014 12:07 AM, Yasumasa Suenaga wrote: Hi David, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. Okay. That would seem potentially inconvenient but not really critical. You simply enable FDS (the mere existence of the debuginfo files shouldn't cause a problem). Then you only need to address #3 - strip-policy. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. Also there may well be differences between how things are handled on the JDK side and hotspot side. libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . Other native library is built with make/common/NativeCompilation.gmk . Additionally strip command is executed in image target which is defined in jdk/make/Images.gmk . This strip ignores STRIP_POLICY . Thus ELF binaries which are contained in JDK/JRE images are stripped when we execute images or all target. strip is also applied to the JVM libraries as part of the hotspot build. What occurs in the JDK images is additional to that.The hotspot build honors the STRIP_POLICY. It appears that STRIP_POLICY is not considered in the new build, though it was for the old. But perhaps I'm missing it somewhere in the autoconf settings? That aside it seems that you should be able to set POST_STRIP_CMD to avoid stripping. Moving forward I think we would need to expose the strip-policy via a configure option and have that pass STRIP_POLICY through to both hotspot and the Images target. So I change: 1. Separating to add -g option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY. As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. As Mike said there is no chance of any changes to this getting into 8 GA. But if these are your own builds I'm not sure that is an issue anyway. That said, as per the above I think you can achieve what you want by enabling FDS but set STRIP_POLICY to none, and set POST_STRIP_CMD to empty. Thanks, David Thanks, Yasumasa On 2014/03/01 8:47, David Holmes wrote: On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: The proper way to fix this is to disable FDS. Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. -g is needed to generate debuginfo. There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Also there may well be differences between how things are handled on the JDK side and hotspot side. I will try to look closer if I get a chance but my time is limited at the moment. David Thanks, Yasumasa On 2014/02/28 23:04, Daniel D. Daugherty wrote: The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information.
Re: JDK-8036003: Add variable not to separate debug information.
Hi Andrew, Separating debuginfo is not something that we want the OpenJDK build to do, and the most useful thing that we could have is a switch to turn all of OpenJDK's stripping and separate debuginfo off. I expect the same applies to all distros. My patch provides SEPARATED_DEBUGINFO_FILES as a switch of separating debuginfo files, and affects STRIP_POILCY in HotSpot and other ELF binaries. I expect the same applies to all distros. We need to pass ' STRIP_POLICY=no_strip POST_STRIP_CMD= ' to make command as David said. And we need to pick up binaries which are NOT named .debuginfo or .diz suffix. Thanks, Yasumasa On 2014/03/03 18:46, Andrew Haley wrote: On 03/01/2014 11:08 PM, Mike Duigou wrote: Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. Exactly. OpenJDK generating separate debuginfo files is very inconvenient for RPM builds. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. The OpenJDK build can't have any idea of what debuginfo files the distro runtime requires, and indeed this may change. There is more than one mechanism for finding debuginfo. Separating debuginfo is not something that we want the OpenJDK build to do, and the most useful thing that we could have is a switch to turn all of OpenJDK's stripping and separate debuginfo off. I expect the same applies to all distros. Andrew.
Re: JDK-8036003: Add variable not to separate debug information.
I think we're having a problem with not all replies making it to all the aliases. Yasumasa's reply to David below that Mike is replying to did not arrive on any of the aliases that I'm on... Folks need to remember to reply to all of the aliases... On 3/1/14 4:08 PM, Mike Duigou wrote: On Mar 1 2014, at 06:07 , Yasumasa Suenaga y...@ysfactory.dip.jp wrote: Hi David, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. Bit of history here. When FDS is disabled, the build system is supposed to go back to what was there prior to the FDS project. On Linux, one of the debug configs didn't actually have any debugging flags enabled (debug or fastdebug, I don't remember which). So when I implemented FDS, disabling FDS meant no debug info. Bug in the original? You betcha. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. Some comments: - Are there important tools that cannot deal with external debuginfo files? Not that I'm aware of. There have been bugs in the way gobjcopy is extracting the debuginfo into the separate files, but no complete failures as far as I know... If we can put debuginfo into external files why would we ever want unstripped binaries? We deliver minimal stripped binaries so that can have stack traces with some amount of information in them when the debuginfo files are not available. We have not yet figured out how to implement minimal stripping on Windows so Windows hs_err_pid files have no symbols in them when the debuginfo bundles are not present. Is strip policy really necessary? Yes. STRIP_POLICY allows folks to configure what they want for the build that they are doing. It is also meant to map to the Oracle policy where we were granted a waiver to deliver minimal symbols in FDS project. Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. No you don't want to strip everything. See above aboutminimal info in stack traces. Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of -g 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use. Hopefully I didn't get any of those steps in the wrong order. What are the you-cant-get-there-from-here gaps and breakdowns from implementing this process? - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. It's STRIP_POLICY=min_strip that leaves the minimal debug info attached to the binaries and the more complete debug info in left in the debuginfo file(s). And I'll disagree about only wanting abbreviated debug info in the separate debuginfo files. There are four basic alternatives for levels of debug info: 1. (-g0) No debug info whatsoever. Only exported functions and globals will have names. 2. (-g1) Abbreviated debug info. All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 3. (-g) Default debugging info. Adds macro expansions and local variable names. Suitable for source level debugging. 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated
Re: JDK-8036003: Add variable not to separate debug information.
* Andrew Haley a...@redhat.com [2014-03-03 04:43]: On 02/28/2014 09:18 AM, Yasumasa Suenaga wrote: For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. That's a bug in Fedora's build. We should fix it. This should fix it: http://pkgs.fedoraproject.org/cgit/java-1.8.0-openjdk.git/commit/?id=1734315551d634b7467f28788d90595b467ea5eb Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681
Re: JDK-8036003: Add variable not to separate debug information.
Hi, * Daniel D. Daugherty daniel.daughe...@oracle.com [2014-03-03 09:57]: On 3/1/14 4:08 PM, Mike Duigou wrote: If we can put debuginfo into external files why would we ever want unstripped binaries? We deliver minimal stripped binaries so that can have stack traces with some amount of information in them when the debuginfo files are not available. We have not yet figured out how to implement minimal stripping on Windows so Windows hs_err_pid files have no symbols in them when the debuginfo bundles are not present. Am I reading it right that when tools (like RPM) strip the debug information from these binaries, hotspot will break completely when it tries to generate a stack trace on crash? Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681
Re: JDK-8036003: Add variable not to separate debug information.
On 3/3/14 12:11 PM, Omair Majid wrote: Hi, * Daniel D. Daugherty daniel.daughe...@oracle.com [2014-03-03 09:57]: On 3/1/14 4:08 PM, Mike Duigou wrote: If we can put debuginfo into external files why would we ever want unstripped binaries? We deliver minimal stripped binaries so that can have stack traces with some amount of information in them when the debuginfo files are not available. We have not yet figured out how to implement minimal stripping on Windows so Windows hs_err_pid files have no symbols in them when the debuginfo bundles are not present. Am I reading it right that when tools (like RPM) strip the debug information from these binaries, hotspot will break completely when it tries to generate a stack trace on crash? Not quite. If you follow this process: - use gobjcopy to copy debug info from libjvm.so - libjvm.debuginfo - use gobjcopy to tell libjvm.so that debug info is found in libjvm.debuginfo - strip libjvm.so (I don't remember the strip everything option) then you can still get symbols in your crashes as long as you have BOTH libjvm.so and libjvm.debuginfo. However, if you are in a situation where you don't have libjvm.debuginfo, then you do not get symbols in your crashes. When I did the Full Debug Symbols (FDS) project, I copied complete debug info from libjvm.so - libjvm.debuginfo and left minimal debug info in libjvm.so so that you could still get symbols on crashes on Linux and Solaris when the libjvm.debuginfo files were not present. Dan Thanks, Omair
Re: JDK-8036003: Add variable not to separate debug information.
* David Holmes david.hol...@oracle.com [2014-02-28 18:48]: There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. What do folks think of modifying the OPENJDK build to use this as the settings for the options above by default. 1. Yes 2. No 3. No Also there may well be differences between how things are handled on the JDK side and hotspot side. Is there any chance of this becoming consistent between hotspot and JDK? It would be much nicer if concepts like strip-policy were consistent between hotspot and the rest of the JDK. Thanks, Omair -- PGP Key: 66484681 (http://pgp.mit.edu/) Fingerprint = F072 555B 0A17 3957 4E95 0056 F286 F14F 6648 4681
Re: JDK-8036003: Add variable not to separate debug information.
On 3/3/14 2:49 PM, Omair Majid wrote: * David Holmes david.hol...@oracle.com [2014-02-28 18:48]: There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Any chance we could clean up the names too? It's not obvious to me why FDS means 'generating debuginfo files'. FDS stands for Full Debug Symbols and is defined that way in quite a few Makefiles... We just call it FDS for short... What do folks think of modifying the OPENJDK build to use this as the settings for the options above by default. 1. Yes 2. No 3. No Whatever we do here has be architecturallyconsistent across the various OpenJDK platforms. The above settings would not work well on non-Linux platforms so I'm not in favor of doing that. However, I don't have a problem with providing support for config'ing a build to work with the downstream Linux repos... Also there may well be differences between how things are handled on the JDK side and hotspot side. Is there any chance of this becoming consistent between hotspot and JDK? It would be much nicer if concepts like strip-policy were consistent between hotspot and the rest of the JDK. When I did the Full Debug Symbols (FDS) project, I made it architecturally consistent between the HotSpot and JDK sides. Since the new build system has arrived, something of the consistency has been broken and I haven't been back on the JDK side to see what happened... Dan Thanks, Omair
Re: JDK-8036003: Add variable not to separate debug information.
HI Mike, - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of -g 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use. rpmbuild will execute 2 to 4 implicitly, Modern ELF binary has .note.gnu.build-id section. Debugging tools (e.g. GDB and SA in JDK) use this section for finding debuginfo files. If OpenJDK8 which is shipped by RPM is crashed and we want to analyze core image, we have to merge debuginfo and executable at first (e.g. eu-unstrip command). GDB and SA can find valid debuginfo through .note.gnu.debug-id or .gnu_debuglink section. But these tools can find /usr/lib/debug/path to executables/executable name.debug . They are cannot find *.debuginfo which are made by makefiles in OpenJDK tree. Yasumasa On 2014/03/02 8:08, Mike Duigou wrote: On Mar 1 2014, at 06:07 , Yasumasa Suenagay...@ysfactory.dip.jp wrote: Hi David, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. Some comments: - Are there important tools that cannot deal with external debuginfo files? If we can put debuginfo into external files why would we ever want unstripped binaries? Is strip policy really necessary? Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of -g 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use. Hopefully I didn't get any of those steps in the wrong order. What are the you-cant-get-there-from-here gaps and breakdowns from implementing this process? - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. There are four basic alternatives for levels of debug info: 1. (-g0) No debug info whatsoever. Only exported functions and globals will have names. 2. (-g1) Abbreviated debug info. All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 3. (-g) Default debugging info. Adds macro expansions and local variable names. Suitable for source level debugging. 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components. - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms? So I change: 1. Separating to add -g option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY. As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too
Re: JDK-8036003: Add variable not to separate debug information.
Hi, On 2/03/2014 12:07 AM, Yasumasa Suenaga wrote: Hi David, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. Okay. That would seem potentially inconvenient but not really critical. You simply enable FDS (the mere existence of the debuginfo files shouldn't cause a problem). Then you only need to address #3 - strip-policy. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. Also there may well be differences between how things are handled on the JDK side and hotspot side. libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . Other native library is built with make/common/NativeCompilation.gmk . Additionally strip command is executed in image target which is defined in jdk/make/Images.gmk . This strip ignores STRIP_POLICY . Thus ELF binaries which are contained in JDK/JRE images are stripped when we execute images or all target. strip is also applied to the JVM libraries as part of the hotspot build. What occurs in the JDK images is additional to that.The hotspot build honors the STRIP_POLICY. It appears that STRIP_POLICY is not considered in the new build, though it was for the old. But perhaps I'm missing it somewhere in the autoconf settings? That aside it seems that you should be able to set POST_STRIP_CMD to avoid stripping. Moving forward I think we would need to expose the strip-policy via a configure option and have that pass STRIP_POLICY through to both hotspot and the Images target. So I change: 1. Separating to add -g option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY. As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. As Mike said there is no chance of any changes to this getting into 8 GA. But if these are your own builds I'm not sure that is an issue anyway. That said, as per the above I think you can achieve what you want by enabling FDS but set STRIP_POLICY to none, and set POST_STRIP_CMD to empty. Thanks, David Thanks, Yasumasa On 2014/03/01 8:47, David Holmes wrote: On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: The proper way to fix this is to disable FDS. Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. -g is needed to generate debuginfo. There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Also there may well be differences between how things are handled on the JDK side and hotspot side. I will try to look closer if I get a chance but my time is limited at the moment. David Thanks, Yasumasa On 2014/02/28 23:04, Daniel D. Daugherty wrote: The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in
Re: JDK-8036003: Add variable not to separate debug information.
Hi David, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. Also there may well be differences between how things are handled on the JDK side and hotspot side. libjvm, libjsig. libsaproc are built with each makefiles in hotspot/make . Other native library is built with make/common/NativeCompilation.gmk . Additionally strip command is executed in image target which is defined in jdk/make/Images.gmk . This strip ignores STRIP_POLICY . Thus ELF binaries which are contained in JDK/JRE images are stripped when we execute images or all target. So I change: 1. Separating to add -g option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY. As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. Thanks, Yasumasa On 2014/03/01 8:47, David Holmes wrote: On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: The proper way to fix this is to disable FDS. Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. -g is needed to generate debuginfo. There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Also there may well be differences between how things are handled on the JDK side and hotspot side. I will try to look closer if I get a chance but my time is limited at the moment. David Thanks, Yasumasa On 2014/02/28 23:04, Daniel D. Daugherty wrote: The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable SEPARATED_DEBUGINFO_FILES . If we pass SEPARATED_DEBUGINFO_FILES=false to make command, make does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add STRIP_POLICY=no_strip to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like --disable-separated-debug-info . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is
Re: JDK-8036003: Add variable not to separate debug information.
On Mar 1 2014, at 06:07 , Yasumasa Suenaga y...@ysfactory.dip.jp wrote: Hi David, 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Currently, 1. depends 2. If FDS set to disable, debug symbols (-g) are not generated. SEPARATED_DEBUGINFO_FILES which my patch provides can separate them. I think David's list (kudos) is very helpful as I haven't seen the requirements articulated this clearly anywhere else. Some comments: - Are there important tools that cannot deal with external debuginfo files? If we can put debuginfo into external files why would we ever want unstripped binaries? Is strip policy really necessary? Strip policy would seem to only be necessary if there are tools which can't handle external debuginfo. Assuming that everything can use external debuginfo then the policy would be to strip everything. Do I understand correctly that rpmbuild can only deal with unstripped binaries and generates the stripped rpm package and debuginfo package. It sounds kind of strange that find-debuginfo.sh wouldn't be able to use already generated debuginfo files. - I would expect that the flow is something like an extended version of https://blogs.oracle.com/dbx/entry/creating_separate_debug_info : 1. Compile source files with some form of -g 2. Create separate debug files for object files. 3. Strip object files. 4. Add gnu_debuglink to object files. 5. Link library or executable which should carry along links to debuginfo. 6a. Debugging, testing, profiling, etc. by developers. 6b. Packaging for program bits and debuginfo. 7. Testing and Use. Hopefully I didn't get any of those steps in the wrong order. What are the you-cant-get-there-from-here gaps and breakdowns from implementing this process? - Whether for the FDS debug symbols or RPM debuginfo packaging it seems that the default debug level isn't quite right. I believe that in both cases what's wanted is abbreviated debug info, mostly function names and line number tables, for building stack traces. This is not the debug info that is currently generated. There are four basic alternatives for levels of debug info: 1. (-g0) No debug info whatsoever. Only exported functions and globals will have names. 2. (-g1) Abbreviated debug info. All functions have names and line number information. This seems to be what is needed by both FDS and RPM debuginfo files to produce nice stack traces. 3. (-g) Default debugging info. Adds macro expansions and local variable names. Suitable for source level debugging. 4. (-g3 or -gdwarf-4 -fvar-tracking-assignments). Full debugging info. Most suitable for source level debugging including debugging of optimized code. It is not clear that anyone would want to build the entire product with this option. Compilations are currently done with a mix of -g, -g1, and -gstabs. It would be good to rationalize this somewhat and provide a mechanism for developers to tweak generated debugging output for files or components. - Eventual alignment with http://gcc.gnu.org/wiki/DebugFission on some platforms? So I change: 1. Separating to add -g option to compiler from executing objcopy. 2. jdk/make/Images.gmk regards STRIP_POLICY. As I mentioned earlier, this issue is related to RPM. So I hope to fix it before JDK8 GA is released. This won't happen (at least not for Java 8u0). Java 8 is already late in it's final candidate stage. It is too late for the initial Java 8 release to consider this magnitude of change. In any event, since the Java 8 rampdown began back in November, any change would first have to be applied to Java 9 and then approved for backport to a Java 8 or an update release (and it is also possibly too late for 8u20). Inability to include your suggested change in Java 8 or 8u20 is in no way a rejection of the ideas or contribution, it's merely a normal consequence of timelines and process. Mike
JDK-8036003: Add variable not to separate debug information.
Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable SEPARATED_DEBUGINFO_FILES . If we pass SEPARATED_DEBUGINFO_FILES=false to make command, make does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add STRIP_POLICY=no_strip to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like --disable-separated-debug-info . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 This changes add CHECKME option to configure script. However, this changes affects configure only. It should change configure.ac .
Re: JDK-8036003: Add variable not to separate debug information.
Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable SEPARATED_DEBUGINFO_FILES . If we pass SEPARATED_DEBUGINFO_FILES=false to make command, make does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add STRIP_POLICY=no_strip to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like --disable-separated-debug-info . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 This changes add CHECKME option to configure script. However, this changes affects configure only. It should change configure.ac .
Re: JDK-8036003: Add variable not to separate debug information.
The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable SEPARATED_DEBUGINFO_FILES . If we pass SEPARATED_DEBUGINFO_FILES=false to make command, make does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add STRIP_POLICY=no_strip to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like --disable-separated-debug-info . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 This changes add CHECKME option to configure script. However, this changes affects configure only. It should change configure.ac .
Re: JDK-8036003: Add variable not to separate debug information.
The proper way to fix this is to disable FDS. Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. -g is needed to generate debuginfo. Thanks, Yasumasa On 2014/02/28 23:04, Daniel D. Daugherty wrote: The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable SEPARATED_DEBUGINFO_FILES . If we pass SEPARATED_DEBUGINFO_FILES=false to make command, make does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add STRIP_POLICY=no_strip to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like --disable-separated-debug-info . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 This changes add CHECKME option to configure script. However, this changes affects configure only. It should change configure.ac .
Re: JDK-8036003: Add variable not to separate debug information.
On 1/03/2014 1:38 AM, Yasumasa Suenaga wrote: The proper way to fix this is to disable FDS. Does this mean I have to pass --disable-debug-symbols to configure ? I've added comment to JDK-8036003, I think if we pass --disable-debug-symbols to configure, gcc/g++ is not passed -g option. -g is needed to generate debuginfo. There are three pieces to all of this: 1. Generating debug symbols in the binaries (via gcc -g or whatever) 2. Generating debuginfo files (zipped or not) (FDS) 3. Stripping debug symbols from the binaries (strip-policy) It may be that we don't have clean separation between them, and if so that should be fixed, but I don't see the current proposal as the way forward. Also there may well be differences between how things are handled on the JDK side and hotspot side. I will try to look closer if I get a chance but my time is limited at the moment. David Thanks, Yasumasa On 2014/02/28 23:04, Daniel D. Daugherty wrote: The proper way to fix this is to disable FDS. We should not need yet another option to control debug info. Dan On 2/28/14 4:13 AM, David Holmes wrote: Hi, As I put in the bug report this seems way too complicated. Seems to me all you need to do to get what you want is not use FDS and not strip the symbols from the binary. The former is trivial. The latter is more awkward as the strip policy stuff does not work as I would want it to work, but still doable. David On 28/02/2014 7:18 PM, Yasumasa Suenaga wrote: Hi all, Currently, configure script can accept --disable-debug-symbols and --disable-zip-debug-info as controlling to generate debug information. However, current makefiles cannot build ELF binaries which is contained debug information with images target. Some Linux distros use RPM as package manager. RPM is built by rpmbuild command, it strips symbols and debug information during to generate rpm packages. https://fedoraproject.org/wiki/Packaging:Debuginfo For example, OpenJDK8 in Fedora20 ships libjvm.so and libjvm.debuginfo . libjvm.debuginfo is generated in OpenJDK's makefiles, however it does not contain debug information. Actual debug information is shipped by OpenJDK debuginfo package. This packaging is important when we have to debug JVM/JNI libraries. If we want to use debugging tools (GDB, SystemTap, etc...), they may requires debuginfo package. Debuggee (e.g. libjvm.so) points libjvm.so.debug which is located at sub directory in /usr/lib/debug . It is defined in ELF section (.note.gnu.build-id) of libjvm.so . However libjvm.so.debug does not contain valid debug information. We need to access libjvm.debuginfo.debug at same location. When we think to build OpenJDK rpm packages, we have to build ELF binaries which contain all debug information. we should not to separate debug information. Thus I want to add a variable SEPARATED_DEBUGINFO_FILES . If we pass SEPARATED_DEBUGINFO_FILES=false to make command, make does not execute objcopy command for generating debuginfo binaries. If we build rpm packages, we also have to add STRIP_POLICY=no_strip to arguments of make command. Or ELF binaries will be stripped. I've uploaded webrev for this enhancement. This is separated 3-part: top of forest, hotspot, jdk http://cr.openjdk.java.net/~ysuenaga/JDK-8036003/ Please review it and sponsoring! Thanks, Yasumasa P.S. I tried to add SEPARATED_DEBUGINFO_FILES as a configure option like --disable-separated-debug-info . I ran configure which is located at jdk9/dev directory after I ran autoconf and common/autoconf/autogen.sh. However it failed. I guess this is caused by changeset as below. JDK-8035495: Improvements in autoconf integration http://hg.openjdk.java.net/jdk9/dev/rev/6e29cd9ac2b4 This changes add CHECKME option to configure script. However, this changes affects configure only. It should change configure.ac .
