----- Original Message ----- > Interesting point. > > Should .diz files be removed when packaging JDK/JRE for Linux ? >
Specifically talking about packaging, distro build systems already handle stripping, etc. themselves so we've been setting STRIP_POLICY=no_strip. However, this still creates the diz files, even though the debuginfo is still in the binaries. I have a couple of changesets to fix this, one for HotSpot & one for JDK, basically making diz production conditional on STRIP_POLICY not being set to no_strip. I'll try and upstream them, if that sounds the correct thing to do. > 2012/9/5 David Holmes <david.hol...@oracle.com>: > > Dan, > > > > Sorry, not meaning to ruffle your feathers. > > > > The reason I say the diz-in-image story is unclear is because there > > are no > > explicit rules that indicates that diz files should end up in a JDK > > or JRE > > image. I can accept that everything should go in a full JDK image. > > But a JRE > > is a JDK with a bunch of things left out and some of those things > > are > > libraries. But because diz files are not mentioned explicitly in > > the rules > > for making images we actually get the diz files for the libraries > > that were > > removed! They are simply copied from the build artifacts location > > to the jre > > image location specifically libattach.diz and libsaproc.diz. Maybe > > that was > > a simple oversight. > > > > Additionally it is unclear to me why we have diz files for some > > libraries > > and not others? > > > > Anyway as I said this will need to be modified for the profile > > work, in > > consultation with all the interested parties. > > > > Thanks, > > David > > > > On 4/09/2012 11:51 PM, Daniel D. Daugherty wrote: > >> > >> On 9/2/12 9:37 PM, David Holmes wrote: > >>> > >>> Hi Dan, > >>> > >>> On 3/09/2012 12:47 PM, Daniel D. Daugherty wrote: > >>>> > >>>> On 9/2/12 7:26 PM, David Holmes wrote: > >>>>> > >>>>> The build will create debuginfo/diz files as requested - that's > >>>>> fine. > >>>>> > >>>>> What I'm unclear about is where those files should actually > >>>>> appear in > >>>>> our build artifacts, specifically the jdk/jre images that are > >>>>> created. > >>>>> > >>>>> Internally when we create binary images RE strips all the > >>>>> debuginfo/diz files out. > >>>> > >>>> > >>>> Not quite. The RE makefiles will put debug info files into a > >>>> separate debuginfo.zip file. This includes .debuginfo, .diz, > >>>> .map and .pdb files. For the regular bundles, the RE makefiles > >>>> will not include debug info files. Not quite the same as > >>>> stripping them out. > >>> > >>> > >>> Not trying to split hairs but if the image contains them and the > >>> RE > >>> bundle of the image does not, then the RE process is stripping > >>> them out. > >> > >> > >> The above is implying that there is a single RE bundle and that > >> hasn't > >> been true for the tar-style universe for quite a long time; the > >> demo > >> bundle was added by around JDK8-B20 or so. For the package-style > >> universe, I don't think that has ever been true. > >> > >> RE's makefiles take the image generated by the build and package > >> up the > >> various files into two or more bundles. I believe each platform > >> has two > >> styles of bundles. For Solaris, SVR4 style packages and tar/zip > >> bundles. > >> For Windows, an EXE installer and tar/zip bundles. For Linux RPM > >> packages > >> and tar/ZIP bundles. > >> > >> So I'll stick with my assertion that the RE process is placing > >> debug info > >> files into a separate bundle and is not stripping them out. > >> > >> > >> > >>> > >>>>> But the build itself seems to treat them in an ad-hoc manner: > >>>>> > >>>>> - The new build deliberately excludes debuginfo/diz files > >>>>> associated > >>>>> with binaries, but will include any related to libraries (via > >>>>> generic > >>>>> copying routine). (It's obvious from the comments related to > >>>>> this that > >>>>> there is some puzzlement as to this reasoning.) > >>>> > >>>> > >>>> Sorry I haven't looked at the new build system. > >>> > >>> > >>> It tries to emulate the old build. > >>> > >>>>> - The old build also tries to exclude the files associated with > >>>>> binaries, but only handles .debuginfo not .diz :( > >>>> > >>>> > >>>> During one round in the FDS project, I included .debuginfo/.diz > >>>> files > >>>> right next to some of the binaries. Apparently that's not > >>>> allowed > >>>> without > >>>> explicit permission. The contents of the "bin" directory are > >>>> controlled > >>>> and there are tests to verify those contents. > >>>> > >>>> I changed the Makefiles for the few binaries that support FDS to > >>>> not > >>>> install the debug info files with the binaries, but they are > >>>> left in > >>>> the normal build artifacts location if someone has the need to > >>>> use > >>>> them. Of course, I've had queries for debug info files for the > >>>> binaries > >>>> to be included in the debuginfo.zip bundles. I'm not planning to > >>>> fight > >>>> that battle. > >>> > >>> > >>> Okay that explains the special handling for binaries. So how do > >>> these > >>> then get into the debuginfo.zip? Or don't they? > >> > >> > >> They don't. They are only available via the build artifacts that > >> are archived. > >> > >> > >> > >>> > >>>>> Those associated with libraries just seem to get copied if they > >>>>> happen > >>>>> to be there > >>>> > >>>> > >>>> All of the logic that copies debug info files to the image > >>>> should do > >>>> so if they happen to be there. Not all component support FDS now > >>>> or > >>>> in the future so the logic needs to adapt to what is built. > >>>> > >>>> > >>>>> As I said this all seems very ad-hoc to me. I would expect to > >>>>> see no > >>>>> debuginfo/diz files in a created image by default, and have a > >>>>> separate > >>>>> target that would produce a tar file of all the debuginfo/diz > >>>>> files > >>>>> ready to overlay on an existing image. > >>>> > >>>> > >>>> Close. The image has to have the debug info or .diz files in > >>>> order for > >>>> the RE makefiles to generate the debuginfo.zip files. It is the > >>>> RE > >>>> makefiles that handle the packaging. Just like the demos are > >>>> always > >>>> built and it is the RE makefiles that put them in a separate > >>>> bundle. > >>> > >>> > >>> So therein lies the problem. RE lies outside of the OpenJDK build > >>> system. If the RE Makefiles can copy an image they can copy the > >>> debuginfo files too. I'd much rather see no debuginfo files in > >>> the > >>> built images (unless requested) and a distinct debuginfo related > >>> target (that RE could use if they choose). That would seem to be > >>> simpler all round. > >> > >> > >> My understanding is that "the image" is supposed to be the > >> complete > >> image if every bundle is extracted. I believe that is how the > >> packages > >> are sanity checked to be complete. RE's Makefiles are meant to > >> process > >> "the image" into bundles. They aren't meant to add files of their > >> own. > >> > >> If you are planning to change the meaning of "the image" or if you > >> are planning to put files into bundles that are not in "the > >> image", > >> then you'll need to coordinate with RE and whoever does the > >> package > >> inventory testing. > >> > >> Also, if you change the way this works, keep in mind that some of > >> the > >> Makefiles generate their objects into a temporary directory and > >> then > >> copy things from that temporary directory to the image. Other > >> Makefiles > >> generate their objects directly into the image. For the latter, > >> you'll > >> have to move the debug info file(s) out of the image into some > >> other > >> place where it can be picked up by whatever bundling process you > >> create > >> to make the debuginfo.zip file. > >> > >> > >> > >>> > >>>>> Thoughts/comments? > >>>>> > >>>>> I have to deal with this for the SE Profile work, where we will > >>>>> not > >>>>> want these files present in any of the images. > >>>> > >>>> > >>>> It should be fine if the Embedded profile does not generate > >>>> debug info > >>>> files or if the Embedded profile generates debug info, but does > >>>> not > >>>> copy the debug info files into the image area. The RE makefiles > >>>> will > >>>> handle the presence or absence of debug info files. > >>> > >>> > >>> These are not embedded profiles, these are just profiles - it is > >>> all > >>> to be part of SE in Java 8. The build is supposed to create an > >>> image > >>> corresponding to each profile. I can certainly specialise the > >>> handling > >>> of the debuginfo files for the profiles, but it seemed to me that > >>> the > >>> overall debuginfo-in-image story was rather unclear. > >> > >> > >> I'm sorry you think that the debuginfo-in-image story is unclear. > >> I thought I did a pretty good job integrating FDS into the whole > >> OpenJDK build process somewhat seamlessly. I guess not. > >> > >> The key things to remember in all this is that debug info files > >> (like the demos) have to end up in a separate bundle or bundles. > >> So if you change the way this is all implemented, you still have > >> to follow the separate bundle rule. > >> > >> > >> > >>> > >>>> However, the non-Embedded profile needs to continue to generate > >>>> and > >>>> copy debug info files for those components that already support > >>>> FDS. > >>>> If you change the non-Embedded logic to not copy the debug info > >>>> files > >>>> into the image area, the RE debuginfo.zip bundle will not be > >>>> created > >>>> and that will break FDS. > >>> > >>> > >>> RE processes will have to change to handle profiles in any case. > >>> I > >>> think they'd rather get a debuginfo.zip out of the build than > >>> having > >>> to create it themselves. > >> > >> > >> Interesting and not something that I have ever heard. I got the > >> impression that RE needed to have control over what was bundled > >> and how it was bundled. It's kind of hard to certify/vouch for a > >> process when you don't do it yourself, but that's not really my > >> call. > >> > >> Good luck with the tweaking of the build and packaging processes. > >> Just remember: There be dragons here! > >> > >> Dan > >> > >> > >> > >>> > >>> Thanks, > >>> David > >>> > >>>> > >>>> Dan > >>>> > >>>> > >>>>> > >>>>> David > -- Andrew :) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) PGP Key: 248BDC07 (https://keys.indymedia.org/) Fingerprint = EC5A 1F5E C0AD 1D15 8F1F 8F91 3B96 A578 248B DC07
