Il 24/02/2014 18:28, Vojtech Szocs ha scritto: > > > ----- Original Message ----- >> From: "Sandro Bonazzola" <[email protected]> >> To: "Vojtech Szocs" <[email protected]> >> Cc: "Eyal Edri" <[email protected]>, "Juan Hernandez" <[email protected]>, >> "Kiril Nesenko" <[email protected]>, >> "arch" <[email protected]> >> Sent: Monday, February 24, 2014 9:08:09 AM >> Subject: [urgent] Re: ovirt-engine build failure >> >> Top posting, >> >> Please give this task really urgent priority, as you can see: >> >> http://jenkins.ovirt.org/job/ovirt-engine_3.4_create_rpms/ >> and >> http://jenkins.ovirt.org/job/ovirt-engine_master_create_rpms/ >> >> are failing consistently so we don't have nightly builds. > > (CC'ing Einav) > > Attached TEST patch, please apply & rebuild ovirt-engine as usual. > > This patch should cause following output during GWT compilation: > > [ERROR] ######### Util.readURLAsBytes > file:/home/vszocs/work/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/gwt/org/ovirt/engine/ui/webadmin/images/search_button.png > > If IOException occurs, exception stacktrace will be printed right below that > line. > > Note: please ignore the [ERROR] prefix, it's a technical issue, more details > at https://www.mail-archive.com/[email protected]/msg05735.html > > I just did a local upstream build with TEST patch applied, GWT compilation > passed OK.
build job running with your patch here: http://jenkins.ovirt.org/job/manual-build-tarball/235/ waiting for a free executor right now. Let's see how it goes. > >> >> >> >> >> Il 20/02/2014 13:34, Vojtech Szocs ha scritto: >>> >>> >>> ----- Original Message ----- >>>> From: "Sandro Bonazzola" <[email protected]> >>>> To: "Vojtech Szocs" <[email protected]> >>>> Cc: "Eyal Edri" <[email protected]>, "Juan Hernandez" >>>> <[email protected]>, "Kiril Nesenko" <[email protected]>, >>>> "arch" <[email protected]> >>>> Sent: Thursday, February 20, 2014 10:02:00 AM >>>> Subject: Re: ovirt-engine build failure >>>> >>>> Il 19/02/2014 16:51, Vojtech Szocs ha scritto: >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: "Sandro Bonazzola" <[email protected]> >>>>>> To: "Vojtech Szocs" <[email protected]> >>>>>> Cc: "Eyal Edri" <[email protected]>, "Juan Hernandez" >>>>>> <[email protected]>, "Kiril Nesenko" <[email protected]>, >>>>>> "arch" <[email protected]> >>>>>> Sent: Wednesday, February 19, 2014 2:43:44 PM >>>>>> Subject: Re: ovirt-engine build failure >>>>>> >>>>>> Il 19/02/2014 14:41, Vojtech Szocs ha scritto: >>>>>>> >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>>> From: "Sandro Bonazzola" <[email protected]> >>>>>>>> To: "Vojtech Szocs" <[email protected]> >>>>>>>> Cc: "Eyal Edri" <[email protected]>, "Juan Hernandez" >>>>>>>> <[email protected]>, "Kiril Nesenko" <[email protected]>, >>>>>>>> "arch" <[email protected]> >>>>>>>> Sent: Wednesday, February 19, 2014 2:23:16 PM >>>>>>>> Subject: Re: ovirt-engine build failure >>>>>>>> >>>>>>>> Il 19/02/2014 14:19, Vojtech Szocs ha scritto: >>>>>>>>> >>>>>>>>> >>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: "Sandro Bonazzola" <[email protected]> >>>>>>>>>> To: "Vojtech Szocs" <[email protected]> >>>>>>>>>> Cc: "Eyal Edri" <[email protected]>, "Juan Hernandez" >>>>>>>>>> <[email protected]>, "Kiril Nesenko" <[email protected]>, >>>>>>>>>> "arch" <[email protected]> >>>>>>>>>> Sent: Wednesday, February 19, 2014 1:18:28 PM >>>>>>>>>> Subject: Re: ovirt-engine build failure >>>>>>>>>> >>>>>>>>>> Il 19/02/2014 12:45, Vojtech Szocs ha scritto: >>>>>>>>>>> Hi, I checked GWT sources: >>>>>>>>>>> >>>>>>>>>>>> [INFO] java.lang.NullPointerException: No ImageRect ever computed >>>>>>>>>>>> for >>>>>>>>>>>> removeBrickCommitRequired >>>>>>>>>>>> [INFO] at >>>>>>>>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475) >>>>>>>>>>> >>>>>>>>>>> This happened because internal image cache for bundled (composite >>>>>>>>>>> PNG) >>>>>>>>>>> images was corrput. >>>>>>>>>>> It seems to be caused by disk I/O error while reading file >>>>>>>>>>> images/glusterAsyncTasks/remove_brick_commit_required.png >>>>>>>>>>> >>>>>>>>>>>> [INFO] java.lang.NullPointerException >>>>>>>>>>>> [INFO] at >>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:40) >>>>>>>>>>> >>>>>>>>>>> This happened because image data was null, again possible disk I/O >>>>>>>>>>> error >>>>>>>>>>> while reading image file. >>>>>>>>>>> >>>>>>>>>>> Can we enable "TRACE" logging during GWT compilation? >>>>>>>>>>> >>>>>>>>>>> -Dgwt.logLevel=TRACE >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Building master right now with your suggestions here: >>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/ >>>>>>>>> >>>>>>>>> Looking at >>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/label=fedora20-host/console >>>>>>>>> >>>>>>>>> So far I see no errors.. There should be total 5 browsers x 8 locales >>>>>>>>> = >>>>>>>>> 40 >>>>>>>>> GWT permutations. >>>>>>>> >>>>>>>> So no changes with and without >>>>>>>> -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera. >>>>>>>> We've still 40 permutations. Maybe you defined >>>>>>>> userAgent=ie8,ie9,gecko1_8,safari,opera somewhere in the POM files? >>>>>>> >>>>>>> You are right. I checked frontend/webadmin/modules/pom.xml -> the >>>>>>> "gwt.userAgent" >>>>>>> property is copied into "locale" GWT property in >>>>>>> {WebAdmin,UserPortal}.gwt.xml >>>>>> >>>>>> Ok, so I'll remove -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera from >>>>>> build >>>>>> command. >>>>> >>>>> Agreed. >>>>> >>>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> This: >>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/227/label=centos6-host/console >>>>>>>> failed, you can look at it, it has -Dgwt.logLevel=TRACE -X. >>>>>>> >>>>>>> Hm, it contains: >>>>>>> >>>>>>> [DEBUG] (f) logLevel = INFO >>>>>>> >>>>>>> Somehow the TRACE log level for GWT wasn't applied.. not sure why. Let >>>>>>> me >>>>>>> check >>>>>>> this on local build. >>>>>> >>>>>> Ok, let me know if I've to change command line. >>>>> >>>>> I wrote upstream patch: http://gerrit.ovirt.org/#/c/24789/ >>>>> >>>>> When it's merged, setting "gwt.logLevel" will work (as you did before) >>>>> and we'll be able to see more details in the build log. >>>> >>>> Here is the new build with your patch merged: >>>> >>>> http://jenkins.ovirt.org/job/manual-build-tarball/228/label=centos6-host/consoleFull >>> >>> Thanks. I did some more investigation on this. >>> >>> Again it's NullPointerException when rebinding ApplicationResources, but >>> this time for different image file: ApplicationResources.wrenchImage >>> ("images/wrench.png"). >>> >>> Looking at stacktrace & GWT sources, the problematic code seems to be >>> (AbstractResourceContext:74): >>> >>> byte[] bytes = Util.readURLAsBytes(resource); >>> >>> where "resource" is (ImageResourceGenerator:311): >>> >>> localized.getUrl(); // LocalizedImage localized >>> >>> Note: in oVirt UI we don't use localized images (i.e. "wrench_fr_FR.png") >>> so "localized" in this case points to "wrench.png". >>> >>> So the problem seems to be reading this file: >>> >>> >>> frontend/webadmin/modules/webadmin/src/main/resources/org/ovirt/engine/ui/webadmin/images/wrench.png >>> >>> Since we're having many GWT permutations, this file ^^ (along with similar >>> ones) is attempted to be read multiple times *from the same (single) GWT >>> compiler Java process, before actual permutation worker Java sub-processes >>> are spawned*. (More precisely, above NullPointerException happens at >>> deferred binding phase which precedes actual GWT compilation per each >>> permutation.) >>> >>> I think the root cause is Util.readURLAsBytes method used internally by GWT >>> compiler: >>> >>> public static byte[] readURLAsBytes(URL url) { >>> try { >>> URLConnection conn = url.openConnection(); >>> conn.setUseCaches(false); >>> return readURLConnectionAsBytes(conn); >>> } catch (IOException e) { >>> return null; // This causes NullPointerException, but this exception >>> isn't logged (wtf, GWT!) >>> } >>> } >>> >>> To summarize, I'll need to write another patch which replaces >>> Util.readURLAsBytes implementation via AOP and prints IOException details. >>> >>>> >>>> >>>> >>>> >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> # CI_MAVEN_SETTINGS=$WORKSPACE/artifactory-ovirt-org-settings.xml >>>>>>>>>> # export MAVEN_OPTS="${MAVEN_OPTS} -Xmx2048M -XX:MaxPermSize=1024m >>>>>>>>>> -Dgwt.compiler.localWorkers=1" >>>>>>>>>> # rpmbuild -D "_topdir $(pwd)/rpmbuild" -ts "${TARBALL}" >>>>>>>>>> # rpmbuild -D "_topdir $(pwd)/rpmbuild" -D "ovirt_build_extra_flags >>>>>>>>>> -gs >>>>>>>>>> ${CI_MAVEN_SETTINGS} -Dgwt.userAgent=ie8,ie9,gecko1_8,safari,opera >>>>>>>>>> -Dgwt.logLevel=TRACE -X" --rebuild rpmbuild/SRPMS/*.src.rpm >>>>>>>>>> >>>>>>>>>> Let me know if you need anything else. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Vojtech >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>> From: "Vojtech Szocs" <[email protected]> >>>>>>>>>>>> To: "Sandro Bonazzola" <[email protected]> >>>>>>>>>>>> Cc: "Eyal Edri" <[email protected]>, "Juan Hernandez" >>>>>>>>>>>> <[email protected]>, "Kiril Nesenko" <[email protected]>, >>>>>>>>>>>> "arch" <[email protected]> >>>>>>>>>>>> Sent: Tuesday, February 18, 2014 6:54:20 PM >>>>>>>>>>>> Subject: Re: ovirt-engine build failure >>>>>>>>>>>> >>>>>>>>>>>> In your log I see following error: >>>>>>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>> [INFO] Computing all possible rebind results for >>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' >>>>>>>>>>>> [INFO] Rebinding >>>>>>>>>>>> org.ovirt.engine.ui.webadmin.ApplicationResources >>>>>>>>>>>> [INFO] Invoking generator >>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineClientBundleGenerator >>>>>>>>>>>> [INFO] [ERROR] Generator >>>>>>>>>>>> 'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator' >>>>>>>>>>>> threw >>>>>>>>>>>> an exception while rebinding >>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' >>>>>>>>>>>> [INFO] java.lang.NullPointerException: No ImageRect ever computed >>>>>>>>>>>> for >>>>>>>>>>>> removeBrickCommitRequired >>>>>>>>>>>> [INFO] at >>>>>>>>>>>> com.google.gwt.resources.rg.ImageResourceGenerator.createAssignment(ImageResourceGenerator.java:475) >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> I checked again logs at >>>>>>>>>>>> http://jenkins.ovirt.org/job/manual-build-tarball/212/label=fedora19-host/consoleFull >>>>>>>>>>>> and indeed there is a similar error: >>>>>>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>> [INFO] Computing all possible rebind results for >>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' >>>>>>>>>>>> [INFO] Rebinding >>>>>>>>>>>> org.ovirt.engine.ui.webadmin.ApplicationResources >>>>>>>>>>>> [INFO] Invoking generator >>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineClientBundleGenerator >>>>>>>>>>>> [INFO] [ERROR] Generator >>>>>>>>>>>> 'com.google.gwt.resources.rebind.context.InlineClientBundleGenerator' >>>>>>>>>>>> threw >>>>>>>>>>>> an exception while rebinding >>>>>>>>>>>> 'org.ovirt.engine.ui.webadmin.ApplicationResources' >>>>>>>>>>>> [INFO] java.lang.NullPointerException >>>>>>>>>>>> [INFO] at >>>>>>>>>>>> com.google.gwt.resources.rebind.context.InlineResourceContext.deploy(InlineResourceContext.java:40) >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> NullPointerException when processing ApplicationResources but >>>>>>>>>>>> occuring >>>>>>>>>>>> from >>>>>>>>>>>> different places (ImageResourceGenerator vs. >>>>>>>>>>>> InlineResourceContext) >>>>>>>>>>>> >>>>>>>>>>>> Now I understand why Maven build fails without any details: >>>>>>>>>>>> - GWT deferred binding fails for ApplicationResources (i.e. >>>>>>>>>>>> uncaught >>>>>>>>>>>> NPE) >>>>>>>>>>>> - GWT compilation proceeds, fails to link with >>>>>>>>>>>> ApplicationResources >>>>>>>>>>>> - GWT compilation task fails with non-OK status code >>>>>>>>>>>> >>>>>>>>>>>> So the issue here is GWT deferred binding generator >>>>>>>>>>>> (InlineClientBundleGenerator) failure during GWT compilation. >>>>>>>>>>>> >>>>>>>>>>>> GWT compiler writes extra Java files during deferred binding >>>>>>>>>>>> phase, so this could point to issue with local environment.. >>>>>>>>>>>> >>>>>>>>>>>> I will try to investigate this and look at GWT source code. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Sandro Bonazzola >>>>>>>>>> Better technology. Faster innovation. Powered by community >>>>>>>>>> collaboration. >>>>>>>>>> See how it works at redhat.com >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Sandro Bonazzola >>>>>>>> Better technology. Faster innovation. Powered by community >>>>>>>> collaboration. >>>>>>>> See how it works at redhat.com >>>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Sandro Bonazzola >>>>>> Better technology. Faster innovation. Powered by community >>>>>> collaboration. >>>>>> See how it works at redhat.com >>>>>> >>>> >>>> >>>> -- >>>> Sandro Bonazzola >>>> Better technology. Faster innovation. Powered by community collaboration. >>>> See how it works at redhat.com >>>> >> >> >> -- >> Sandro Bonazzola >> Better technology. Faster innovation. Powered by community collaboration. >> See how it works at redhat.com >> -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Arch mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/arch
