See <https://builds.apache.org/job/Hcatalog-trunk-build/112/changes>
Changes: [heyongqiang] HIVE-2706 [jira] StackOverflowError when using custom UDF after adding archive after adding jars (Kevin Wilfong via Yongqiang He) Summary: https://issues.apache.org/jira/browse/HIVE-2706 The issue was that the current thread's classloader and the classloader in the conf differed due to the prehook updating only the current thread's classloader with new jars. Now, it updates both classloaders, fixing the issue. When a custom UDF is used in a query after add an archive, such as a zip file, after adding jars, the XMLEncoder enters an infinite loop when serializing the map reduce task, as part of sending it to be executed. This results in a stack overflow error. Test Plan: Verified it fixed the stack overflow error. Reviewers: JIRA, heyongqiang, njain Reviewed By: heyongqiang CC: heyongqiang Differential Revision: https://reviews.facebook.net/D1167 ------------------------------------------ [...truncated 5620 lines...] create-dirs: [echo] Project: contrib [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/contrib> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/contrib/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/contrib/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/contrib/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/contrib/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/contrib/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/contrib/src/test/resources> does not exist. init: [echo] Project: contrib create-dirs: [echo] Project: service [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/service> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/service/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/service/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/service/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/service/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/service/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/service/src/test/resources> does not exist. init: [echo] Project: service create-dirs: [echo] Project: cli [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/cli> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/cli/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/cli/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/cli/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/cli/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/cli/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/cli/src/test/resources> does not exist. init: [echo] Project: cli create-dirs: [echo] Project: jdbc [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/jdbc> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/jdbc/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/jdbc/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/jdbc/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/jdbc/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/jdbc/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/jdbc/src/test/resources> does not exist. init: [echo] Project: jdbc create-dirs: [echo] Project: hwi [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hwi> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hwi/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hwi/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hwi/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hwi/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hwi/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/hwi/src/test/resources> does not exist. init: [echo] Project: hwi create-dirs: [echo] Project: hbase-handler [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hbase-handler> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hbase-handler/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hbase-handler/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hbase-handler/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hbase-handler/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hbase-handler/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/hbase-handler/src/test/resources> does not exist. init: [echo] Project: hbase-handler create-dirs: [echo] Project: pdk [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/pdk> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/pdk/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/pdk/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/pdk/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/pdk/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/pdk/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/pdk/src/test/resources> does not exist. init: [echo] Project: pdk create-dirs: [echo] Project: builtins [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/builtins> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/builtins/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/builtins/test> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/builtins/test/src> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/builtins/test/classes> [mkdir] Created dir: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/builtins/test/resources> [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/builtins/src/test/resources> does not exist. init: [echo] Project: builtins jar: [echo] Project: hive create-dirs: [echo] Project: shims [copy] Warning: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/shims/src/test/resources> does not exist. init: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-resolve: [echo] Project: shims [ivy:resolve] :: loading settings :: file = <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/ivy/ivysettings.xml> [ivy:report] Processing <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml> to <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/ivy/report/org.apache.hive-hive-shims-default.html> ivy-retrieve: [echo] Project: shims compile: [echo] Project: shims ivy-init-settings: [echo] Project: shims ivy-retrieve-hadoop-source: [echo] Project: shims [ivy:retrieve] :: loading settings :: file = <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/ivy/ivysettings.xml> [ivy:retrieve] :: resolving dependencies :: org.apache.hive#hive-shims;0.9.0-SNAPSHOT [ivy:retrieve] confs: [default, master, compile, runtime] [ivy:retrieve] found org.apache.hadoop#hadoop-core;0.20.1 in hadoop-source [ivy:retrieve] found org.apache.hadoop#hadoop-core;0.20.3-CDH3-SNAPSHOT in hadoop-source [ivy:retrieve] found org.apache.zookeeper#zookeeper;3.3.1 in default [ivy:retrieve] found org.apache.thrift#libthrift;0.7.0 in maven2 [ivy:retrieve] found commons-logging#commons-logging;1.0.4 in maven2 [ivy:retrieve] found commons-logging#commons-logging-api;1.0.4 in maven2 [ivy:retrieve] found com.google.guava#guava;r06 in maven2 [ivy:retrieve] :: resolution report :: resolve 1368ms :: artifacts dl 3ms --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 7 | 0 | 0 | 0 || 7 | 0 | | master | 0 | 0 | 0 | 0 || 0 | 0 | | compile | 0 | 0 | 0 | 0 || 0 | 0 | | runtime | 0 | 0 | 0 | 0 || 0 | 0 | --------------------------------------------------------------------- [ivy:retrieve] :: retrieving :: org.apache.hive#hive-shims [ivy:retrieve] confs: [default, master, compile, runtime] [ivy:retrieve] 7 artifacts copied, 0 already retrieved (289263kB/743ms) install-hadoopcore-internal: [echo] Project: shims BUILD FAILED <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/build.xml>:229: The following error occurred while executing this line: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build.xml>:296: The following error occurred while executing this line: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build.xml>:150: The following error occurred while executing this line: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/shims/build.xml>:76: The following error occurred while executing this line: <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build-common.xml>:133: Error while expanding <https://builds.apache.org/job/Hcatalog-trunk-build/ws/trunk/hive/external/build/hadoopcore/hadoop-0.20.1.tar.gz> java.io.IOException: Not in GZIP format at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:143) at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58) at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:67) at org.apache.tools.ant.taskdefs.Untar$UntarCompressionMethod.decompress(Untar.java:235) at org.apache.tools.ant.taskdefs.Untar.expandStream(Untar.java:158) at org.apache.tools.ant.taskdefs.Untar.expandFile(Untar.java:114) at org.apache.tools.ant.taskdefs.Expand.execute(Expand.java:132) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:390) at org.apache.tools.ant.Target.performTasks(Target.java:411) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:390) at org.apache.tools.ant.Target.performTasks(Target.java:411) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:302) at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:221) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:68) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:398) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:390) at org.apache.tools.ant.Target.performTasks(Target.java:411) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:390) at org.apache.tools.ant.Target.performTasks(Target.java:411) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399) at org.apache.tools.ant.Project.executeTarget(Project.java:1368) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1251) at org.apache.tools.ant.Main.runBuild(Main.java:809) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109) Total time: 9 seconds Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results
