Hi, I'm running an oozie job that executes a java action using HDP 2.3.4. When this action runs, it has a lot of files in its "current directory" (see below for a full list). Some of these files are there because I'm shipping them with the action (e.g., myapp.properties). Many of those jar files are present whether or not I have "oozie.use.system.libpath" set to true or false in the java action. Because they're in my current directory, they are very high up in my action's class path and causing some problems. For example, jackson-*-2.2.3.jar or guava-11.0.2.jar. I understand that they may need to be SOMEWHERE in the class path, because they are core hadoop dependencies, but they are present in other places too:
/grid/5/hadoop/yarn/local/usercache/SVBIG01S/appcache/application_1458846497308_8586/container_e3048_1458846497308_8586_01_000002/mr-framework/hadoop/share/hadoop/common/lib/jackson-core-2.2.3.jar /grid/5/hadoop/yarn/local/usercache/SVBIG01S/appcache/application_1458846497308_8586/container_e3048_1458846497308_8586_01_000002/mr-framework/hadoop/share/hadoop/common/lib/guava-11.0.2.jar For example, I want to use a different version of jackson and have it included in my utilities-1.2.0-SNAPSHOT.jar, but it shares the class path precedence with the jackson-*.jar files that are in the current directory and sometimes those get loaded. Can I: * keep these extra jar files out of my current directory entirely (my preferred solution) * failing that, make sure that the jars I provide have the absolute highest class path priority (acceptable solution) Thanks! -Aaron Files in current dir:/grid/5/hadoop/yarn/local/usercache/MYID/appcache/application_1458846497308_8586/container_e3048_1458846497308_8586_01_000002/. ====================== File: hadoop-azure-2.7.1.2.3.4.0-3485.jar File: jackson-databind-2.2.3.jar File: json-simple-1.1.jar Dir: tmp File: hdfs-site.xml File: job.xml File: storm-hive-ddl-1.2.0-SNAPSHOT.jar File: aws-java-sdk-1.7.4.jar File: oozie-sharelib-oozie-4.2.0.2.3.4.0-3485.jar File: action.xml File: joda-time-2.1.jar File: jaas.conf File: hive-jdbc.jar File: foundation-builder-1.2.0-SNAPSHOT.jar File: guava-11.0.2.jar File: container_tokens Dir: mr-framework Dir: hadoop File: azure-storage-2.2.0.jar File: launch_container.sh File: jackson-annotations-2.2.3.jar File: firefly.properties File: .action.xml.crc File: myapp.properties File: propagation-conf.xml File: .job.xml.crc File: core-site.xml File: commons-lang3-3.3.2.jar File: utilities-1.2.0-SNAPSHOT.jar File: hadoop-aws-2.7.1.2.3.4.0-3485.jar File: jackson-core-2.2.3.jar File: oozie-hadoop-utils-hadoop-2-4.2.0.2.3.4.0-3485.jar
