jojochuang commented on a change in pull request #3758:
URL: https://github.com/apache/hadoop/pull/3758#discussion_r773616025
##########
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core/src/main/java/org/apache/hadoop/yarn/service/client/ServiceClient.java
##########
@@ -1016,6 +1018,131 @@ ApplicationId submitApp(Service app) throws
IOException, YarnException {
return submissionContext.getApplicationId();
}
+ /**
+ * Compress (tar) the input files to the output file.
+ *
+ * @param files The files to compress
+ * @param output The resulting output file (should end in .tar.gz)
+ * @param bundleRoot
+ * @throws IOException
+ */
+ public static File compressFiles(Collection<File> files, File output,
+ String bundleRoot) throws IOException {
+ try (FileOutputStream fos = new FileOutputStream(output);
+ TarArchiveOutputStream taos = new TarArchiveOutputStream(
+ new BufferedOutputStream(fos))) {
+ taos.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU);
+ for (File f : files) {
+ addFilesToCompression(taos, f, "sysfs", bundleRoot);
+ }
+ }
+ return output;
+ }
+
+ /**
+ * Compile file list for compression and going recursive for
+ * nested directories.
+ *
+ * @param taos The archive
+ * @param file The file to add to the archive
+ * @param dir The directory that should serve as
+ * the parent directory in the archive
+ * @throws IOException
+ */
+ private static void addFilesToCompression(TarArchiveOutputStream taos,
+ File file, String dir, String bundleRoot) throws IOException {
+ if (!file.isHidden()) {
+ // Create an entry for the file
+ if (!dir.equals(".")) {
+ if (File.separator.equals("\\")) {
+ dir = dir.replaceAll("\\\\", "/");
+ }
+ }
+ taos.putArchiveEntry(
+ new TarArchiveEntry(file, dir + "/" + file.getName()));
+ if (file.isFile()) {
+ // Add the file to the archive
+ try (FileInputStream input = new FileInputStream(file)) {
+ IOUtils.copy(input, taos);
+ taos.closeArchiveEntry();
+ }
+ } else if (file.isDirectory()) {
+ // close the archive entry
+ if (!dir.equals(".")) {
+ taos.closeArchiveEntry();
+ }
+ // go through all the files in the directory and using recursion, add
+ // them to the archive
+ File[] allFiles = file.listFiles();
+ if (allFiles != null) {
+ for (File childFile : allFiles) {
+ addFilesToCompression(taos, childFile,
+ file.getPath().substring(bundleRoot.length()), bundleRoot);
+ }
+ }
+ }
+ }
+ }
+
+ private void addYarnSysFs(Path path,
Review comment:
i suspect the the change in the ServiceClient.java is totally
unnecessary, because the method is not used at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
