Hi Masatake, Although it seems like this security issue happens with symlinks (introduced by HADOOP-15170), it could very much happen with hard links as well. Thus, we need to secure this piece of code too - https://github.com/apache/hadoop/blame/20112bd8e81815714c3665741fafd0db29a58bb3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L781-L785. A git blame for this piece of code takes us to HDFS-8554, which is Hadoop 2.8.
In conclusion, all the versions >= 2.8 are affected and HADOOP-18155 needs to be backported. Thanks, --Gautham On Thu, 14 Apr 2022 at 10:25, Masatake Iwasaki <iwasak...@apache.org> wrote: > Hi Gautham, > > Let me know about versions affected. > Is the version prior to 3.1.0 affected? > The cause seems to be the code introduced by HADOOP-15170. > > I want to know this for releasing 2.10.2. > If branch-2.10 too is affected, I need to backport HADOOP-18155. > > Thanks, > Masatake Iwasaki > > On Mon, Apr 11, 2022 at 5:13 PM Gautham Banasandra <gaur...@apache.org> > wrote: > > > > Hi Akira, > > > >> What are the fixed versions? > > > > 3.2.3, all releases 3.4 onwards contain the fix. > > > >> What's the mitigation? > > > > Here are the possible mitigations (mutually exclusive of each other) in > the order of effectiveness - > > 1. Upgrade to the fixed versions. > > 2. Do not run any of the YARN daemons in admin mode on Windows. > > 3. Do not use symlinks in tar. > > > >> Would you update the CVE list in > https://hadoop.apache.org/cve_list.html ? > > > > Sure, I'll do it shortly. > > > > Thanks, > > --Gautham > > > > On Sat, 9 Apr 2022 at 13:46, Akira Ajisaka <aajis...@apache.org> wrote: > >> > >> HI Gautham, > >> > >> Thank you for your announcement. > >> What are the fixed versions? What's the mitigation? Would you update > the CVE list in https://hadoop.apache.org/cve_list.html ? > >> > >> -Akira > >> > >> > >> > >> On Thu, Apr 7, 2022 at 11:36 PM Gautham Banasandra <gaur...@apache.org> > wrote: > >>> > >>> The unTar function [1] uses unTarUsingJava function on Windows and the > >>> built-in tar utility on Unix and other OSes: > >>> > >>> if(Shell.WINDOWS) { > >>> // Tar is not native to Windows. Use simple Java based > implementation for > >>> // tests and simple tar archives > >>> unTarUsingJava(inFile, untarDir, gzipped); > >>> } > >>> else { > >>> // spawn tar utility to untar archive for full fledged unix behavior > such > >>> // as resolving symlinks in tar archives > >>> unTarUsingTar(inFile, untarDir, gzipped); > >>> } > >>> > >>> The function verifies that the extracted TAR entry is under the > expected > >>> targetDirPath[2]: > >>> > >>> if (!outputFile.getCanonicalPath().startsWith(targetDirPath)) { > >>> throw new IOException("expanding " + entry.getName() > >>> + " would create entry outside of " + outputDir); > >>> } > >>> > >>> However it doesn't apply the same restriction to the target of an > extracted > >>> symlink[3]: > >>> > >>> if (entry.isSymbolicLink()) { > >>> // Create symbolic link relative to tar parent dir > >>> Files.createSymbolicLink(FileSystems.getDefault() > >>> .getPath(outputDir.getPath(), entry.getName()), > >>> FileSystems.getDefault().getPath(entry.getLinkName())); > >>> return; > >>> } > >>> > >>> As a result, a TAR entry may create a symlink under the expected > extraction > >>> directory which points to an external directory. A subsequent TAR > entry may > >>> extract an arbitrary file into the external directory using the symlink > >>> name. This however would be caught by the same targetDirPath[4] check > on > >>> Unix because of the getCanonicalPath call. However on Windows, > >>> getCanonicalPath doesn't resolve symbolic links, which bypasses the > check. > >>> > >>> unpackEntries during TAR extraction follows symbolic links which allows > >>> writing outside expected base directory on Windows. > >>> > >>> [1]= > >>> > https://github.com/apache/hadoop/blob/125e3b616040b4f98956aa946cc51e99f7d596c2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L850 > >>> [2]= > >>> > https://github.com/apache/hadoop/blob/125e3b616040b4f98956aa946cc51e99f7d596c2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L964-L967 > >>> [3]= > >>> > https://github.com/apache/hadoop/blob/125e3b616040b4f98956aa946cc51e99f7d596c2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L983-L989 > >>> [4]= > >>> > https://github.com/apache/hadoop/blob/125e3b616040b4f98956aa946cc51e99f7d596c2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L964-L967 > >>> > >>> Credit: > >>> > >>> This issue was reported by a member of GitHub Security Lab, Jaroslav > >>> Lobačevski (https://github.com/JarLob). > >>> > >>> References: > >>> > >>> https://lists.apache.org/thread/hslo7wzw2449gv1jyjk8g6ttd7935fyz > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@hadoop.apache.org > For additional commands, e-mail: general-h...@hadoop.apache.org > >
