Hi Guys, Any help here?
On Wed, Aug 8, 2018 at 7:56 AM Fawze Abujaber <fawz...@gmail.com> wrote: > Hello Community, > > I'm using Spark 2.3 and Spark 1.6.0 in my cluster with Cloudera > distribution 5.13.0. > > Both are configured to run on Yarn, but i'm unable to see completed > application in Spark2 history server, while in Spark 1.6.0 i did. > > 1) I checked the HDFS permissions for both folders and both have the same > permissions. > > drwxrwxrwt - cloudera-scm spark 0 2018-08-08 00:46 > /user/spark/applicationHistory > drwxrwxrwt - cloudera-scm spark 0 2018-08-08 00:46 > /user/spark/spark2ApplicationHistory > > The applications file itself running with permissions 770 in both. > > -rwxrwx--- 3 fawzea spark 4743751 2018-08-07 23:32 > /user/spark/spark2ApplicationHistory/application_1527404701551_672816_1 > -rwxrwx--- 3 fawzea spark 134315 2018-08-08 00:41 > /user/spark/applicationHistory/application_1527404701551_673359_1 > > 2) No error in the Spark2 history server log. > > 3) Compared the configurations between Spark 1.6 and Spark 2.3 like system > user, enable log, etc ... all looks the same. > > 4) Once i changed the permissions for the above Spark2 applications to > 777, i was able to see the application in the spark2 history server UI. > > Tried to figure out if the 2 Sparks UIs running with different users but > was unable to find it. > > Anyone who ran into this issue and solved it? > > Thanks in advance. > > > -- > Take Care > Fawze Abujaber > -- Take Care Fawze Abujaber
