Hi Raman, Since you are using YARN, you collect the yarn logs (which also contains the application logs) by executing below command and move to the desired location
yarn logs -applicationId <your_yarn_app_id> Best Girish On Fri, Aug 2, 2019 at 10:17 AM Srinath C <srinat...@gmail.com> wrote: > Hi Raman, > > Probably use the rolling file appender in log4j to compress the rotated > log file? > > Regards. > > > On Fri, Aug 2, 2019 at 12:47 AM raman gugnani <ramangugnani....@gmail.com> > wrote: > >> HI , >> >> I am looking for right solution for logging the logs produced by the >> executors. Most of the places I have seen logging done by log4j properties, >> but no where people I have seen any solution where logs are being >> compressed. >> >> Is there anyway I can compress the logs, So that further those logs can >> be shipped to S3. >> >> -- >> Raman Gugnani >> > -- Girish bhat m
