+1 for approach 2.
Regards, Ajay On Wed, Nov 23, 2016 at 12:16 PM, David Yan <da...@datatorrent.com> wrote: > The goal of this log4j appender is to provide a log offset and the fixed > name of the container log file (instead of apex.log becoming apex.log.1 and > then apex.log.2, etc due to rotation) as part of an error STRAM event so > users can easily locate the log entries around the error. > > The user can override the appender, but in that case, the engine detects > that and will not include the log location as part of the STRAM event. > > David > > On Tue, Nov 22, 2016 at 7:10 PM, Priyanka Gugale <priya...@datatorrent.com > > > wrote: > > > Hi, > > > > Thomas, > > Yes log4j is ultimately owned by user, and they should be able to > override > > it. What I am trying to do is provide a default behavior for Apex. In > case > > user isn't using any logger of their own we should use this new appender > of > > Apex rather than using standard log4j appender as per hadoop config. > > > > Sanjay, > > Archetype is the good place to put this and I will add it there, but many > > time people won't use it. So I wanted to keep it at ~/.dt as well. Is > there > > any other default config folder for Apex? > > > > Also I am not relying on anything. If we fail to find config in app jar > or > > ~/.dt we are going to skip usage of this new appender. > > > > -Priyanka > > > > On Wed, Nov 23, 2016 at 5:58 AM, Sanjay Pujare <san...@datatorrent.com> > > wrote: > > > > > The only way to “enforce” this new appender is to update the archetypes > > > (apex-app-archetype and apex-conf-archetype under apex-core/ ) to use > > the > > > new ones as default. But there does not seem to be a way to enforce > this > > > for anyone not using the archetypes. > > > > > > I agree with not relying on ~/.dt in apex-core. > > > > > > On 11/22/16, 1:08 PM, "Thomas Weise" <t...@apache.org> wrote: > > > > > > The log4j configuration is ultimately owned by the user, so how do > > you > > > want > > > to enforce a custom appender? > > > > > > I don't think that this should rely on anything in ~/.dt either > > > > > > Thomas > > > > > > On Tue, Nov 22, 2016 at 10:00 AM, Priyanka Gugale < > > > priya...@datatorrent.com> > > > wrote: > > > > > > > Hi, > > > > > > > > I am working on APEXCORE-563 > > > > <https://issues.apache.org/jira/browse/APEXCORE-563> > > > > As per this Jira we should put log file name in > container/operator > > > events. > > > > The problem is current RollingFileAppender keeps renaming files > > from > > > 1 to 2 > > > > to ... n as files reach maximum allowed file size. Because of > > > constant > > > > renaming of files we can't put a fixed file name in stram event. > > > > > > > > To overcome this I would like to add a new log4j appender to > > > ApexCore. > > > > There are two ways I can implement this: > > > > 1. Have Daily rolling file appender. The current file will be > > > recognized > > > > based on timestamp in file name. Also to control max file size, > we > > > need to > > > > keep rolling files based on size as well. > > > > 2. Have Rolling File Appender but do not rename files. When max > > file > > > size > > > > is reached create new file with next number e.g. crate log file > > > dt.log.2 > > > > after dt.log.1 is full. Also to recognize the latest file keep > the > > > softlink > > > > named dt.log pointing to current log file. > > > > > > > > I would prefer to implement approach 2. Please provide your > > > > comments/feedback if you feel otherwise. > > > > > > > > Also to use this new appender we need to use our custom > > > log4j.properties > > > > file instead of one present in hadoop conf. For that we need to > set > > > jvm > > > > option -Dlog4j.configuration. I am planning to update file > > > dt-site.xml in > > > > folder ~/.dt and default properties file available in apex > > > archetype to > > > > set jvm options as follows: > > > > <property> > > > > <name>dt.attr.CONTAINER_JVM_OPTIONS</name> > > > > <value>-Dlog4j.configuration=log4j.props</value> > > > > </property> > > > > > > > > And I will copy log4j.props file in ~/.dt folder as well as > > > > apex-archetypes. > > > > > > > > Lastly if someone still miss this new log4j properties file or > jvm > > > option > > > > to set -Dlog4j.configuration we will not put log file name in > event > > > raised > > > > by container or operator. > > > > > > > > Please provide your feedback on this approach. > > > > > > > > -Priyanka > > > > > > > > > > > > > > > > > > >
