Hi All, Can we consider introducing "Diagnostic Context" (e.g. [1] )while on this effort ? which will improve our logging information considerably.
[1] https://www.javacodegeeks.com/2013/01/effective-logging-in-javajee-mapped-diagnostic-context.html Cheers, Ruwan On Mon, Jul 31, 2017 at 4:40 PM, KasunG Gajasinghe <[email protected]> wrote: > > > On Mon, Jul 31, 2017 at 3:50 PM, KasunG Gajasinghe <[email protected]> > wrote: > >> Hi Asma, >> >> Can you explain why we need to do this from a user perspective? What are >> the advantages/disavantages? >> >> Log4j does not support the Log4j properties file format. So, this change >> is not backward-compatible. >> >> On Mon, Jul 31, 2017 at 2:53 PM, Asma Jabir <[email protected]> wrote: >> >>> The logging framework for C4 will be implemented using pax-logging as >>> illustrated in the simple diagram below. >>> >>> [image: Inline image 3] >>> >>> As shown above: >>> >>> - org.wso2.carbon.logging will act as the Logging API >>> - *org.ops4j.pax.logging.pax-logging-log4j2.jar* will bridge the >>> logs to log4J2 >>> - The custom appenders will be residing in a fragment of >>> *org.ops4j.pax.logging.pax-logging-log4j2* >>> >>> Also, >>> >>> - pax-logging.properties file will be added which points to the >>> log4j2.properties file for configurations >>> - org.wso2.carbon.logging.propfile_1.0.0.jar will be removed as it >>> is no longer necessary to provide the configuration in the classpath of >>> the >>> org.wso2.carbon.logging >>> >>> Moreover, there is a log4j.properties file in org.wso2.carbon.server >>> which is used for writing the logs before OSGi is started. Also there is >>> logging-bridge.properties file which is the configuration file used for JUL >>> (java-util-logging). Therefore, similar to C5, the following changes will >>> be done in this module to reduce configuration files. >>> >>> - Currently org.wso2.carbon.server uses JCL (commons-logging) and >>> this will be changed to JUL >>> - *This eliminates the need for a separate log4j configuration file >>> for org.wso2.carbon.server.* Thus log4j.properties residing >>> inside org.wso2.carbon.server will be removed >>> - This resolves [1] >>> - All the configurations related to org.wso2.carbon.server will be >>> written in >>> * $CARBON_HOME/repository/conf/etc/logging-bridge.properties* >>> >>> >>> We are looking forward to move forward with this method and appreciate >>> your comments/suggestions on this. >>> >>> >>> >>> Regards, >>> Asma >>> >>> On Mon, Jul 17, 2017 at 11:38 AM, Asma Jabir <[email protected]> wrote: >>> >>>> Hi >>>> >>>> We already have a central logging module in the kernel which is >>>> org.wso2.carbon.logging [1] and that acts similar to the a Logging API. So >>>> we can can continue to use it and all the logging specific dependencies and >>>> imports will be residing here. But the custom appenders are located in >>>> org.wso2.carbon.utils [2] since it is the first to get resolved and >>>> therefore, this would have changes when re-writing the appenders. Overall, >>>> the changes will be only inside the kernel and application will not need >>>> any changes. >>>> >>>> Moreover, after further experimenting for approach 1 and 2 which were >>>> implementable, the following were understood. >>>> >>>> - When using plain log4j 2, it cannot be used with with the default >>>> functionality >>>> - The bridge from jcl-log4j is implemented using a ServiceLoader >>>> which needs special handling in OSGi environment [3]. >>>> - Also this needs additional OSGi fragments as stated in point 2 >>>> in the first mail >>>> - The configuration file(log4j.properties) is currently packed >>>> to carbon logging using a fragmented bundle ( >>>> org.wso2.carbon.logging.propfile_1.0.0.jar) - same method can be >>>> follwed for log4j2 or the file can be given through a system >>>> property which >>>> makes org.wso2.carbon.logging.propfile_1.0.0.jar unnecessary >>>> >>>> >>>> - Also, external .jar files are not able to find the logging >>>> implementation which we are still investigating as to why this happens >>>> >>>> The above had been happening since the log4j2 implementation was not >>>> found in OSGi environment and it was not related to external jars. >>>> Therefore, by providing the configuration files *log4j-provider.properties >>>> *and *log4j2.properties,* in carbon-logging the issue was fixed. >>>> >>>> >>>> - Pax-logging overcomes the above issues and therefore, the default >>>> pax-logging api can be used >>>> - This provides log4j2 bridge internally which will map any type >>>> of logging to log4j2 >>>> - By replacing the log4j1 libraries with pax-logging and >>>> re-writing the appenders, this can be gotten to work >>>> - The properties file will be pax-logging.properties/log4j2. >>>> properties >>>> - The configuration file will be set as a system property - The >>>> org.wso2.carbon.logging.propfile_1.0.0.jar will not be needed >>>> therein >>>> >>>> >>>> [1] https://github.com/wso2/carbon-kernel/tree/4.5.x/core/or >>>> g.wso2.carbon.logging >>>> [2] https://github.com/wso2/carbon-kernel/tree/4.5.x/core/or >>>> g.wso2.carbon.utils/src/main/java/org/wso2/carbon/utils/logg >>>> ing/appenders >>>> [3] http://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html >>>> >>>> <http://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html> >>>> >>>> <http://blog.osgi.org/2013/02/javautilserviceloader-in-osgi.html> >>>> Regards, >>>> Asma >>>> >>>> On Sat, Jul 15, 2017 at 11:57 PM, Ruwan Abeykoon <[email protected]> >>>> wrote: >>>> >>>>> Hi Irunika, >>>>> In C4, applications use commons logging. We never wanted to use Log4J >>>>> directly. Hence the architecture you proposed is already there in C4. >>>>> In C5 we use slf4j, in place of commons logging, which architecturally >>>>> the same. >>>>> >>>>> Hence I do not see any value in introducing another WSO2 specific >>>>> logging framework/wrapper/etc. >>>>> >>>>> For the original ${subject}, would not it work simply using log4j2 >>>>> library, removing all log4j 1 library with change in log4j.property file >>>>> appropriately? >>>>> Yes, it needs to have a change in Carbon**Appenders to use log4j2 API, >>>>> but this is only at Carbon-Logging module. I guess we can have new version >>>>> with log4j2. >>>>> In short, IMO, there will be no change in applications, but may be >>>>> little change in kernel implementation(which is transparent to apps) in >>>>> log4j2 migration. >>>>> >>>>> Cheers, >>>>> Ruwan >>>>> >>>>> >>>>> On Fri, Jul 14, 2017 at 3:37 PM, Irunika Weeraratne <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Asma, >>>>>>> >>>>>>> The following approaches were found and evaluated to identify the >>>>>>> most suitable one. >>>>>>> >>>>>>> 1. Using pax-logging >>>>>>> - The pax-logging api and the pax-logging-log4j bridge will have >>>>>>> to be used. >>>>>>> - Supposedly, pax-logging will feed the front-end logging >>>>>>> into its log4j back-end using its built-in adapter which we >>>>>>> are trying to achieve at the moment >>>>>>> 2. Using Log4J 2.x >>>>>>> - This will need the log4j2-core, log4j2-api and the bridges >>>>>>> to map the front-end logging api to Log4J 2.x back-end >>>>>>> - Also this needs two OSGi fragments to customize the >>>>>>> log4j-core and log4j-api work in OSGi >>>>>>> - This requires the log4j2.properties file to be in the >>>>>>> fragment used with log4j-core and therefore, the file cannot be >>>>>>> edited later >>>>>>> - Also, external .jar files are not able to find the logging >>>>>>> implementation which we are still investigating as to why this >>>>>>> happens >>>>>>> 3. Using both Log4j 1.x and 2.x >>>>>>> - While C4 will use the latter as the back-end, the external >>>>>>> jars needing v1 will use the former >>>>>>> - This could lead to threading issues since appenders from >>>>>>> two log4J versions will write to a single file simultaneously >>>>>>> 4. Using the Log4J 1.x bridge [4] >>>>>>> - The Log4J 1.x jar is replaced with the 1.x bridge api and >>>>>>> the required log4j2 files. This will redirect all the logs from >>>>>>> Log4j 1.x >>>>>>> to 2.x >>>>>>> - This only works if there are no custom appenders >>>>>>> >>>>>>> IMHO all these approaches also have a common issue. When we try to >>>>>> use any of those APIs we have to have to change our code base again and >>>>>> again according to the API we are using. >>>>>> >>>>>> IMO what we should do is write our own API for Logging and in the >>>>>> implementation, we can use any of those as the backend. So only the >>>>>> implementation will be changed while our code base remains same. >>>>>> >>>>>> Please find the diagram below. >>>>>> >>>>>> [image: Inline image 1] >>>>>> >>>>>> WDYT? >>>>>> >>>>>> Thanks and best regards, >>>>>> Irunika >>>>>> >>>>>> >>>>>> >>>>>> *Irunika Weeraratne* >>>>>> *Software Engineer | WSO2, Inc. <http://wso2.com/>* >>>>>> *Email : [email protected] <[email protected]>* >>>>>> *LinkedIn : https://lk.linkedin.com/in/irunika >>>>>> <https://lk.linkedin.com/in/irunika>* >>>>>> *Mobile : +94712403314 <+94%2071%20240%203314>* >>>>>> *Lean . Enterprise . Middleware* >>>>>> >>>>>> >>>>>> On Wed, Jul 12, 2017 at 7:53 PM, Asma Jabir <[email protected]> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> We have been looking into $subject and these are the findings we >>>>>>> have arrived at so far. >>>>>>> >>>>>>> Carbon 4.x.x currently uses Log4J 1.x as the logging back-end and >>>>>>> following will have to be done when migrating to Log4J 2.x. >>>>>>> >>>>>>> 1. Rewriting custom appenders >>>>>>> - Current Log4J 1.x method - Rewriting of Logging Events by >>>>>>> extending appropriate classes (i.e.: >>>>>>> AppenderSkeleton/ConsoleAppender/DailyRollingFileAppender) >>>>>>> [1] >>>>>>> - Proposed Log4J 2.x method - Use of plugins to convert the >>>>>>> log record into a required pattern with custom fields [2] >>>>>>> 2. Replace log4j.properties with a log4j2.properties file >>>>>>> - The properties file should be changed to log4j2.properties >>>>>>> and the syntax will different from v1 format [3] >>>>>>> 3. Remove >>>>>>> - Direct use of log4j in org.wso2.carbon.server.Main - used >>>>>>> to remove all appenders added in the non-OSGi environment >>>>>>> - Package org.wso2.carbon.utils.logging.appenders as this >>>>>>> should be rewritten as converter plugins >>>>>>> >>>>>>> >>>>>>> The following approaches were found and evaluated to identify the >>>>>>> most suitable one. >>>>>>> >>>>>>> 1. Using pax-logging >>>>>>> - The pax-logging api and the pax-logging-log4j bridge will have >>>>>>> to be used. >>>>>>> - Supposedly, pax-logging will feed the front-end logging >>>>>>> into its log4j back-end using its built-in adapter which we >>>>>>> are trying to achieve at the moment >>>>>>> 2. Using Log4J 2.x >>>>>>> - This will need the log4j2-core, log4j2-api and the bridges >>>>>>> to map the front-end logging api to Log4J 2.x back-end >>>>>>> - Also this needs two OSGi fragments to customize the >>>>>>> log4j-core and log4j-api work in OSGi >>>>>>> - This requires the log4j2.properties file to be in the >>>>>>> fragment used with log4j-core and therefore, the file cannot be >>>>>>> edited later >>>>>>> - Also, external .jar files are not able to find the logging >>>>>>> implementation which we are still investigating as to why this >>>>>>> happens >>>>>>> 3. Using both Log4j 1.x and 2.x >>>>>>> - While C4 will use the latter as the back-end, the external >>>>>>> jars needing v1 will use the former >>>>>>> - This could lead to threading issues since appenders from >>>>>>> two log4J versions will write to a single file simultaneously >>>>>>> 4. Using the Log4J 1.x bridge [4] >>>>>>> - The Log4J 1.x jar is replaced with the 1.x bridge api and >>>>>>> the required log4j2 files. This will redirect all the logs from >>>>>>> Log4j 1.x >>>>>>> to 2.x >>>>>>> - This only works if there are no custom appenders >>>>>>> >>>>>>> As per the current state, first approach will need the least changes >>>>>>> and since pax-logging is OSGi compatible this seems to be suitable. >>>>>>> However, the 2nd approach is also being looked into to fix the >>>>>>> aforementioned drawbacks and issues. >>>>>>> >>>>>>> Thoughts please >>>>>>> >>>>>>> >>>>>>> [1] https://logging.apache.org/log4j/1.2/apidocs/org/apache/ >>>>>>> log4j/spi/LoggingEvent.html >>>>>>> [2] http://logging.apache.org/log4j/2.x/manual/plugins.html >>>>>>> [3] https://logging.apache.org/log4j/2.0/manual/configuratio >>>>>>> n.html#Properties >>>>>>> [4] https://logging.apache.org/log4j/2.x/manual/migration.html >>>>>>> >>>>>>> -- >>>>>>> Asma Zinneera Jabir >>>>>>> Software Engineer >>>>>>> WSO2 Inc: http://wso2.com/ >>>>>>> Contact No: +94 77 332 4752 <+94%2077%20332%204752> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Asma Zinneera Jabir >>>> Software Engineer >>>> WSO2 Inc: http://wso2.com/ >>>> Contact No: +94 77 332 4752 <+94%2077%20332%204752> >>>> >>>> >>>> >>> >>> >>> -- >>> Asma Zinneera Jabir >>> Software Engineer >>> WSO2 Inc: http://wso2.com/ >>> Contact No: +94 77 332 4752 <+94%2077%20332%204752> >>> >>> >>> >> >> >> -- >> >> *Kasun Gajasinghe*Associate Technical Lead, WSO2 Inc. >> email: kasung AT spamfree wso2.com >> linked-in: http://lk.linkedin.com/in/gajasinghe >> blog: http://kasunbg.org >> phone: +1 650-745-4499 <+1%20650-745-4499>, 77 678 0813 >> >> > > > > -- > > *Kasun Gajasinghe*Associate Technical Lead, WSO2 Inc. > email: kasung AT spamfree wso2.com > linked-in: http://lk.linkedin.com/in/gajasinghe > blog: http://kasunbg.org > phone: +1 650-745-4499 <(650)%20745-4499>, 77 678 0813 > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
