Hi Jayanga, +1 for maintaining separate directory for log, pre-QA patches/updates with a proper name. I assume that the pre-QA patches/updates are still valid in C5. So names such as "information-collector" or "log-collector" will not be an option. Because this directory will not only contains log patches/updates. Moreover maintaining separate directory will provide us more convenient way to filter the changes into wso2 packed jars and generate a log file with this changes. Since applying log, pre-QA patch/update is only one time task, We can provide command-line option to get updates/patches from this separate directory as Ruwan mentioned. So it will not affect use-cases such as adding DB drivers, configuring additional transports (e.g: JMS) and etc.
Thanks, Nisala On Mon, May 22, 2017 at 5:20 PM, Jayanga Dissanayake <[email protected]> wrote: > Hi Ramith/Nira > > On Mon, May 22, 2017 at 2:56 PM, Niranjan Karunanandham <[email protected] > > wrote: > >> >> On Mon, May 22, 2017 at 12:34 PM, Ramith Jayasinghe <[email protected]> >> wrote: >> >>> what I was thinking is if there is a anyway we can record the fact that >>> this patch is a log/preqa may be by adding (custom?) meta data. >>> then at least a warning saying 'there is a pre-qa/log patch available >>> during the server start up. >>> >> I agree with Ramith. In 4.4.x, we mention when patches are applied in the >> patch.log. IMO it would be better if we can record something similar to >> this when temp updates are applied to the system. This will help us in the >> support front by checking the logs if there are any temp jars applied. >> >> Thanks for the suggestion. Currently, we log a sample message like >> below when we are updateing the bundles.info file. >> "Successfully updated the OSGi bundle information of Carbon Runtime: >> default" >> >> But this happens only once, at the very first time you are starting with >> the new jar. Consequent restarts would not log the given message as there >> is no update. In C4 we had patch log, in the C5 we havent thought of such >> file yet. I would be better to have the custom user bundles and log/pre-qa >> bundle information in the logs. We will check the possibility of adding >> this to the logs with minimal effect to the server startup time. >> > >> >>> >>> >>> On Mon, May 22, 2017 at 12:17 PM, Jayanga Dissanayake <[email protected]> >>> wrote: >>> >>>> Hi Ramith, >>>> >>>> Inside the server, there is no way to distinguish whether it is a >>>> log/pre-qa, it will just be a bundle with same symbolic name and the >>>> version >>>> We might specify log/pre-qa in the zip file we provide to the client. >>>> >>>> Can you please explain more on the validations that you would expect, >>>> so that I could look more into the possibilities whether those are >>>> feasible with the currently proposed methodology. >>>> >>>> Thanks, >>>> Jayanga. >>>> >>>> >>>> *Jayanga Dissanayake* >>>> Associate Technical Lead >>>> WSO2 Inc. - http://wso2.com/ >>>> lean . enterprise . middleware >>>> email: [email protected] >>>> mobile: +94772207259 <077%20220%207259> >>>> <http://wso2.com/signature> >>>> >>>> On Mon, May 22, 2017 at 12:01 PM, Ramith Jayasinghe <[email protected]> >>>> wrote: >>>> >>>>> is there a way we can flag a patch to be something 'temporary' (such >>>>> as a log/pre-qa)? if so may be we can do some validation surrounding that? >>>>> >>>>> On Mon, May 22, 2017 at 11:52 AM, Jayanga Dissanayake < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Niranjan, >>>>>> >>>>>> Having another directory for patches would add an empty directory to >>>>>> the current pack structure. And having the name "patch" is not >>>>>> correct according to the C5 way, as we are proving fixes as updates. >>>>>> >>>>>> Another point is, having many places to put bundles will >>>>>> complicate our story, Hence, I would prefer to have one place where users >>>>>> put bundles into. If that bundle is having a symbolic name and version >>>>>> equivalent to a one in the plugins directory, it will act as a "patch" or >>>>>> else it will be treated as a separate bundle. >>>>>> >>>>>> Thanks, >>>>>> Jayanga. >>>>>> >>>>>> *Jayanga Dissanayake* >>>>>> Associate Technical Lead >>>>>> WSO2 Inc. - http://wso2.com/ >>>>>> lean . enterprise . middleware >>>>>> email: [email protected] >>>>>> mobile: +94772207259 <077%20220%207259> >>>>>> <http://wso2.com/signature> >>>>>> >>>>>> On Mon, May 22, 2017 at 10:52 AM, Niranjan Karunanandham < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Jayanga, >>>>>>> >>>>>>> IMO we should also take into consideration of apply fixes other than >>>>>>> just log patches. In C5 onwards, the current approach is to use WUM to >>>>>>> deliver all updates. But there can be scenarios where we would need to >>>>>>> provide an immediate solution or say a Pre-QA fix. IMO we would need to >>>>>>> address this here. >>>>>>> >>>>>>> IMO applying the temp fix in lib folder is not correct. IMO the Lib >>>>>>> folder is used for addition components that are required by the server, >>>>>>> i.e., components which are not provided by wso2. IMO it would be better >>>>>>> to >>>>>>> have a separate folder for this. In carbon 4.4.X, we had a separate >>>>>>> folder >>>>>>> called patches. >>>>>>> >>>>>>> Regards, >>>>>>> Nira >>>>>>> >>>>>>> On Fri, May 19, 2017 at 10:46 AM, Jayanga Dissanayake < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Vidura, >>>>>>>> >>>>>>>> Adding a separate command line would easily lead to more human >>>>>>>> errors. With C5, we are trying to minimize the configurations, command >>>>>>>> line parameters, etc. to make the users' life easier and to reduce the >>>>>>>> human errors much as possible. Log patch is just a one case I >>>>>>>> highlighted. >>>>>>>> There are other use cases like; >>>>>>>> >>>>>>>> - Adding DB drivers >>>>>>>> - Configuring additional transports e.g: JMS >>>>>>>> - Putting custom mediators (in C4), etc. >>>>>>>> >>>>>>>> Above requirements are based on the C4 experience and there will >>>>>>>> definitely be some similar set of requirements in the C5 as well. >>>>>>>> Hence, if >>>>>>>> we introduce a separate command line, the user will have to start the >>>>>>>> server with that parameter each time they modify the lib directory, >>>>>>>> even >>>>>>>> they do a version upgrade of a given library. >>>>>>>> Hence I would prefer to make the library update >>>>>>>> decision programmatically rather a user input. >>>>>>>> >>>>>>>> @Lackman: >>>>>>>> With C5, we will abandon the word "patch". Everything including >>>>>>>> features, fixes, etc. will be provided via updates. Hence there is no >>>>>>>> point >>>>>>>> having a separate directory call "patches". A "log patch" is just >>>>>>>> another >>>>>>>> jar file having the same bundle symbolic name and version as its >>>>>>>> original >>>>>>>> bundle in the plugins directory. Once it is removed from the lib >>>>>>>> directory, >>>>>>>> the server will use the original bundle in the plugins directory. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jayanga. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> *Jayanga Dissanayake* >>>>>>>> Associate Technical Lead >>>>>>>> WSO2 Inc. - http://wso2.com/ >>>>>>>> lean . enterprise . middleware >>>>>>>> email: [email protected] >>>>>>>> mobile: +94772207259 <+94%2077%20220%207259> >>>>>>>> <http://wso2.com/signature> >>>>>>>> >>>>>>>> On Thu, May 18, 2017 at 12:06 PM, Lakshman Udayakantha < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I also think running a separate tool or add startup parameter to >>>>>>>>> say there is a change to the existing bundles and deploy them also is >>>>>>>>> a >>>>>>>>> better approach, since this is a special scenario as Vidura mentioned >>>>>>>>> also. >>>>>>>>> >>>>>>>>> For the reverting to a backup point, IMO it is an unnecessary step >>>>>>>>> because if we have a good set of integration and unit tests to cover >>>>>>>>> the >>>>>>>>> wum update, then there is the least importance of reverting to a >>>>>>>>> backup >>>>>>>>> point. Another thing is, since subsequent updates are going on >>>>>>>>> previous >>>>>>>>> updates, all the changes will come with new updates. In that case, >>>>>>>>> How are >>>>>>>>> we going to apply subsequent updates on previous updates? >>>>>>>>> >>>>>>>>> Anyway, I feel that we should go with a separate patch folder if >>>>>>>>> we are going to apply temporary patch rather than lib folder. It is >>>>>>>>> more >>>>>>>>> convenient because the patch is not an additional library. What is the >>>>>>>>> reason are we stopping doing this? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Lakshman. >>>>>>>>> >>>>>>>>> On Wed, May 17, 2017 at 6:25 PM, Vidura Nanayakkara < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Jayanga and Ruwan, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Can we look into a way this can be triggered selectively with >>>>>>>>>>> some command-line option or an offline command? Reason to keep this >>>>>>>>>>> logic >>>>>>>>>>> from slowing down the startup in production run. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> +1 for having a separate command-line option to apply the >>>>>>>>>> temporary patch. The reason is applying a temporary log patch to >>>>>>>>>> collect >>>>>>>>>> logs is a special case scenario and is not worth checking the files >>>>>>>>>> list in >>>>>>>>>> the [product_home]/lib to check whether there are any changes to the >>>>>>>>>> files. >>>>>>>>>> If the client can put the effort to apply a log patch and collect >>>>>>>>>> the logs >>>>>>>>>> certainly the client can start the server with an additional >>>>>>>>>> parameter. >>>>>>>>>> >>>>>>>>>> WDYT? >>>>>>>>>> >>>>>>>>>> Best Regards, >>>>>>>>>> Vidura Nanayakkara >>>>>>>>>> >>>>>>>>>> On Wed, May 17, 2017 at 6:01 PM, Jayanga Dissanayake < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Ruwan, >>>>>>>>>>> >>>>>>>>>>> Thanks for the comments. >>>>>>>>>>> >>>>>>>>>>> In every server startup, it checks the files list in the >>>>>>>>>>> [product_home]/lib to check whether there are any changes to the >>>>>>>>>>> files. >>>>>>>>>>> Yes, we could save some 'time' in the server startup if we provide a >>>>>>>>>>> separate tool. But there is a possibility that someone would miss >>>>>>>>>>> running >>>>>>>>>>> that script and expect the new changes. Anyway, I'll check on the >>>>>>>>>>> possibilities of minimizing startup time taken. >>>>>>>>>>> >>>>>>>>>>> Regarding the checkpointing thing, the suggested approach keep >>>>>>>>>>> the original bundles.info file as a backup and always update it >>>>>>>>>>> with the bundles given in the [product_home]/lib. And we haven >>>>>>>>>>> planned on >>>>>>>>>>> keeping the historical changes. Hence it will always be from >>>>>>>>>>> original state >>>>>>>>>>> to latest state change. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Jayanga. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Jayanga Dissanayake* >>>>>>>>>>> Associate Technical Lead >>>>>>>>>>> WSO2 Inc. - http://wso2.com/ >>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>> email: [email protected] >>>>>>>>>>> mobile: +94772207259 <+94%2077%20220%207259> >>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>> >>>>>>>>>>> On Wed, May 17, 2017 at 10:17 AM, Ruwan Abeykoon < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> + 1 on the idea of the four steps mentioned. >>>>>>>>>>>> Can we look into a way this can be triggered selectively with >>>>>>>>>>>> some command-line option or an offline command? Reason to keep >>>>>>>>>>>> this logic >>>>>>>>>>>> from slowing down the startup in production run. >>>>>>>>>>>> >>>>>>>>>>>> In addition to that, in future, we could able to enhance the >>>>>>>>>>>> same logic to allow similar behavior like "windows System Restore >>>>>>>>>>>> Point" or >>>>>>>>>>>> "OSX time machine" to revert to known point of WUM, by version >>>>>>>>>>>> controlling >>>>>>>>>>>> the bundle-info and the respective jars/configs. >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Ruwan >>>>>>>>>>>> >>>>>>>>>>>> On Wed, May 17, 2017 at 9:21 AM, Jayanga Dissanayake < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> Existing OSGi deployment logic [1] keep updating the >>>>>>>>>>>>> bundles.info file as and when a bundle is added/removed from >>>>>>>>>>>>> the [product_home]/lib directory. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. It first read the bundles.info file of the current runtime >>>>>>>>>>>>> 2. Then take the list of bundles coming from "plugins" >>>>>>>>>>>>> directory. >>>>>>>>>>>>> 3. Adds the bundles in the [product_home]/lib directory to the >>>>>>>>>>>>> previous list >>>>>>>>>>>>> 4. Get distinct bundles based on the bundle symbolic name and >>>>>>>>>>>>> version. (ignores bundles with same symbolic name and version >>>>>>>>>>>>> already in >>>>>>>>>>>>> the list, hence bundles in the plugins directory get the priority) >>>>>>>>>>>>> 5. Update the bundles.info file >>>>>>>>>>>>> >>>>>>>>>>>>> The above-mentioned approach assumes all basic bundles for the >>>>>>>>>>>>> runtime coming from the "plugins" directory and adds the external >>>>>>>>>>>>> bundles >>>>>>>>>>>>> on top of it. >>>>>>>>>>>>> >>>>>>>>>>>>> Recently we were looking into the C5 update/patching model and >>>>>>>>>>>>> there was a concern on how we could add a temporary log patches, >>>>>>>>>>>>> etc. to >>>>>>>>>>>>> identify an issue. (in a case where all other possibilities >>>>>>>>>>>>> failed). >>>>>>>>>>>>> >>>>>>>>>>>>> The most convenient to the client is to just add the given log >>>>>>>>>>>>> patch to the pack, restart, collect the relevant logs, and >>>>>>>>>>>>> finally revert >>>>>>>>>>>>> the log patch. Any additional steps would be an overhead as the >>>>>>>>>>>>> client is >>>>>>>>>>>>> already undergoing a severe issue and client is trying to help us >>>>>>>>>>>>> to >>>>>>>>>>>>> identify the issue by applying the log patch. Hence this should >>>>>>>>>>>>> be modeled >>>>>>>>>>>>> in a convenient way to the users. >>>>>>>>>>>>> >>>>>>>>>>>>> There is a possibility to let the clients add the log patch >>>>>>>>>>>>> into the [product_home]/lib directory and remove it once done. >>>>>>>>>>>>> But with >>>>>>>>>>>>> the current OSGi bundle deployment logic, we can't do that as it >>>>>>>>>>>>> always >>>>>>>>>>>>> gives the priority to the bundles in the plugins directory. If we >>>>>>>>>>>>> put a >>>>>>>>>>>>> bundle with the same bundle-symbolic-name and version into the >>>>>>>>>>>>> plugins >>>>>>>>>>>>> directory it will be ignored. >>>>>>>>>>>>> >>>>>>>>>>>>> To achieve this behavior we have to modify the existing OSGi >>>>>>>>>>>>> bundle deployment logic as follows. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. In the first run make a backup of the original bundles.info >>>>>>>>>>>>> file >>>>>>>>>>>>> (bundles.info.original this will be used as the baseline for each >>>>>>>>>>>>> time it >>>>>>>>>>>>> updated the bundles.info file) >>>>>>>>>>>>> 2. Read the bundles.info.original file >>>>>>>>>>>>> 3. Add the bundles in the [product_home]/lib directory >>>>>>>>>>>>> 4. Update the bundles.info file >>>>>>>>>>>>> And >>>>>>>>>>>>> The logic in selecting effective bundles list should be >>>>>>>>>>>>> changed to not to give priority to bundles in the plugins >>>>>>>>>>>>> directory. >>>>>>>>>>>>> Instead modify the entries, if a similar bundle (symbolic name >>>>>>>>>>>>> and version) >>>>>>>>>>>>> is found in the [product_home]/lib >>>>>>>>>>>>> >>>>>>>>>>>>> Above suggested approach allows easily add the patched jars >>>>>>>>>>>>> into the [product_home]/lib directory for temporary purposes. And >>>>>>>>>>>>> reverting >>>>>>>>>>>>> the patch is also possible as we have a backup of the original >>>>>>>>>>>>> bundles.info file >>>>>>>>>>>>> >>>>>>>>>>>>> WDYT? >>>>>>>>>>>>> >>>>>>>>>>>>> [1] https://github.com/wso2/carbon-kernel/blob/v5.2.0-m3/lau >>>>>>>>>>>>> ncher/src/main/java/org/wso2/carbon/launcher/extensions/OSGi >>>>>>>>>>>>> LibBundleDeployerUtils.java >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> *Jayanga Dissanayake* >>>>>>>>>>>>> Associate Technical Lead >>>>>>>>>>>>> WSO2 Inc. - http://wso2.com/ >>>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>>> email: [email protected] >>>>>>>>>>>>> mobile: +94772207259 <+94%2077%20220%207259> >>>>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> *Ruwan Abeykoon* >>>>>>>>>>>> *Associate Director/Architect**,* >>>>>>>>>>>> *WSO2, Inc. http://wso2.com <https://wso2.com/signature> * >>>>>>>>>>>> *lean.enterprise.middleware.* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Best Regards, >>>>>>>>>> >>>>>>>>>> *Vidura Nanayakkara* >>>>>>>>>> Software Engineer >>>>>>>>>> >>>>>>>>>> Email : [email protected] >>>>>>>>>> Mobile : +94 (0) 717 919277 <+94%2071%20791%209277> >>>>>>>>>> Web : http://wso2.com >>>>>>>>>> Blog : https://medium.com/@viduran >>>>>>>>>> LinkedIn : https://lk.linkedin.com/in/vidura-nanayakkara >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Lakshman Udayakantha >>>>>>>>> WSO2 Inc. www.wso2.com >>>>>>>>> lean.enterprise.middleware >>>>>>>>> Mobile: *0717429601 <071%20742%209601>* >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Architecture mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> >>>>>>> *Niranjan Karunanandham* >>>>>>> Associate Technical Lead - WSO2 Inc. >>>>>>> WSO2 Inc.: http://www.wso2.com >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Ramith Jayasinghe >>>>> Technical Lead >>>>> WSO2 Inc., http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> E: [email protected] >>>>> P: +94 777542851 <+94%2077%20754%202851> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>> >>> >>> -- >>> Ramith Jayasinghe >>> Technical Lead >>> WSO2 Inc., http://wso2.com >>> lean.enterprise.middleware >>> >>> E: [email protected] >>> P: +94 777542851 <+94%2077%20754%202851> >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >>> >> Regards, >> Nira >> >> -- >> >> >> *Niranjan Karunanandham* >> Associate Technical Lead - WSO2 Inc. >> WSO2 Inc.: http://www.wso2.com >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Nisala Niroshana Nanayakkara,* Software Engineer Mobile | +94 717600022 WSO2 Inc | http://wso2.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
