On Wed, Aug 23, 2017 at 2:46 PM, Lakmal Warusawithana <[email protected]> wrote:
> > > On Wed, Aug 23, 2017 at 2:39 PM, Isuru Haththotuwa <[email protected]> > wrote: > >> While I fully agree that its necessary to be able to inject configs via >> environment variables to be container friendly, I think Jayanga do have a >> point. When we are troubleshooting an issue, the first thing to check is >> the configurations. When up configuration values are looked up via >> environment variables, how do we know what are the configuration parameters >> that are used in the system? One possible option is to write to a log as >> Chamila has mentioned, or else write the currently used config params to a >> log file at startup. WDYT? However this does not take into account the >> possibility of changing an environment value even after the server has >> started. >> > > @Isuru, it is coming with correct namespace right? and first priority is > given to environment variables. So there should not be an issue with > identifying the our config from environment variables IMO. > Yes. What I want to highlight is the troubleshooting perspective. We can have a particular set of config values coming from environment variables, another set of config values from deployment.yaml and defaults from the code. If we have log file file with all the currently used configurations values (irrespective of whether they are coming from environment variables, deployment.yaml or defaults in code), which can be written at server startup, it would be useful for the troubleshooting purpose IMHO. > >> On Wed, Aug 23, 2017 at 1:35 PM, Chamila De Alwis <[email protected]> >> wrote: >> >>> Hi Jayanga, >>> >>> From Support's point of view, requesting all the data that can have an >>> effect is not an overhead IMO. Especially since we already do that now, we >>> request for possible environmental factors, at least after all other >>> efforts are exhausted. When this change is introduced, we would not >>> additionally ask for anything. >>> >>> Furthermore, environment variables would mainly be used in scenarios >>> where Container Cluster Management systems are used, in which case there is >>> a clear cut definition of which environment variables with what values are >>> being applied to Containers. This is how most systems are debugged in CMSs >>> like Kubernetes even now, where commands like `kubectl describe` would give >>> you all the runtime information. >>> >>> Non-Containerized environments like VMs would mostly use the file based >>> approach where it is easy to be coupled with Config Automation tools like >>> Puppet. >>> >>> The concern of troubleshooting can be tackled by enabling a verbose log >>> that would display the environment variable name from which a certain value >>> was retrieved, at the time configs are loaded. This would obviously have to >>> be verbose than INFO, however it would greatly help any debugging process. >>> This is how tools like Puppet handle environment variable injected configs. >>> >>> The argument for file-less environment variables in the lookup hierarchy >>> is how quickly it becomes cumbersome to handle configuration files across >>> different Container images. Having files would soon present us with a >>> situation similar to what we handle now in C4 with configuration >>> repositories for different "patterns" of deployment in Containers, that >>> requires a separate effort to be maintained for different releases. Having >>> low-config convention-based environment variables support is a valuable >>> addition to the C5 configuration lookup model since we've already reduced >>> the number of configuration files. deployment.yaml will anyway be there, >>> and I'm willing to bet that most clients would use that to make >>> configuration changes. However, there are a set of clients who would >>> require the convention-based environment variables for a >>> smoother integration to Containerization than others. >>> >> >>> >>> Regards, >>> Chamila de Alwis >>> Committer and PMC Member - Apache Stratos >>> Senior Software Engineer | WSO2 >>> Blog: https://medium.com/@chamilad >>> >>> >>> >>> On Wed, Aug 23, 2017 at 1:03 PM, Jayanga Dissanayake <[email protected]> >>> wrote: >>> >>>> @Nuwan, configs should be able to inject via environment properties. >>>> What I am saying is all these configs should be specified the in the >>>> deployment.yaml as {env:placeholder name}. >>>> >>>> Thanks, >>>> Jayanga. >>>> >>>> *Jayanga Dissanayake* >>>> Associate Technical Lead >>>> WSO2 Inc. - http://wso2.com/ >>>> lean . enterprise . middleware >>>> email: [email protected] >>>> mobile: +94772207259 <+94%2077%20220%207259> >>>> <http://wso2.com/signature> >>>> >>>> On Wed, Aug 23, 2017 at 12:51 PM, Nuwan Dias <[email protected]> wrote: >>>> >>>>> @Jayanga, how do you make the server run elegantly on containers >>>>> without being able to inject properties via env variables? >>>>> >>>>> On Wed, Aug 23, 2017 at 12:43 PM, Jayanga Dissanayake < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> -1, for injecting configs without specifying it in the deployment.yaml >>>>>> >>>>>> Reason: When analyzing most of the issues we request config files >>>>>> (with C5 deployment.yaml) from the users to identify what are the >>>>>> configuration changes. If we have at-least the place holders in the >>>>>> config >>>>>> files, there is a guarantee that we know what are the injected >>>>>> configuration and we could request the actual values of the configs we >>>>>> need. >>>>>> >>>>>> But with the suggested approach, we have to request the 1)config >>>>>> files, 2)all the environment variables and get the union to identify the >>>>>> changed configurations. IMO an additional effort. >>>>>> >>>>>> Hence considering the overhead on the support front I am -1 for the >>>>>> suggested approach. >>>>>> >>>>>> Thanks, >>>>>> Jayanga. >>>>>> >>>>>> *Jayanga Dissanayake* >>>>>> Associate Technical Lead >>>>>> WSO2 Inc. - http://wso2.com/ >>>>>> lean . enterprise . middleware >>>>>> email: [email protected] >>>>>> mobile: +94772207259 <+94%2077%20220%207259> >>>>>> <http://wso2.com/signature> >>>>>> >>>>>> On Wed, Aug 23, 2017 at 9:16 AM, Danesh Kuruppu <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Created git issue[1] to track this improvement. >>>>>>> >>>>>>> 1. https://github.com/wso2/carbon-config/issues/48 >>>>>>> >>>>>>> On Tue, Aug 22, 2017 at 9:50 AM, Danesh Kuruppu <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Chamila, >>>>>>>> >>>>>>>> +1 for the suggested approach. So we can have both ways to set >>>>>>>> environment variables either from placeholder ${env:<key>} where we can >>>>>>>> specify any key for the variable or from the new approach. For the new >>>>>>>> approach, we need adhere to a certain format as suggested. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Danesh >>>>>>>> >>>>>>>> On Mon, Aug 21, 2017 at 5:59 PM, Chamila De Alwis < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> In C5, the configuration lookup is done in the environment >>>>>>>>> variables, system properties, deployment.yaml file, and the value >>>>>>>>> provided >>>>>>>>> in the configuration bean class, in that order. However, it looks like >>>>>>>>> environment variable lookup happens only if the particular >>>>>>>>> configuration is >>>>>>>>> noted to do so in the deployment.yaml, specifically if the value of >>>>>>>>> the >>>>>>>>> config is prefixed by a placeholder env:. >>>>>>>>> >>>>>>>>> gateWayEndpoint: ${env:gateWayEndpoint} >>>>>>>>> >>>>>>>>> While this pattern may be easy to implement, it is cumbersome to >>>>>>>>> be followed in a Containerization approach and IMO breaks the 12Factor >>>>>>>>> approach to configuration. This is because of the need to edit a >>>>>>>>> physical >>>>>>>>> file in order for the environment variables to be queried. This makes >>>>>>>>> it >>>>>>>>> harder to maintain a single Container Image for different >>>>>>>>> deployments, and >>>>>>>>> manipulate the Container runtimes via environment variables. >>>>>>>>> >>>>>>>>> AFAIU, this pattern emerges from the requirements in SecureVault; >>>>>>>>> having a ${sec:} prefix can mark the configuration value to be >>>>>>>>> resolved >>>>>>>>> through Secure Vault resolver. However IMO this, value resolution, is >>>>>>>>> not >>>>>>>>> the same as value lookup. >>>>>>>>> >>>>>>>>> IMO, the following should be the lookup order, without having to >>>>>>>>> mark a particular config in the deployment.yaml file. >>>>>>>>> >>>>>>>>> 1. Environment variables: PREFIX_wso2.NAMESPACE_CONFIGKEY="value", >>>>>>>>> PREFIX_wso2.NAMESPACE_CONFIGPARENT_CONFIG2="value2" >>>>>>>>> 2. deployment.yaml: >>>>>>>>> >>>>>>>>> wso2.NAMESPACE: >>>>>>>>> >>>>>>>>> CONFIG_KEY: value >>>>>>>>> >>>>>>>>> CONFIG_PARENT: >>>>>>>>> >>>>>>>>> CONFIG2: valu2 >>>>>>>>> >>>>>>>>> 3. Default value in the code >>>>>>>>> >>>>>>>>> This approach would be most Container friendly one as there is no >>>>>>>>> need to change files depending on deployment patterns, environments, >>>>>>>>> etc. >>>>>>>>> >>>>>>>>> WDYT? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Chamila de Alwis >>>>>>>>> Committer and PMC Member - Apache Stratos >>>>>>>>> Senior Software Engineer | WSO2 >>>>>>>>> Blog: https://medium.com/@chamilad >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Danesh Kuruppu* >>>>>>>> Senior Software Engineer | WSO2 >>>>>>>> >>>>>>>> Email: [email protected] >>>>>>>> Mobile: +94 (77) 1690552 <+94%2077%20169%200552> >>>>>>>> Web: WSO2 Inc <https://wso2.com/signature> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Danesh Kuruppu* >>>>>>> Senior Software Engineer | WSO2 >>>>>>> >>>>>>> Email: [email protected] >>>>>>> Mobile: +94 (77) 1690552 <+94%2077%20169%200552> >>>>>>> Web: WSO2 Inc <https://wso2.com/signature> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Nuwan Dias >>>>> >>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>> email : [email protected] >>>>> Phone : +94 777 775 729 <+94%2077%20777%205729> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >> >> >> -- >> Thanks and Regards, >> >> Isuru H. >> +94 716 358 048 <+94%2071%20635%208048>* <http://wso2.com/>* >> >> >> > > > -- > Lakmal Warusawithana > Director - Cloud Architecture; WSO2 Inc. > Mobile : +94714289692 <071%20428%209692> > Blogs : https://medium.com/@lakwarus/ > http://lakmalsview.blogspot.com/ > > -- Thanks and Regards, Isuru H. +94 716 358 048* <http://wso2.com/>*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
