Hi All, The "User Store" configuration can be considered as a deployment artifact if we look at the following aspects.
a) "Secondary User Stores" are added, removed and updated per tenant basis. (Same as SP, and IdP configs) b) "User Store", "IdP" and "SP", XACML policies, etc behaves as a collection of business rules, defining the authentication and authorization flows per the tenant. c) A change in a particular "User Store" usually affect the "Authentication" decisions done via SP config. Hence they have tight coupling. d) All "User Store", "SP", "IdP", etc, need to be taken as one unit, when we consider environment to environment promotion of these configs. (Dev->QA->staging->Prod) Hence IMHO, treating "User Store" as the file-based artifact was a right decision, when our products have been designed for deployment on bear-metal or VM. However moving forward to container, and cloud nativeness, posses the challenge on sharing these artifacts. Also, considering the CI/CD pipelines, the governance aspect of changing the configurations, etc, these type of configs need to be considered as artifacts. We might need to version control these artifacts in future and may need to push and pull them from VCS systems. What we need to do is to have a delegation pattern implemented for all current file based (and registry based artifacts), where we can switch the repository from file based one to different system. DB based repository would be the first such(simple) implementation. We may need to implement Git based repository when we properly support cloud use cases for example. We can abstract the storage system, and retain all the parsing and generation logic unchanged for artifacts. it would be a minimal change and most versatile way to extend IMO. We would need to implement property or "environment variable" binding logic, to get proper support for environment to environment promotion of artifacts. yet, it can be done with a separate effort than this IMO. Hence +1 to treat - Persist data as a blob (marshalled to text form) - In a separate table structure. Cheers, Ruwan A On Tue, Jun 4, 2019 at 3:50 PM Johann Nallathamby <joh...@wso2.com> wrote: > +1 to get rid of the artifacts for user stores. I think this was a wrong > decision we made early on. > > On Tue, Jun 4, 2019 at 1:19 PM Hasanthi Purnima Dissanayake < > hasan...@wso2.com> wrote: > >> Hi All, >> >> *Problem * >> Currently, some artifacts like userstores , tenants' data, etc are stored >> in the file system (not in the database). So when using a clustered setup >> those artifacts should be shared among all the nodes by using one of the >> following file sharing mechanisms. >> >> - Dep Sync >> - rSync >> - Shared File System >> >> >> *Solution * >> In order to avoid a shared file system and to reduce the deployment and >> maintenance overhead, those artifacts ca be persisted in the database >> itself. >> >> *Approach* >> After discussing with @Ruwan Abeykoon <ruw...@wso2.com> and @Isura >> Karunaratne <is...@wso2.com> we have two options to persist above >> discussed artifact details. >> >> - In the configuration store which is already implemented as >> discussed in [1][2]. >> - In a separate table structure. >> >> If we are to go with option 01, then we need to consider the artifacts as >> configurations and persist in the existing schema. >> > The advantage of using this is we can re-use the existing implementation >> including the database schema and existing rest APIs and functionalities >> (pagination, searching, etc) . >> > > >> The drawback is the conceptual difference between an artifact and >> configuration. >> > > I think this is not a problem. In fact I believe we made a wrong decision > of considering user stores as artifacts. I can't remember exactly as to why > we decided like that. User stores are not development artifacts; they are > one time configurations. They don't have metadata, versioning, lifecycles > or any other properties associated with other artifacts in WSO2. > > >> Further if we are to use the configuration store there is no way to >> include specific input validations for the user store configuration feature. >> > > I am not sure I completely understand this. But please check the SAML2 > metadata for identity provider feature where I believe we've done some > validations on the properties. I think they were done through interceptors > in identity provider service. > > >> >> If we are to go with the option 02, then the flow will be as follows. >> >> *Existing Flow* >> >> [image: Untitled Diagram (9).png] >> >> >> >> >> >> >> *Suggested Flow* >> >> [image: Untitled Diagram (10).png] >> >> >> >> - With the suggested approach, as the storage mechanisms, file system >> and database can be used and any other storage mechanism is pluggable. >> >> I don't agree with classifying user stores as artifacts. Therefore I > guess for me this option doesn't qualify at all :). > > However, just to understand this option better, have we come across any > customer who has asked us to support multiple types of storage mechanisms > for artifacts and/or configurations? Where is this requirement coming from? > I've only seen such requirements for user stores. For configurations I feel > this is just over engineering. May be it is a valid requirement for > artifacts? Even if we agree that there are valid reasons to support this > then it has to be supported for all configurations and/or artifacts. > >> >> - There should be a way to identify the repository where the data is >> loaded from. The repository can be the file system, database or any other >> storage mechanism. >> >> It sounds like this can get too complicated. > >> >> - In both the read write operations the enduser should have the >> control to decide the storage mechanism. >> >> Hmm.. this sounds more like a requirement to optimize database read write > performance. Doesn't sound right for artifacts. > >> >> - If the user needs to migrate a userstore from one storage mechanism >> (file system) to another then they can do it via UI. >> >> Again too many options for the user can make the product fragile. > > >> When persisting the data in the database there are two options we can use >> : >> >> - Persist data as a blob >> >> If we persist as blob then we lose the granular control over each > property for validation, transformation, etc. > >> >> - Persists data as key value pair >> >> +1 for this. > > >> If we are to go with the option one then we can persist the file as a >> blob and reuse most of the existing parsing logics. >> > > Given the understanding I think I prefer option 1 with properties. > > Thanks & Regards, > Johann. > > >> >> Highly appreciate your suggestions and feedbacks on the above approach. >> >> [1] [Architecture][IAM][JDBC based Configuration Store] Database Schema >> [2] [Architecture] [IS] JDBC based Configuration Store for WSO2 IS >> >> Thanks, >> Hasanthi >> >> -- >> >> Hasanthi Dissanayake | Senior Software Engineer | WSO2 Inc. >> (m) +94718407133 | (w) +94112145345 | Email: hasan...@wso2.com >> >> > > -- > *Johann Dilantha Nallathamby* | Associate Director/Solutions Architect | > WSO2 Inc. > (m) +94 (77) 7776950 | (w) +94 (11) 2145345 | (e) joh...@wso2.com > [image: Signature.jpg] > -- Ruwan Abeykoon | Director/Architect | WSO2 Inc. (w) +947435800 | Email: ruw...@wso2.com
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
