No Sanjeewa, in the method I'm proposing the system "will not break" even if someone goes and puts Japanese characters in the config file. That is by design.
One design principle from 3.0.0 onwards is to have no migration script involved. In the method I'm proposing we avoid migration 100% (for this part). I personally think that is a huge gain. On Tue, May 9, 2017 at 8:52 PM, Sanjeewa Malalgoda <[email protected]> wrote: > > > On Tue, May 9, 2017 at 4:57 PM, Nuwan Dias <[email protected]> wrote: > >> Regarding adding entries to the config file, you don't need to even open >> the swagger file. What you need to do is to find the resource from the docs >> and enter it into the config file. By expecting a sys admin to edit the >> Swagger file my main worry is we're expecting the sys-admin to play with an >> extremely critical file in the system and expecting him to behave well (no >> funky mistakes). If he or an automation tool does 1 silly mistake, the >> system will basically not work. I think that is a very fragile design. >> > This issue is same for config file as well IMO. If they made simple > mistake to config then parser will give error. Its extremely critical when > it comes to skeleton, client generation etc only. If we consider this > specific use case its only used for storing resource path and scopes. My > main worry is redoing what swagger is already support by default(and it > will be same for future 3.0.0 version as well). And we have parser, > execution logic and everything we need for this(tested and working fine). > In future with OAS 3.0.0 we will get parser and we can use same. When we > have that why do we need to maintain extra file + parser + object model > etc? > >> >> Regarding migration, my objective is to avoid "the need to migrate". From >> my experience every migration script comes with a huge risk. If there's a >> simple issue in the migration the whole system doesn't work. So if we have >> a design that basically ensures the system works without having to run a >> migration, that is very solid. If we maintain a custom config file, user's >> simply have to move it to the new version without having to touch it at >> all. If someone screws up trying to edit that file it doesn't impact the >> behavior of the server because >> > > >> if the server cannot find a matching resource in the config file it'll >> simply default to the resources in the Swagger we ship with the product. >> > To support above what we need is, swagger file based permission model > implementation(what option 01 propose). So we will have to keep swagger > based implementation anyway. > So there is no significant advantage with having config file. We will have > to maintain both implementation, fix issues in 2 different flows and user > need to aware about this additional config. > > When we go to swagger 2.0.0 to OAS 3.0.0 we anyway need migration and we > cannot avoid that. Since all APIs having swagger representation we must do > that and we cannot avoid it under any circumstance. So we can use exact > same tool for this migration. There will not be any additional work with > this approach. > > >> >> In summary, whoever or whatever screws up with the config file doesn't >> impact the server since we default to the standard swagger file in the >> system. But if we allow folks to play with the swagger we basically have to >> hope and pray people don't break it and make a promise to ourselves that we >> will seamlessly migrate user changes across all versions. >> > If someone break swagger syntax it will get parser error and its same for > config file as well. So that is a common problem for any model. In this > case we read only paths and scopes from swagger file. So if syntax correct > and have correct path scope everything will work. If you modify other place > no effect or change to system as skeletons already generated. We can think > of this as same config file with swagger notation(what people already know > and we have tool to parse and build data model). > > > >> >> On Tue, May 9, 2017 at 4:14 PM, Sanjeewa Malalgoda <[email protected]> >> wrote: >> >>> Previous one mistakenly sent before type everything. >>> >>> On Tue, May 9, 2017 at 3:41 PM, Sanjeewa Malalgoda <[email protected]> >>> wrote: >>> >>>> >>>> >>>> On Tue, May 9, 2017 at 12:59 PM, Nuwan Dias <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Tue, May 9, 2017 at 12:26 PM, Sanjeewa Malalgoda <[email protected] >>>>> > wrote: >>>>> >>>>>> >>>>>> >>>>>> On Tue, May 9, 2017 at 11:52 AM, Nuwan Dias <[email protected]> wrote: >>>>>> >>>>>>> There are several problems in allowing to edit the Swagger file >>>>>>> directly. >>>>>>> >>>>>>> 1. We change it on new product releases. So now users have to find a >>>>>>> way to merge whatever their changes when doing product upgrades. This is >>>>>>> error prone. >>>>>>> >>>>>> If we changed resource paths then user need to update configuration >>>>>> file as well. >>>>>> >>>>> >>>>> We can only "change" resource paths on a major version upgrade. Minor >>>>> upgrades are fare more frequent. So I don't think we will ever encounter a >>>>> situation where we have to deal with this unless on a major version >>>>> upgrade. >>>>> >>>> If its minor change then we may not need to edit swagger file. If path >>>> mapping not changed we dont need to worry about it in same way we do for >>>> config file. Users cannot see or use this swagger file. >>>> >>>>> >>>>>>> 2. The Swagger file is 100s of lines long and it has lots of content >>>>>>> the sys-admin doesn't really need to know/understand in order to change >>>>>>> the >>>>>>> resource to scope mapping. So its too hard for him to configure it. >>>>>>> Plus, I >>>>>>> think its too risky. Ex: if he accidentally removes a path param, the >>>>>>> generated code and swagger will be out of sync and result in weird >>>>>>> errors. >>>>>>> >>>>>> Having many lines in swagger is a problem when we need to edit. Apart >>>>>> from that this is common to config file as well. He may randomly delete >>>>>> path from that config file and things may not synchronize. >>>>>> >>>>> >>>>> I don't think so. In reality, we're talking about editing 5 lines vs >>>>> 100s. And most of it is stuff that doesn't interest the sys-admin >>>>> >>>> Even if you need to change 5 lines you have to go through swagger file >>> to see supported resource paths, HTTP verbs etc. Once you go through >>> swagger file only you can modify configuration file. >>> So why dont we directly edid then and there. Sys-admins are smart people >>> they can grep and find resource path easily and edit it :) >>> >>> If we have config file then then have to follow these steps, >>> >>> 1. Open swagger file(they need to find swagger definition from >>> somewhere), >>> 2. Then find resource path for required operation, >>> 3. Then copy resource path and paste it to configuration file >>> 4. Then add required scopes >>> >>> >>> If we let them to edit swagger, >>> >>> 1. Open swagger file available in conf directory, >>> 2. Then find resource path for required operation and update >>> scopes >>> >>> >>> So if i'm user then user i will choose option 2 because option one has >>> more work for me. >>> >>>> >>>>>>> 3. If we provide WUM updates to the Swagger file (for new features >>>>>>> or bug fixes), the sys-admin needs to sync up the Swagger file on each >>>>>>> update that has a change in the Swagger file. >>>>>>> >>>>>> If we read only path and mapped scopes from that file why do we need >>>>>> to worry about other changes. What sys admin suppose to do with swagger >>>>>> file? >>>>>> >>>>> >>>>> The other file will "only" have the ones the sys-admin wants to >>>>> override. Others will still default to the swagger file. sys-admin is not >>>>> supposed to touch the swagger file. The Swagger is for us (product) only. >>>>> >>>>> Ex: If sys-admin wants to say I need a different scope (foo) for Add >>>>> Application, there will be just 1 line in the config file. >>>>> >>>>> POST /application foo >>>>> >>>> If we add more conf file we need to parse it and build object model for >>> that. And if we have more scopes then you need to again edit this conf and >>> change object model. But if we use swagger then it will automatically build >>> nicely structured object model. We have already used this. >>> >>> Other point is request authentication interceptor anyway parse swagger >>> file to build object model. Then use that to validate scopes(this is >>> already implemented and using). If we have swagger based conf file then we >>> can simply point that swagger to this implementation and from that point >>> onward everything will be same as before. If we add new conf file then we >>> need to rewrite this whole logic. >>> >>>> >>>>>>> 4. We have to marry the sys-admin with Swagger-2.0. When we upgrade >>>>>>> to OAS 3.0, we have to expect the sys-admin to now know the syntax of >>>>>>> OAS >>>>>>> 3.0. Its better if we just make him aware of REST. >>>>>>> >>>>>> OAS 3.0.0 will not have huge syntax different with current swagger >>>>>> when it comes to resource paths. So i don't see issue with this. Its >>>>>> just a >>>>>> url path for them. And if they configure Nginx/F5 they need to deal with >>>>>> same. >>>>>> >>>>> >>>>> OAS 3.0 will break Swagger 2.0. [1] is an interview with Tony Tam >>>>> where he mentions "First, the 3.0 specification will be breaking from >>>>> 2.0. Many tooling vendors will be able to support both 2.0 and 3.0 >>>>> specifications with updated tooling, but 2.0 tooling will almost certainly >>>>> not work with 3.0 specifications." >>>>> >>>> He said tooling will failed because object model stricture was changed. >>> But open API v3 also have resource path, http verb, scopes like common >>> attributes. If we do swagger 2.0.0 to OAS 3.0.0 migration then API >>> definitions available in system need to migrate to OAS 3.0 seamlessly(we >>> need to support this with our migration client or WUM update). Then why >>> cant we point this swagger conf file to same client and get it to OAS >>> 3.0.0? Same scopes to resource mapping will be there without any issue if >>> we use same migration process with this file. >>> >>> Thanks, >>> sanjeewa. >>> >>>> >>>>> [1] - https://www.infoq.com/articles/open-api-initiative-update >>>>> >>>>>> >>>>>> Thanks, >>>>>> sanjeewa. >>>>>> >>>>>>> >>>>>>> On Tue, May 9, 2017 at 11:20 AM, Sanjeewa Malalgoda < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> IMO its more like keep different representation of same data in >>>>>>>> different location. To edit/update optional place you suggested user >>>>>>>> anyway >>>>>>>> need to refer swagger file. So why don't we simply let users to edit it >>>>>>>> self without having another file? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> sanjeewa. >>>>>>>> >>>>>>>> On Tue, May 9, 2017 at 11:01 AM, Nuwan Dias <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, May 9, 2017 at 10:46 AM, Sanjeewa Malalgoda < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, May 9, 2017 at 10:15 AM, Nuwan Dias <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> @Sanjeewa, if someone edits the Swagger file in conf, how do we >>>>>>>>>>> ensure the next restart doesn't override that file? >>>>>>>>>>> >>>>>>>>>> If file exists it will not override else it will write to file >>>>>>>>>> system. If its containerized automated deployment then automation >>>>>>>>>> process >>>>>>>>>> need to copy file. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> The root cause of the problem here is that the "resource to >>>>>>>>>>> scope" mapping is both a server configuration as well as it might >>>>>>>>>>> be a user >>>>>>>>>>> configuration when users want to find/corse grain the permissions >>>>>>>>>>> of the >>>>>>>>>>> API. >>>>>>>>>>> >>>>>>>>>> I think this is server configuration and not a user >>>>>>>>>> configuration. We should not users to edit this randomly. This >>>>>>>>>> mapping need >>>>>>>>>> to determine by system administrator by the time deployment >>>>>>>>>> initialize and >>>>>>>>>> should update only when required. Ex if you allow to update some >>>>>>>>>> resource >>>>>>>>>> to given role and randomly revoke that its not nice. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, by "user" I actually meant sys-admin. The guy configuring the >>>>>>>>> server actually. >>>>>>>>> >>>>>>>>>> >>>>>>>>>>> What if we allow the default resource to scope mapping remain in >>>>>>>>>>> the Swagger doc and introduce a new config file to override whatever >>>>>>>>>>> resource to scope mappings a user needs? To determine the scope of a >>>>>>>>>>> particular resource our code should first be checking the >>>>>>>>>>> custom/optional >>>>>>>>>>> config file and if an entry for the particular resource doesn't >>>>>>>>>>> exist, then >>>>>>>>>>> default to the Swagger file. >>>>>>>>>>> >>>>>>>>>> Having more config files will make things complicate IMO. >>>>>>>>>> >>>>>>>>> >>>>>>>>> If we copy the Swagger to the conf directory, that becomes a >>>>>>>>> duplicate config file anyway (one in the jar and one outside the >>>>>>>>> jar). And >>>>>>>>> when we do that we grant full access to the sys-admin to play with the >>>>>>>>> product rest API (the full Swagger doc). Which is not needed. What he >>>>>>>>> only >>>>>>>>> needs is to override the resource to scope mapping. Which I think >>>>>>>>> should be >>>>>>>>> possible to provide by having a optional place to override whatever >>>>>>>>> mapping >>>>>>>>> he needs only. >>>>>>>>> >>>>>>>>> We also need to think about product upgrades. If the sys admin >>>>>>>>> edits the swagger of one version and then upgrades the server, he'll >>>>>>>>> be >>>>>>>>> expected to merge whatever his changes with the newer swagger doc of >>>>>>>>> the >>>>>>>>> product. Which can be troublesome. But if we use an optional config >>>>>>>>> that >>>>>>>>> only overrides whatever resources he wants, he can simply use the >>>>>>>>> same file >>>>>>>>> on the new version of the product. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> sanjeewa. >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This way the optional config file only needs to bear the >>>>>>>>>>> resource to scope mappings that need to be overridden in a >>>>>>>>>>> particular >>>>>>>>>>> deployment and hence won't be that long. It also ensures that during >>>>>>>>>>> product upgrades users don't have to meddle with the current config >>>>>>>>>>> file >>>>>>>>>>> and can keep using it on newer versions too. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> NuwanD. >>>>>>>>>>> >>>>>>>>>>> On Mon, May 8, 2017 at 11:46 PM, Sanjeewa Malalgoda < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, May 8, 2017 at 3:42 PM, Ishara Cooray <[email protected] >>>>>>>>>>>> > wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> *Motivation:* >>>>>>>>>>>>> Before c5, API Manager product REST APIs resources have pre >>>>>>>>>>>>> defined scopes and they cannot be changed. >>>>>>>>>>>>> >>>>>>>>>>>>> But what if an admin needs to give access to Create, Update, >>>>>>>>>>>>> Delete actions to different users? >>>>>>>>>>>>> if he can customize the scopes associated with each resource, >>>>>>>>>>>>> then he will be able to fine grain the access to each resource. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> *Design:*With C5, we thought of allowing admin users to >>>>>>>>>>>>> add/change scopes in product REST APIs to meet their fine grained >>>>>>>>>>>>> requirements. >>>>>>>>>>>>> >>>>>>>>>>>>> At the moment we can think of two ways to do this. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. *Allow to edit the scopes defined per resource * >>>>>>>>>>>>> In this case we can copy the swagger file into conf >>>>>>>>>>>>> directory at build time, so that it can be maintained as a >>>>>>>>>>>>> usual >>>>>>>>>>>>> configuration file. >>>>>>>>>>>>> >>>>>>>>>>>>> When server startup we can copy these swagger files to conf >>>>>>>>>>>> directory and refer it from there. Maintain separate file for >>>>>>>>>>>> mapping make >>>>>>>>>>>> things more complex IMO. >>>>>>>>>>>> Lets follow this for all swagger based APIs. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> sanjeewa. >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 1. >>>>>>>>>>>>> 2. *Introduce a new config file to track resource to scope >>>>>>>>>>>>> mapping.* >>>>>>>>>>>>> In this case the issue is resource to scope mapping will >>>>>>>>>>>>> be duplicated. >>>>>>>>>>>>> >>>>>>>>>>>>> Appreciate your insight on this. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>>> Ishara Cooray >>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>> Mobile : +9477 262 9512 <+94%2077%20262%209512> >>>>>>>>>>>>> WSO2, Inc. | http://wso2.com/ >>>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> >>>>>>>>>>>> *Sanjeewa Malalgoda* >>>>>>>>>>>> WSO2 Inc. >>>>>>>>>>>> Mobile : +94713068779 <+94%2071%20306%208779> >>>>>>>>>>>> >>>>>>>>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>>>>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>>>>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Nuwan Dias >>>>>>>>>>> >>>>>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>>>>> email : [email protected] >>>>>>>>>>> Phone : +94 777 775 729 <077%20777%205729> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> *Sanjeewa Malalgoda* >>>>>>>>>> WSO2 Inc. >>>>>>>>>> Mobile : +94713068779 <+94%2071%20306%208779> >>>>>>>>>> >>>>>>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Nuwan Dias >>>>>>>>> >>>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>>> email : [email protected] >>>>>>>>> Phone : +94 777 775 729 <077%20777%205729> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Sanjeewa Malalgoda* >>>>>>>> WSO2 Inc. >>>>>>>> Mobile : +94713068779 <+94%2071%20306%208779> >>>>>>>> >>>>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Nuwan Dias >>>>>>> >>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>> email : [email protected] >>>>>>> Phone : +94 777 775 729 <077%20777%205729> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Sanjeewa Malalgoda* >>>>>> WSO2 Inc. >>>>>> Mobile : +94713068779 <+94%2071%20306%208779> >>>>>> >>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>>>> :http://sanjeewamalalgoda.blogspot.com/ >>>>>> <http://sanjeewamalalgoda.blogspot.com/> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Nuwan Dias >>>>> >>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>> email : [email protected] >>>>> Phone : +94 777 775 729 <077%20777%205729> >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> *Sanjeewa Malalgoda* >>>> WSO2 Inc. >>>> Mobile : +94713068779 <071%20306%208779> >>>> >>>> <http://sanjeewamalalgoda.blogspot.com/>blog >>>> :http://sanjeewamalalgoda.blogspot.com/ >>>> <http://sanjeewamalalgoda.blogspot.com/> >>>> >>>> >>>> >>> >>> >>> -- >>> >>> *Sanjeewa Malalgoda* >>> WSO2 Inc. >>> Mobile : +94713068779 <+94%2071%20306%208779> >>> >>> <http://sanjeewamalalgoda.blogspot.com/>blog >>> :http://sanjeewamalalgoda.blogspot.com/ >>> <http://sanjeewamalalgoda.blogspot.com/> >>> >>> >>> >> >> >> -- >> Nuwan Dias >> >> Software Architect - WSO2, Inc. http://wso2.com >> email : [email protected] >> Phone : +94 777 775 729 <077%20777%205729> >> > > > > -- > > *Sanjeewa Malalgoda* > WSO2 Inc. > Mobile : +94713068779 <+94%2071%20306%208779> > > <http://sanjeewamalalgoda.blogspot.com/>blog :http://sanjeewamalalgoda. > blogspot.com/ <http://sanjeewamalalgoda.blogspot.com/> > > > -- Nuwan Dias Software Architect - WSO2, Inc. http://wso2.com email : [email protected] Phone : +94 777 775 729
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
