Hi all, Following is an update on the current progress of this task.
Tasks done upto: 1. 2.6.0 REST API and implementation changes. - Change GET applications REST API to enable get all apps of a given tenant too, instead of all tenants - Support cross tenant APIs GET for migration only for super admin - Return application keys too (conditionally) to use for migration - Support cross tenant application export for migration for super admin 2. Export APIs as a bulk with export-apis command with import-export tool - All the APIs are exported by single command but implementation is as iterative api export, set by set separately (export sets of APIs separately, instead of exporting all at once) - API export resume capability (error handling) with the support of two files migration-apis-export-metadata.yaml and last-succeeded-api.log - *--force* flag with *export-apis* command will forcefully export all the apis from beginning, disregarding the fact that all/some apis were exported successfully by last execution of command. - Able to skip exporting particular APIs (if they were failed once and not needed to export if execute the command next time without --force option) 3. Fixing the gaps, which had blocked functionality in APIM 3.0.0 import-export tool related to API import/export process. Slight changes on the design of the approach is updated in the same doc shared previously. [1] Will be updating this thread on the progress and appreciate any concerns. [1] https://docs.google.com/document/d/1N8ZXPf1-dKpYf09w7AZ9M6RHXnsU2Orht37N1yETbnE/edit?usp=sharing Regards, Samitha On Fri, Sep 7, 2018 at 11:12 AM Samitha Chathuranga <[email protected]> wrote: > Hi all, > > This is an update on the up to now state on this task. Had several > discussions, design reviews within team and below doc [1] contains the > overall design of migration process. And each operation executed at API > export/import process (for migration) is detailed under Section 7 (*Using > the Import Export CLI tool for migration - User Steps and Steps executed in > each command*). > > Appreciate any comments and suggestions. > > Meanwhile I have worked on changing required REST APIs for this migration > process and are almost done. API/App export related operations from CLI > tool is done upto an extent allowing to be finalized based on finalized > design. > > [1] > https://docs.google.com/document/d/1N8ZXPf1-dKpYf09w7AZ9M6RHXnsU2Orht37N1yETbnE/edit?usp=sharing > > Regards, > Samitha > > On Wed, Aug 1, 2018 at 10:31 AM Samitha Chathuranga <[email protected]> > wrote: > >> Hi Nuwan, >> >> One main reason is that, if we follow approach 1, migration story cannot >> be done in single step. Specially two CLI tools will have to be used for >> the two sub steps of exporting and importing. >> >> And also normal API/Application export flow will change as major >> modifications will be required for certain aspects. i.e. >> >> - bulk export/import >> - correlate the difference in the format of archives supported by 2.6 >> vs 3.0 >> - exporting and importing additional artifacts such as throttle >> policies, access tokens, user role permissions >> >> Importing the 2.6 format API/Applications to 3.0 would require completely >> new effort too. So anyway we will have to change the 3.0 import tool highly. >> >> And my perception is that normal import export and migration oriented >> export-import are tow separate tasks and thought it is better to isolate >> the two objectives. (as similarly we do in earlier versions). We will also >> have to think about specific permissions required for the migration >> oriented export-import in same tool. >> >> Eventhough we may go with Approach 2, we can reuse the same code bases >> when possible. >> >> But anyway it is a matter of decision, whether we manage the normal >> import/export and migration-oriented-import/export separately. Approach 1 >> would result with some higher level of import/export while in approach 2, >> we could highlight it as a migration. >> >> Regards, >> Samitha >> >> On Tue, Jul 31, 2018 at 11:08 PM, Nuwan Dias <[email protected]> wrote: >> >>> I would rather go with approach 1. In approach 2 you are suggesting to >>> develop a separate web app and a separate tool, why do you think it is >>> better than enhancing the current App and CLI? Its obviously going to be a >>> lot more work than approach 1, plus at the end of the day we will throw >>> away one of those tools because we don't need two tools doing the same >>> thing. >>> >>> On Tue, Jul 31, 2018 at 10:10 PM Samitha Chathuranga <[email protected]> >>> wrote: >>> >>>> >>>> Hi all, >>>>> >>>>> This is regarding the initial design on possible approaches to proceed >>>>> with upgrading APIM 2.6.0(to be released) to 3.0.0. The migration process >>>>> has been already decided to implement through an export/import approach as >>>>> a database migration approach won't be realistic as for older versions. So >>>>> the simple generic approach is that each artifact specially APIs and >>>>> Applications would be exported from 2.6.0 environment and then will be >>>>> imported in to 3.0.0 environment. >>>>> >>>>> Below I have listed the possible two approaches for both these two >>>>> steps (importing & exporting) followed by their pros and cons. >>>>> >>>>> Exporting : >>>>> >>>>> Approach 1 : Change currently existing Api Import/export web app and >>>>> CLI tool in 2.6 >>>>> >>>>> >>>>> >>>>> - >>>>> >>>>> Can use API Import/export Web app and CLI tool and modify >>>>> - >>>>> >>>>> (REST APIs for export api/applications is currently provided >>>>> via this WEB app. CLI tool uses this web app) >>>>> - >>>>> >>>>> Required changes: >>>>> - >>>>> >>>>> For app export: Consumer key, Consumer Secret, Token exporting >>>>> should be supported in CLI tool >>>>> - >>>>> >>>>> Will have to change admin REST api implementation for above >>>>> task and for exporting other artifacts mentioned at the end. >>>>> - >>>>> >>>>> Modify CLI tool to support exporting in bulk. >>>>> >>>>> >>>>> >>>>> - >>>>> >>>>> Pros >>>>> - >>>>> >>>>> Will not have to develop/maintain a Web app and CLI tool from >>>>> the scratch >>>>> - >>>>> >>>>> Cons >>>>> - >>>>> >>>>> API export flow will change as modifications will be required >>>>> - >>>>> >>>>> Have to manage and consider also about normal export feature >>>>> when doing changes for migration-exporting. >>>>> - >>>>> >>>>> Migration story cannot be done in single step. Exporting and >>>>> importing will be 2 step process. >>>>> >>>>> >>>>> >>>>> Approach 2 : Develop a separate tool without using API Import/export >>>>> Web app or CLI tool. >>>>> >>>>> >>>>> - >>>>> >>>>> Create a WEB App introducing new REST APIs required and develop >>>>> separate CLI tool. >>>>> >>>>> >>>>> >>>>> - >>>>> >>>>> Pros >>>>> - >>>>> >>>>> Migration story can be done in single step. (Exporting and >>>>> importing can be done in single command.) >>>>> - >>>>> >>>>> API/Application import-export tool won’t have any effect or >>>>> change. >>>>> - >>>>> >>>>> Don’t have to manage 2 separate use cases in single tool. >>>>> - >>>>> >>>>> Cons >>>>> - >>>>> >>>>> Will have to develop/maintain a Web app and/or CLI tool from >>>>> the scratch >>>>> >>>>> >>>>> >>>>> Importing : >>>>> >>>>> >>>>> - >>>>> >>>>> Has to create separate REST APIs for importing 2.6-exported >>>>> APIs/Applications/other artifacts as 3.0.0 expecting archives in 3.0 >>>>> specific format. >>>>> >>>>> >>>>> Approach 1 >>>>> >>>>> - >>>>> >>>>> Change/improve API Import/export CLI tool in 3.0 to support >>>>> importing 2.6-exported API/application archives. >>>>> >>>>> Approach 2 >>>>> >>>>> - >>>>> >>>>> Develop separate CLI tool from the scratch which will be used for >>>>> exporting from 2.6 also >>>>> >>>>> >>>>> >>>>> Other artifacts required to be exported/imported (other than >>>>> APIs/Applications) >>>>> >>>>> >>>>> - >>>>> >>>>> Throttling Policies >>>>> - >>>>> >>>>> Custom API Lifecycles >>>>> - >>>>> >>>>> Workflows >>>>> - >>>>> >>>>> User Role Permissions >>>>> - >>>>> >>>>> Custom sequences (at least for reference -> has to decide >>>>> regarding this) >>>>> >>>>> >>>>> >>>>> My perception is that proceeding with Approach 2 under both exporting >>>>> and importing would be better considering the pros and cons of each. >>>>> >>>>> Highly appreciate your thoughts on this. >>>>> >>>>> Regards, >>>>> >>>>> Samitha >>>>> >>>>> >>>>> -- >>>>> *Samitha Chathuranga* >>>>> *Senior Software Engineer*, *WSO2 Inc.* >>>>> lean.enterprise.middleware >>>>> Mobile: +94715123761 >>>>> >>>>> [image: http://wso2.com/signature] <http://wso2.com/signature> >>>>> >>>> >>>> >>>> >>> >>> -- >>> Nuwan Dias >>> >>> Director - WSO2, Inc. http://wso2.com >>> email : [email protected] >>> Phone : +94 777 775 729 >>> >> >> >> >> -- >> *Samitha Chathuranga* >> *Senior Software Engineer*, *WSO2 Inc.* >> lean.enterprise.middleware >> Mobile: +94715123761 >> >> [image: http://wso2.com/signature] <http://wso2.com/signature> >> > > > -- > *Samitha Chathuranga* > *Senior Software Engineer*, *WSO2 Inc.* > lean.enterprise.middleware > Mobile: +94715123761 > > [image: http://wso2.com/signature] <http://wso2.com/signature> > -- *Samitha Chathuranga* *Senior Software Engineer*, *WSO2 Inc.* lean.enterprise.middleware Mobile: +94715123761 [image: http://wso2.com/signature] <http://wso2.com/signature>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
