Hi Sarangan, As per the current implementation, even the url based references are not getting resolved by the OAS Parser. We will need to address that requirement and support url based ref support in the default flow of swagger import as well.
For example, if an imported API definition(file/url) contains url based external references, we should resolve them before persisting it in the registry. In order to achieve this, *setResolve=true* option can be passed to OAS parser when parsing the content. Following is a sample code snippet you can refer to when modifying the default import/validation flow[1](apart from the archive-based approach you are implementing). OpenAPIV3Parser openAPIV3Parser = new OpenAPIV3Parser(); ParseOptions options = new ParseOptions(); options.setResolve(true); SwaggerParseResult parseAttemptForV3 = openAPIV3Parser.readContents(apiDefinition, null, options); [1] https://github.com/wso2/carbon-apimgt/blob/master/components/apimgt/org.wso2.carbon.apimgt.impl/src/main/java/org/wso2/carbon/apimgt/impl/definitions/OAS3Parser.java#L563 On Mon, May 11, 2020 at 1:09 PM Janakan Sarangan <[email protected]> wrote: > Hi ! > > + @[email protected] > > What is the current behavior of the OAS parser with external references? >> Does it throw validation exceptions or just ignore them? >> > > @Rukshan Premathunga <[email protected]> There's no exceptions in carbon > logs , but it will give a " *OpenAPI content validation failed*" error > message in the UI. > > Maybe we can identify the correct swagger by checking its swagger or >> openapi tag. There is a function already to check that. It may help in >> CI/CD flow if we didn't enforce naming conventions. >> > > Okay ! Then we will move from naming convention to validate through > openapi and swagger keywords. > +1. But in this approach, we will have to read the content of all the files available within the archive. Hence, try to come up with an optimized way of parsing these contents to identifying the parent file. > > Do we only store the flattened API definition or do we store both the >> parameterized definition and flattened definition? If we only store the >> flattened definition, how will we support editing of the references? >> > > @Ruwini Wijesiri <[email protected]> When we are saving the Api definition > we are only saving the flattened definition. That means Remote References > will be converted to local references . > > If we have a remote reference as below(Referencing Order element in > ref.yaml file), > > > *schema: $ref: "ref.yaml#/definitions/Order"* > > Then, the final swagger definition after resolving remote references will > be as below , > > > > > > > > > > > > *schema: $ref: '#/components/schemas/Order' components: schemas: > Order: properties: customerName: type: string > delivered: type: boolean* > > > Thus we can edit the references without any issues . This is the same as > editing local and URL references . > > On Mon, May 11, 2020 at 12:27 PM Thilini Shanika <[email protected]> > wrote: > >> Hi Sarangan >> >> This has to be a public mail :). As a practice, please make sure to >> initiate technical discussions in public mail threads. >> >> On Mon, May 11, 2020 at 10:40 AM Ruwini Wijesiri <[email protected]> wrote: >> >>> Hi janakan, >>> >>> >>>> - The extracted archive is processed and the open API definition is >>>> flattened by resolving the external references via the corresponding >>>> version of the OpenAPI Parsers. >>>> - The flattened openAPI definition will be saved as the definition >>>> of the API. >>>> >>>> Do we only store the flattened API definition or do we store both the >>> parameterized definition and flattened definition? If we only store the >>> flattened definition, how will we support editing of the references? >>> >>> What is the current behavior of the OAS parser with external references? >>>> Does it throw validation exceptions or just ignore them? >>> >>> @Rukshan Premathunga <[email protected]> i think it ignores it, doesn't >>> throw a validation exception. >>> >>> Regards, >>> Ruwini >>> >>> On Mon, May 11, 2020 at 10:34 AM Rukshan Premathunga <[email protected]> >>> wrote: >>> >>>> Hi Janakan, >>>> >>>> >>>> On Mon, May 11, 2020 at 10:23 AM Janakan Sarangan <[email protected]> >>>> wrote: >>>> >>>>> When importing an existing API with file or url based >>>>> external references , these references do not get resolved .We are >>>>> now supporting local reference and URL reference , but remote reference >>>>> also needs to be validated and handled in API definition [1] >>>>> >>>>> For file based external references [2], >>>>> >>>>> - If there are external ref files for a given open API definition, >>>>> those files have to be uploaded along with the master definition as an >>>>> archive. >>>>> - The archive will be copied to a temp location >>>>> - The extracted archive is processed and the open API definition >>>>> is flattened by resolving the external references via the corresponding >>>>> version of the OpenAPI Parsers. >>>>> - The flattened openAPI definition will be saved as the definition >>>>> of the API. >>>>> - We need to identify the master swagger file using a standard >>>>> naming convention or by using swagger validations .(Currently we are >>>>> hoping >>>>> to implement by enforcing some naming convention.*The master >>>>> swagger file needs to be named as main.yaml*) >>>>> >>>>> Maybe we can identify the correct swagger by checking its swagger or >>>> openapi tag. There is a function already to check that. It may help in >>>> CI/CD flow if we didn't enforce naming convention. >>>> >>>>> >>>>> - >>>>> >>>>> Your thoughts are highly appreciated in this proposed solution ! >>>>> >>>> What is the current behavior of the OAS parser with external >>>> references? Does it throw validation exceptions or just ignore them? >>>> >>>>> >>>>> [1] https://github.com/wso2/product-apim/issues/4464 >>>>> [2] https://swagger.io/docs/specification/using-ref/ >>>>> >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> *Sarangan* >>>>> *Software Engineer | APIM |** WSO2 Inc* >>>>> >>>>> *M : 0767882078 | E: [email protected] <[email protected]>* >>>>> >>>>> >>>>> *[image: http://wso2.com/signature] <http://wso2.com/signature>* >>>>> >>>> >>>> >>>> -- >>>> Rukshan C. Premathunga | Associate Technical Lead | WSO2 Inc. >>>> (m) +94711822074 | (w) +94112145345 | Email: [email protected] >>>> GET INTEGRATION AGILE >>>> Integration Agility for Digitally Driven Business >>>> >>> >>> >>> -- >>> Ruwini Wijesiri >>> Senior Software Engineer, >>> WSO2 Inc. >>> >>> Mobile : +94716133480 >>> >>> <http://wso2.com/signature> >>> >> >> >> -- >> Thilini Shanika >> Technical Lead >> WSO2, Inc.; http://wso2.com >> 20, Palmgrove Avenue, Colombo 3 >> Mobile: +94710892258 >> >> >> > > -- > Thanks and Regards, > *Sarangan* > *Software Engineer | APIM |** WSO2 Inc* > > *M : 0767882078 | E: [email protected] <[email protected]>* > > > *[image: http://wso2.com/signature] <http://wso2.com/signature>* > -- Thilini Shanika Technical Lead WSO2, Inc.; http://wso2.com 20, Palmgrove Avenue, Colombo 3 Mobile: +94710892258
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
