On Wed, 24 Oct 2018 at 15:05, Dushani Wellappili <dusha...@wso2.com> wrote:
> Hi all, > > As discussed, now all API documentation data will be stored in only one > table, which is the AM_API_DOCS with the following table schema. > > `AM_API_DOCS` ( > `UUID` VARCHAR(255), > `API_ID` VARCHAR(255), > `NAME` VARCHAR(255), > `SUMMARY` VARCHAR(1024), > `TYPE` VARCHAR(255), > `OTHER_TYPE_NAME` VARCHAR(255), > `CONTENT` VARCHAR(1024), > `CONTENT_BINARY_VALUE` LONGBLOB, > `FILE_NAME` VARCHAR(255), > `SOURCE_TYPE` VARCHAR(255), > `VISIBILITY` VARCHAR(30), > `AM_DOC_PERMISSION` int(11) DEFAULT '7', > CREATED_BY VARCHAR(100), > CREATED_TIME TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP(6), > UPDATED_BY VARCHAR(100), > LAST_UPDATED_TIME TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP(6), > PRIMARY KEY (`UUID`), > FOREIGN KEY (`API_ID`) REFERENCES `AM_API`(`UUID`) ON UPDATE CASCADE ON > DELETE CASCADE > ) > > The REST API call for adding documentation will add the values except for > CONTENT_BINARY_VALUE column when SOURCE_TYPE is 'FILE'. The CONTENT column > will have either the documentation URL or the inline content. > > When adding a documentation of source type either URL or INLINE, there > will be only REST call needed which is to add a new Document to API. When > adding a documentation of source type 'FILE', we would first do the 'Add a > new Document to API' REST call and then do the 'Upload content of an API > document' which will update the same record's CONTENT_BINARY_VALUE column. > > Similarly, for the REST call of getting a document of an API, we will get > values except the value in CONTENT_BINARY_VALUE column. For the REST call > of getting content of the document of an API, the CONTENT_BINARY_VALUE > column value will be retrieved after checking whether the SourceType is > 'FILE'. > > One of the few concerns I have on the above design is, whether it is okay > to retrieve both URL value and inline content in 'Get a document of an API' > call which we expect only to retrieve API doc metadata. If so, I suppose we > can't use the term API doc metadata for that REST API operation description > anymore? If not, we can change the REST API to only retrieve doc metadata > without the CONTENT column value and retrieve doc content for all 3 types > of docs using 'Get the content of an API document' REST call. > This table is no longer just a meta data table so Im +1 for stopping the use of the word meta data in the REST API. We need to be able to retrieve the Inline and URL content in asingle call when we are listing the documents in the UI. The other concern is, in the above schema, we have only one of UPDATED_BY, LAST_UPDATED_TIME values for each document. So for both REST calls of getting the document (metadata) and getting document content data, is it okay to check with the same fingerprint (LAST_UPDATED_TIME), or do we need to add another separate audit columns for content updates as CONTENT_LAST_UPDATED_TIME, CONTENT_UPDATED_BY? We can simply update UPDATED_BY, LAST_UPDATED_TIME again at the time when we actually do a document upload. So that way the fingerprint will always be the latest. No need to add another column since the meta and content data are actually related. Appreciate your response. Thank you *Dushani Wellappili* Software Engineer - WSO2 Email : dusha...@wso2.com Mobile : +94779367571 Web : https://wso2.com/ On Tue, Oct 23, 2018 at 11:09 AM Harsha Kumara <hars...@wso2.com> wrote: > > > On Tue, Oct 23, 2018 at 10:07 AM Uvindra Dias Jayasinha <uvin...@wso2.com> > wrote: > >> @Bhathiya Jayasekara <bhath...@wso2.com> >> yes, this is also an option. In reality though very few customers upload >> lots of additional docs. We could just rename DOC_META_DATA table to DOCS >> and store the docs in that table itself. So at the very minimum we have a >> swagger and ballerina definition per an API stored in the RESOURCES table. >> > Definitely we should have a separate table for docs. > >> >> Can we make a call on this soon? Implementation is getting delayed >> because of this. >> >> On Mon, 22 Oct 2018 at 21:08, Bhathiya Jayasekara <bhath...@wso2.com> >> wrote: >> >>> This looks ok. Btw, I was wondering if it's a good idea to have all >>> files (swaggers, wsdls, docs etc.) in a single table. Won't that make the >>> table grow faster and make querying slower? How about keeping a separate >>> table for this? Then we can have a foreign key as well. >>> >>> Thanks, >>> Bhathiya >>> >>> On Mon, Oct 22, 2018 at 2:57 PM Uvindra Dias Jayasinha <uvin...@wso2.com> >>> wrote: >>> >>>> +1, CONTENT is fine >>>> >>>> On Mon, 22 Oct 2018 at 14:45, Mushthaq Rumy <musht...@wso2.com> wrote: >>>> >>>>> Hi Uvindra, >>>>> >>>>> Please find the data types below >>>>> >>>>> *AM_API_DOC_META_DATA* >>>>> API_ID (This will be a foreign key referred to AM_API table) >>>>> VARCHAR(255) >>>>> RESOURCE_ID VARCHAR(255) >>>>> DOC_CONTENT VARCHAR(1024) >>>>> >>>>> *AM_API_RESOURCES* >>>>> RESOURCE_NAME VARCHAR(255) >>>>> >>>>> And I think we can change it DOC_CONTENT to CONTENT . >>>>> WDYT? >>>>> >>>>> Thanks & Regards, >>>>> Mushthaq >>>>> >>>>> On Mon, Oct 22, 2018 at 2:35 PM Uvindra Dias Jayasinha < >>>>> uvin...@wso2.com> wrote: >>>>> >>>>>> HI Mushtaq, >>>>>> >>>>>> Can you provide the data types of the columns so that this is more >>>>>> clear? I believe that DOC_CONTENT should be a VARCHAR, in that case >>>>>> better >>>>>> to call it something like SHORT_CONTENT, since this communicates that it >>>>>> is >>>>>> meant to share relatively short text values such as URL and inline text. >>>>>> The name DOC_CONTENT is misleading and might suggest we can store larger >>>>>> documents in this column. >>>>>> >>>>>> On Mon, 22 Oct 2018 at 14:02, Mushthaq Rumy <musht...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> We had an internal discussion and decided to do some modifications >>>>>>> in the DB Schema related to API Docs (AM_API_RESOURCES and >>>>>>> AM_API_DOC_META_DATA). We will be removing the foreign key constraint >>>>>>> added >>>>>>> to the AM_API_DOC_META_DATA table as it it has a primary key referred >>>>>>> to >>>>>>> the primary key of AM_API_RESOURCES which is not a good practice. >>>>>>> >>>>>>> And the following columns will be added to the respective tables. >>>>>>> >>>>>>> *AM_API_DOC_META_DATA* >>>>>>> API_ID (This will be a foreign key referred to AM_API table) >>>>>>> RESOURCE_ID >>>>>>> DOC_CONTENT >>>>>>> >>>>>>> *AM_API_RESOURCES* >>>>>>> RESOURCE_NAME >>>>>>> >>>>>>> So the content of INLINE and URL of API documents will be saved in >>>>>>> *AM_API_DOC_META_DATA >>>>>>> (*DOC_CONTENT column*) *and content of FILE of API documents will >>>>>>> be saved in *AM_API_RESOURCES *which will have a reference in >>>>>>> *AM_API_DOC_META_DATA >>>>>>> (*RESOURCE_ID column*)*. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> Mushthaq >>>>>>> >>>>>>> >>>>>>> On Fri, Oct 19, 2018 at 9:23 PM Harsha Kumara <hars...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Oct 19, 2018 at 10:30 AM Uvindra Dias Jayasinha < >>>>>>>> uvin...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Here is my take on this. When I originally designed the schema I >>>>>>>>> wasn't taking into consideration any of the practical implications >>>>>>>>> associated with API resources being saved and retrieved at DB level. >>>>>>>>> But >>>>>>>>> now that we are at implementation stage some of these implications >>>>>>>>> are much >>>>>>>>> more clearer now. >>>>>>>>> >>>>>>>>> The AM_API_RESOURCES is a generic API resource table(For storing >>>>>>>>> all file based resources associated with APIs). It will be storing the >>>>>>>>> Swagger file, Ballerina file and documentation associated with the >>>>>>>>> API. >>>>>>>>> >>>>>>>>> The AM_API_DOC_META_DATA table is specialized to store additional >>>>>>>>> meta data only associated with documentation. >>>>>>>>> >>>>>>>>> Practically we need to do two calls for document uploads and >>>>>>>>> adding meta data because we are dealing with two different content >>>>>>>>> types(application/json for meta data and multipart/form-data for the >>>>>>>>> file). >>>>>>>>> >>>>>>>>> All files have a name associated with them so it makes sense to >>>>>>>>> have the file name in the AM_API_RESOURCES table. I don't think its a >>>>>>>>> good >>>>>>>>> idea to have a NULL value in a column that we are going to update >>>>>>>>> later, >>>>>>>>> this could lead to all kinds of complications that we will need to >>>>>>>>> handle >>>>>>>>> at code level. So its better to have the file name in AM_API_RESOURCES >>>>>>>>> where we can ensure that we always have a valid name at the time of >>>>>>>>> upload. >>>>>>>>> It is also very easy for us to enforce that a file name for a given >>>>>>>>> type >>>>>>>>> does not get duplicated with a table level constraint if we go with >>>>>>>>> this >>>>>>>>> option. >>>>>>>>> >>>>>>>>> Joining between two tables like this in case we need to get the >>>>>>>>> file name is trivial so I don't think we should let that affect us >>>>>>>>> coming >>>>>>>>> up with the best possible solution. >>>>>>>>> >>>>>>>> +1 it's a not good practice to add record which will update from >>>>>>>> null to some value cause of update going for another table. >>>>>>>> >>>>>>>>> >>>>>>>>> So Im +1 for option 2. WDYT? >>>>>>>>> >>>>>>>>> On Thu, 18 Oct 2018 at 17:31, Mushthaq Rumy <musht...@wso2.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Adding @dev-wso2 <d...@wso2.org> >>>>>>>>>> >>>>>>>>>> On Thu, Oct 18, 2018 at 5:25 PM Nuwan Dias <nuw...@wso2.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Please discuss technical problems externally. >>>>>>>>>>> >>>>>>>>>>> On Thu, Oct 18, 2018 at 3:44 PM Mushthaq Rumy <musht...@wso2.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> While I was implementing the view page for API document (File) >>>>>>>>>>>> I came across an issue where we get the file name as null when >>>>>>>>>>>> using the >>>>>>>>>>>> micro-service to get the content of the the API document. >>>>>>>>>>>> While analyzing, when adding a file as an API document, I found >>>>>>>>>>>> out that first we save only the doc metadata and then we save the >>>>>>>>>>>> file >>>>>>>>>>>> content using a second call. >>>>>>>>>>>> >>>>>>>>>>>> After analyzing the DB scripts I figured out that the fileName >>>>>>>>>>>> is stored in AM_API_DOC_META_DATA table and the content is stored >>>>>>>>>>>> in >>>>>>>>>>>> AM_API_RESOURCES. So during the first call we do not have the file >>>>>>>>>>>> name and >>>>>>>>>>>> it is saved as null. During the second call the fileName is passed >>>>>>>>>>>> to the >>>>>>>>>>>> micro-service but it is not stored anywhere. Hence, the fileName >>>>>>>>>>>> is null >>>>>>>>>>>> when we get the content of the file. So to solve this issue, I >>>>>>>>>>>> thought of >>>>>>>>>>>> two solutions. >>>>>>>>>>>> >>>>>>>>>>>> 1. During the second call while adding a file document for API >>>>>>>>>>>> as we get the FileName to the micro-service we can retrieve the >>>>>>>>>>>> document >>>>>>>>>>>> metadata using the documentId and update the fileName apart from >>>>>>>>>>>> saving the >>>>>>>>>>>> content. Hence, it will be available when retrieving the content. >>>>>>>>>>>> >>>>>>>>>>>> 2. We can change the fileName field from AM_API_DOC_META_DATA >>>>>>>>>>>> to AM_API_RESOURCES as the content of the document is stored in >>>>>>>>>>>> this table. >>>>>>>>>>>> And while saving the content we can save it with the fileName. >>>>>>>>>>>> Hence, it >>>>>>>>>>>> will be available when retrieving the content. >>>>>>>>>>>> >>>>>>>>>>>> IMO as option 1 will have more DB calls, option 2 would be the >>>>>>>>>>>> preferred solution. >>>>>>>>>>>> >>>>>>>>>>>> Appreciate your valuable inputs. >>>>>>>>>>>> >>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>> Mushthaq >>>>>>>>>>>> -- >>>>>>>>>>>> Mushthaq Rumy >>>>>>>>>>>> *Senior Software Engineer* >>>>>>>>>>>> Mobile : +94 (0) 779 492140 >>>>>>>>>>>> Email : musht...@wso2.com >>>>>>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>>>>>> lean . enterprise . middleware. >>>>>>>>>>>> >>>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Nuwan Dias* | Director | WSO2 Inc. >>>>>>>>>>> (m) +94 777 775 729 | (e) nuw...@wso2.com >>>>>>>>>>> [image: Signature.jpg] >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Mushthaq Rumy >>>>>>>>>> *Senior Software Engineer* >>>>>>>>>> Mobile : +94 (0) 779 492140 >>>>>>>>>> Email : musht...@wso2.com >>>>>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>>>>> lean . enterprise . middleware. >>>>>>>>>> >>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Uvindra >>>>>>>>> >>>>>>>>> Mobile: 777733962 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> *Harsha Kumara* >>>>>>>> >>>>>>>> Associate Technical Lead, WSO2 Inc. >>>>>>>> Mobile: +94775505618 >>>>>>>> Email: hars...@wso2.coim >>>>>>>> Blog: harshcreationz.blogspot.com >>>>>>>> >>>>>>>> GET INTEGRATION AGILE >>>>>>>> Integration Agility for Digitally Driven Business >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Mushthaq Rumy >>>>>>> *Senior Software Engineer* >>>>>>> Mobile : +94 (0) 779 492140 >>>>>>> Email : musht...@wso2.com >>>>>>> WSO2, Inc.; http://wso2.com/ >>>>>>> lean . enterprise . middleware. >>>>>>> >>>>>>> <http://wso2.com/signature> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Uvindra >>>>>> >>>>>> Mobile: 777733962 >>>>>> >>>>> >>>>> >>>>> -- >>>>> Mushthaq Rumy >>>>> *Senior Software Engineer* >>>>> Mobile : +94 (0) 779 492140 >>>>> Email : musht...@wso2.com >>>>> WSO2, Inc.; http://wso2.com/ >>>>> lean . enterprise . middleware. >>>>> >>>>> <http://wso2.com/signature> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Uvindra >>>> >>>> Mobile: 777733962 >>>> >>> >>> >>> -- >>> *Bhathiya Jayasekara* >>> *Technical Lead,* >>> *WSO2 inc., http://wso2.com <http://wso2.com>* >>> >>> *Phone: +94715478185* >>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj >>> <http://www.linkedin.com/in/bhathiyaj>* >>> *Twitter: https://twitter.com/bhathiyax <https://twitter.com/bhathiyax>* >>> *Blog: http://movingaheadblog.blogspot.com >>> <http://movingaheadblog.blogspot.com/>* >>> >> >> >> -- >> Regards, >> Uvindra >> >> Mobile: 777733962 >> > > > -- > > *Harsha Kumara* > > Associate Technical Lead, WSO2 Inc. > Mobile: +94775505618 > Email: hars...@wso2.coim > Blog: harshcreationz.blogspot.com > > GET INTEGRATION AGILE > Integration Agility for Digitally Driven Business > -- Regards, Uvindra Mobile: 777733962
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
