Regarding the security, we will have to go with something like OAuth when security is needed, where we delegate the resource access to interested parties. i.e. Users will have an account under storage server, and anyone who access the resources on behalf of user, will have to get an access token(Anyway, invite Prabath as well for the meeting, so we can discuss it further).
When we are using this as the thumbnail storage mechanism of ES, then we can either have images as non-secured resources or ES itself can act as a storage server user. @Chan, please check with SS guys as well and everything you do should be a part of SS. Then we can have a comprehensive story for SS. /Ruchira On Wed, Mar 5, 2014 at 10:17 AM, Chan <[email protected]> wrote: > Nope. I talked with SameeraM regarding it and we found that starting from > scratch would be the best approach since it's complicated out by publisher > specific code. But the concept of putting it into the storage is the same > as what you guys did for ES. > > I think we should schedule a review regarding the unified storage and it's > use cases and of course the security aspect as well. > > Cheers~ > > > On Tue, Mar 4, 2014 at 7:47 PM, Nuwan Bandara <[email protected]> wrote: > >> Are we using any parts of the storage impl we did for ES >> >> Regards, >> /Nuwan >> >> >> On Mon, Mar 3, 2014 at 12:35 PM, Chan <[email protected]> wrote: >> >>> Hi folks, >>> I have finished the basic functionality of the Storage app[1]. Below are >>> the next set of functions I'll be working on. >>> >>> - Support h2 db (support h2 database) >>> - Support content-type storage (Store the content type of the file >>> for better rendering) >>> - Make engines and storage-providers different (engine stores UUIDs, >>> tokens etc where as storage-provider is the actual storage) >>> - Support consumer tokens and validation for consumers >>> - Route with given name - requirement that came from ES team >>> (storage/resource/:uuid/:filename) >>> >>> Another question I have is whether we have a component that deals with >>> token generation. I want to generate tokens for file requests. >>> >>> [1] - https://github.com/dulichan/storage >>> >>> Cheers~ >>> >>> >>> On Fri, Feb 28, 2014 at 9:03 PM, Shani Ranasinghe <[email protected]>wrote: >>> >>>> Hi Chan, >>>> >>>> In SS we have the HDFS browser facility. This is a feature where you >>>> can upload/download files to the HDFS. Please refer >>>> https://docs.wso2.org/display/SS110/HDFS+Browser. Please check if >>>> this fits your requirement. >>>> >>>> Also, as far as UES is concerned, implementing the HDFS UI as a jaggery >>>> app is a roadmap item https://redmine.wso2.com/issues/1744. >>>> >>>> Mutitenancy is yet to be completed and in the current release is not >>>> production recommended. With the introduction of Hadoop 2.x in SS (in a >>>> future release) we are planning to look into possibilities of federating >>>> the file system and thereby address multitenancy. It is yet to be decided. >>>> >>>> >>>> >>>> On Fri, Feb 28, 2014 at 6:33 AM, Chan <[email protected]> wrote: >>>> >>>>> *Simple spec for the Unified Storage proposal * >>>>> The goal of the Unified Storage proposal is to provide a REST API and >>>>> jaggery module wrapper which will provide below features - >>>>> >>>>> - Persist files >>>>> - Get files >>>>> >>>>> >>>>> *REST API* >>>>> * Persist file* >>>>> >>>>> URL - storage/resource/ >>>>> VERB - PUT >>>>> >>>>> Headers >>>>> >>>>> consumer token - Get the token of the provider >>>>> >>>>> consumer user - User that's authenticated to the service provider >>>>> >>>>> Body >>>>> >>>>> File data >>>>> >>>>> Response >>>>> >>>>> Returns id >>>>> >>>>> >>>>> *Get file* >>>>> >>>>> URL - storage/ >>>>> resource >>>>> /:id >>>>> >>>>> VERB - GET >>>>> Attributes >>>>> >>>>> id - The ID obtained from the PUT call >>>>> >>>>> Headers >>>>> >>>>> consumer token - Get the token of the provider >>>>> >>>>> consumer user - User that's authenticated to the service provider >>>>> >>>>> Response >>>>> >>>>> Returns the file >>>>> >>>>> *Jaggery Module* >>>>> A jaggery module will be implemented which wrap the REST API and >>>>> provide a developer friendly javascript api. >>>>> >>>>> var storage = require("storage").storage; >>>>> /* >>>>> To be configured with >>>>> { >>>>> provider-name: sdf >>>>> provider-secret: sdf >>>>> } >>>>> */ >>>>> storage.configure({}); >>>>> var file = new File(); >>>>> >>>>> /* >>>>> persist a file >>>>> meta will contain >>>>> { >>>>> id = id to access the file >>>>> } >>>>> */ >>>>> var meta = storage.put(file); >>>>> >>>>> /* >>>>> request a file by sending the file id >>>>> */ >>>>> var file = storage.get(id); >>>>> >>>>> >>>>> We can improve on the spec by providing security, multi-tenancy >>>>> *Buckets* >>>>> A bucket is a namespace for files or objects. This will be useful for >>>>> a use-case where namespaces for files are required. >>>>> >>>>> *Multi-tenancy* >>>>> There are some limitations in multi-tenancy in HDFS (@Deep or @Shani >>>>> can better explain this). We can implement the tenant isolation in the >>>>> storage app layer. >>>>> >>>>> *Security* >>>>> A token can be requested from storage for a resource before issuing >>>>> the url to a client. A use-case is where store takes a token and append it >>>>> to the url before sending it to the android device. There will be a >>>>> governance model on the time period and invalidation to secure the >>>>> executable resource. Also another layer of security will be implemented so >>>>> that Storage app can trust the Service-Provider (which is Publisher or >>>>> Store). >>>>> >>>>> Any ideas are welcome. >>>>> >>>>> Cheers~ >>>>> >>>>> >>>>> On Mon, Feb 24, 2014 at 1:23 PM, Chan <[email protected]> wrote: >>>>> >>>>>> @Dilshan as I have mentioned - SS only handles provisioning of >>>>>> storage. The proposed solution is for File Storage (not for >>>>>> provisioning). >>>>>> >>>>>> >>>>>> On Mon, Feb 24, 2014 at 12:51 PM, Dilshan Edirisuriya < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Chan, >>>>>>> >>>>>>> I think we discussed this few months ago with SS team (Prabath, >>>>>>> Deep). What was the final conclusion? >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Dilshan >>>>>>> >>>>>>> >>>>>>> On Thu, Feb 20, 2014 at 7:49 PM, Chan <[email protected]> wrote: >>>>>>> >>>>>>>> Hi folks, >>>>>>>> In our platform - we don't yet have a solution for storing static >>>>>>>> files in the cloud. A solution that provides a simple service of >>>>>>>> persisting >>>>>>>> a file and getting back a URI. The Enterprise Store team built a >>>>>>>> storage >>>>>>>> module in the Publisher app that provides persisting of files to a >>>>>>>> database >>>>>>>> and getting back URIs. My proposal is to build a jaggery-app that >>>>>>>> specifically handles file storage with a REST interface extending what >>>>>>>> the >>>>>>>> ES team has done. >>>>>>>> >>>>>>>> *Use case* >>>>>>>> I will be using the Enterprise Store as an example for the Storage >>>>>>>> App. >>>>>>>> [image: Inline image 1] >>>>>>>> 1:- Developer uploads a file to the publisher app using an entry >>>>>>>> form >>>>>>>> 2:- Publisher app calls the storage module to contact the storage >>>>>>>> provider >>>>>>>> 3:- Storage module contacts the Storage app with a HTTP PUT call >>>>>>>> and returns the URI of the file persisted. Publisher app will persist >>>>>>>> this >>>>>>>> URI to the registry along with other information. >>>>>>>> 4:- A client contacts the server requesting an asset page. >>>>>>>> 5:- The server will contact the storage module to request a token. >>>>>>>> It will in turn call the Storage app requesting a token. >>>>>>>> After retrieving the token It will append the token to the URI when >>>>>>>> rendering the pages server side. >>>>>>>> 6:- Browser will contact the Storage app directly when requesting >>>>>>>> the file using HTTP GET. Since the browser is passing a valid token - >>>>>>>> Storage app will return the file in the response. >>>>>>>> >>>>>>>> *Architecture overview* >>>>>>>> Storage app will have service providers. These service providers >>>>>>>> will be apps that require the Storage service from the Storage app. >>>>>>>> First >>>>>>>> let's start off with the API. Storage app will provide an HTTP >>>>>>>> interface as >>>>>>>> below >>>>>>>> GET >>>>>>>> >>>>>>>> GET interface is used to request a file. The request can be >>>>>>>> for permanent links or token based links. >>>>>>>> >>>>>>>> PUT >>>>>>>> >>>>>>>> PUT can only called by a registered service provider. A token >>>>>>>> (Provider token) has to be passed to validate the service provider. It >>>>>>>> will >>>>>>>> also allow options of - security, permanent links etc. >>>>>>>> >>>>>>>> DELETE >>>>>>>> >>>>>>>> URI is passed to the DELETE API as well as the provider token. Used >>>>>>>> to delete files. >>>>>>>> >>>>>>>> >>>>>>>> Next is what the real storage options are. For the first cut we can >>>>>>>> have the storage app connected to a database. Afterwards we can move to >>>>>>>> HDFS type storage or Cassandra perhaps. >>>>>>>> >>>>>>>> As with Multi-tenancy - we can handle multi-tenancy the same way we >>>>>>>> handle it for jaggery apps (SaaS multi-tenancy). >>>>>>>> >>>>>>>> Ultimate objective of this proposal is to build a cloud storage >>>>>>>> service like Amazon S3, Google Cloud storage [1]. WDYT guys? >>>>>>>> >>>>>>>> [1] - >>>>>>>> http://www.quora.com/Amazon-S3/What-are-the-best-alternatives-to-S3-and-what-are-the-pros-and-cons >>>>>>>> >>>>>>>> Cheers~ >>>>>>>> -- >>>>>>>> Chan (Dulitha Wijewantha) >>>>>>>> Software Engineer - Mobile Development >>>>>>>> WSO2Mobile >>>>>>>> Lean.Enterprise.Mobileware >>>>>>>> * ~Email [email protected] <[email protected]>* >>>>>>>> * ~Mobile +94712112165 <%2B94712112165>* >>>>>>>> >>>>>>>> * ~Website dulithawijewantha.com <http://dulithawijewantha.com/> >>>>>>>> * >>>>>>>> >>>>>>>> * ~Blog blog.dulithawijewantha.com >>>>>>>> <http://dulichan.github.io/chan/>* >>>>>>>> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dilshan Edirisuriya >>>>>>> Senior Software Engineer - WSO2Mobile >>>>>>> Mob: + 94 772245502 >>>>>>> http://wso2mobile.com/ >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Chan (Dulitha Wijewantha) >>>>>> Software Engineer - Mobile Development >>>>>> WSO2Mobile >>>>>> Lean.Enterprise.Mobileware >>>>>> * ~Email [email protected] <[email protected]>* >>>>>> * ~Mobile +94712112165 <%2B94712112165>* >>>>>> >>>>>> * ~Website dulithawijewantha.com <http://dulithawijewantha.com/> * >>>>>> >>>>>> * ~Blog blog.dulithawijewantha.com >>>>>> <http://dulichan.github.io/chan/>* >>>>>> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Chan (Dulitha Wijewantha) >>>>> Software Engineer - Mobile Development >>>>> WSO2Mobile >>>>> Lean.Enterprise.Mobileware >>>>> * ~Email [email protected] <[email protected]>* >>>>> * ~Mobile +94712112165 <%2B94712112165>* >>>>> >>>>> * ~Website dulithawijewantha.com <http://dulithawijewantha.com/> * >>>>> >>>>> * ~Blog blog.dulithawijewantha.com >>>>> <http://dulichan.github.io/chan/>* >>>>> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >>>>> >>>> >>>> >>>> >>>> -- >>>> Thanks and Regards >>>> *, Shani Ranasinghe* >>>> Software Engineer >>>> WSO2 Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> mobile: +94 77 2273555 >>>> linked in: lk.linkedin.com/pub/shani-ranasinghe/34/111/ab >>>> >>> >>> >>> >>> -- >>> Chan (Dulitha Wijewantha) >>> Software Engineer - Mobile Development >>> WSO2Mobile >>> Lean.Enterprise.Mobileware >>> * ~Email [email protected] <[email protected]>* >>> * ~Mobile +94712112165 <%2B94712112165>* >>> * ~Website dulitha.me <http://dulitha.me>* >>> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >>> *~SO @chan <http://stackoverflow.com/users/813471/chan>* >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> >> >> >> *Thanks & Regards,Nuwan BandaraTechnical Lead; **WSO2 Inc. * >> *lean . enterprise . middleware | http://wso2.com <http://wso2.com> * >> >> *blog : http://nuwanbando.com <http://nuwanbando.com>; email: >> [email protected] <[email protected]>; phone: +1 812 606 7390 >> <%2B1%20812%20606%207390> * >> <http://www.nuwanbando.com/> >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Chan (Dulitha Wijewantha) > Software Engineer - Mobile Development > WSO2Mobile > Lean.Enterprise.Mobileware > * ~Email [email protected] <[email protected]>* > * ~Mobile +94712112165 <%2B94712112165>* > * ~Website dulitha.me <http://dulitha.me>* > * ~Twitter @dulitharw <https://twitter.com/dulitharw>* > *~SO @chan <http://stackoverflow.com/users/813471/chan>* > -- *Ruchira Wageesha**Associate Technical Lead* *WSO2 Inc. - lean . enterprise . middleware | wso2.com <http://wso2.com>* *email: [email protected] <[email protected]>, blog: ruchirawageesha.blogspot.com <http://ruchirawageesha.blogspot.com>, mobile: +94 77 5493444*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
