Hi Srinath, It would be great to have this capability to write queries on DAS for IoTS. However I see following blocks that will not make it directly possible. If we are to writing queries directly to DAS then we need to have an additional column of who is publishing the data and able to provide permission based on that and from what I see there are two problems.
1) We cant always get the username of the user who publishes the event. Thrift/http client publishes directly to DAS, which in case we can get the user, however if we use MQTT receiver then client publishes directly to the message broker and receiver subscribes to it. Therefore in this case AFAIK, there isn't a way to get the user for the subscriber on who publishes the data. Therefore from what I see we cant always get the username and this would be restricted with the communication paradigm that we are depending on(Pub/Sub or Request/Response). 2) Permission model to allow to view a set of users data to another user but not all. In here we need to allow another user to view a part of the data. Eg: Assume below is the table username deviceId …. Other attribute user X device 1 user X device 1 user X device 2 user Y device 3 User "X" can share his device to User "Z". So when user "X" shares his permission to view the data of device 1 to to User "Z", then User "Z" should only be allowed to view the data of the combination of username = user X and deviceId = device 1). Therefore from what I see we need a column/row level restriction - on which column against can a user write a query and the values of this row (eg in the above example username and device Id). Then we need to consider the values of this column which is user X and device 1. So I believe if we can find a solution for both of this then we can allow writing queries. Thanks Ayyoob *Ayyoob Hamza* *Software Engineer* WSO2 Inc.; http://wso2.com email: [email protected] cell: +94 77 1681010 <%2B94%2077%207779495> On Mon, Mar 28, 2016 at 4:26 PM, Nuwan Dias <[email protected]> wrote: > Having to publish a single event after collecting all possible data > records from the server would be good in terms of scalability aspects of > the DAS/Analytics platform. However I see that it introduces new challenges > for which we would need solutions. > > 1. How to guarantee a event is always published to DAS? In the case of API > Manager, a request has multiple exit points. Such as auth failures, > throttling out, back-end failures, message processing failures, etc. So we > need a way to guarantee that an event is always sent out whatever the state. > > 2. With this model, I'm assuming we only have 1 stream definition. Is this > correct? If so would this not make the analytics part complicated? For > example, say I have a spark query to summarize the throttled out events > from an App, since I can only see a single stream the query would have to > deal with null fields and have to deal with the whole bulk of data even if > in reality it might only have to deal with a few. The same complexity would > arise for the CEP based throttling engine and the new alerts we're building > as well. > > Thanks, > NuwanD. > > On Mon, Mar 28, 2016 at 2:43 PM, Srinath Perera <[email protected]> wrote: > >> Hi Ayyoob, Ruwan, Suho, >> >> I think where to handle ( within DAS vs. at higher level API in APIM or >> IoT server) is decided by what level user customizations are needed for >> analytics queries. >> >> If we need individual users to write their own queries as well, then we >> need to build user support into DAS. However, if queries can be changed by >> tenant admins only, doing this via a high-level API is OK. >> >> Where does APIM and IoT server stands on this? >> >> --Srinath >> >> >> >> On Sat, Mar 26, 2016 at 9:28 AM, Ayyoob Hamza <[email protected]> wrote: >> > >> > Hi, >> > Yes we require user level separation but just wondered whether we need >> this separation in DAS level or whether can we enforce it device type API >> level. This is because IMO, DAS provides a low level API which we cannot >> expose it directly so we need a proxy that maps this to a high level API to >> expose the data. So wondered whether can we do the restriction in the high >> level API endpoint. However if the user level separation is required across >> products such as APIM then I guess the separation should be in the DAS >> level. >> > >> > Further just wanted to bring another concern that we have, we have a >> requirement on device sharing so what this mean is that we can share the >> data of a device to another, which means a drill down permission model, >> where the separation would be in user, device level(eg: Does the user X has >> permission to view the data of the device d of user Y). So in this case I >> wonder whether this needs to be handled in DAS level? rather I see that it >> needs to be handled in the high level API that we provide to expose the >> data. >> > >> > Thanks >> > >> > >> > Ayyoob Hamza >> > Software Engineer >> > WSO2 Inc.; http://wso2.com >> > email: [email protected] cell: +94 77 1681010 >> > >> > On Sat, Mar 26, 2016 at 1:03 AM, Ruwan Yatawara <[email protected]> >> wrote: >> >> >> >> Hi Suho, >> >> >> >> Yes, you are right. We require user level isolation in IoT Server. >> >> >> >> Thanks and Regards, >> >> >> >> Ruwan Yatawara >> >> >> >> Senior Software Engineer, >> >> WSO2 Inc. >> >> >> >> email : [email protected] >> >> mobile : +94 77 9110413 >> >> blog : http://ruwansrants.blogspot.com/ >> >> www: :http://wso2.com >> >> >> >> >> >> On Fri, Mar 25, 2016 at 11:55 PM, Sriskandarajah Suhothayan < >> [email protected]> wrote: >> >>> >> >>> Hi >> >>> >> >>> User level isolation is needed for the IoT server, as in the IoT >> server context user registers a device and use that, hence he/she should >> only be able to see his/her devices and not any other users devices or >> data. >> >>> >> >>> @Pabath & Sumedha correct me if I'm wrong. >> >>> >> >>> Regards >> >>> Suho >> >>> >> >>> On Fri, Mar 25, 2016 at 9:02 AM, Srinath Perera <[email protected]> >> wrote: >> >>>> >> >>>> For the data published from APIM and IoT servers, what kind of >> isolation do we need? >> >>>> >> >>>> Option 1: Tenant level - DAS already has this. However, this means >> that multiple users (e.g. publishers, subscribers, or IoT users) can see >> other people's stats of they are in the same tenant >> >>>> >> >>>> Option 2: User level - DAS does not have this concept yet. >> >>>> >> >>>> Also a related question is that if user add their own queries, at >> what level they are isolated. >> >>>> >> >>>> --Srinath >> >>>> >> >>>> -- >> >>>> ============================ >> >>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> >>>> Site: http://home.apache.org/~hemapani/ >> >>>> Photos: http://www.flickr.com/photos/hemapani/ >> >>>> Phone: 0772360902 >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> S. Suhothayan >> >>> Technical Lead & Team Lead of WSO2 Complex Event Processor >> >>> WSO2 Inc. http://wso2.com >> >>> lean . enterprise . middleware >> >>> >> >>> cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >> >>> twitter: http://twitter.com/suhothayan | linked-in: >> http://lk.linkedin.com/in/suhothayan >> >>> >> >>> _______________________________________________ >> >>> Architecture mailing list >> >>> [email protected] >> >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >>> >> >> >> >> >> >> _______________________________________________ >> >> Architecture mailing list >> >> [email protected] >> >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> >> > >> > >> > _______________________________________________ >> > Architecture mailing list >> > [email protected] >> > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> > >> >> >> >> -- >> ============================ >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://home.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > Nuwan Dias > > Technical Lead - WSO2, Inc. http://wso2.com > email : [email protected] > Phone : +94 777 775 729 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
