Hi Anjana, Yes. Requirement is for the internal log related REST API which is being written using osgi services. In the perspective of log analysis data, we have one master table to persist all the log events from different log sources. The way log data comes in to log REST API is as arbitrary fields. So different log sources have different set of arbitrary fields which leads log REST API to change the schema of master table every time it receives log events from a new/updated log source. That's what i meant inaccurate which can be solved much cleaner way by having that flag to index or not to index arbitrary fields for a particular stream.
Thanks, Malith On Tue, Dec 1, 2015 at 6:06 PM, Anjana Fernando <[email protected]> wrote: > Hi Malith, > > No, it cannot be done like that. How the indexing and all happens is, it > looks up the table schema for a table and do the indexing according to > that. So the table schema must be set before hand. It is not a dynamic > thing that can be set, when arbitrary fields are sent to the receiver, and > it cannot always load the current schema and set it always for each event, > even though we can cache that information and do some operations, but that > gets complicated. So the idea is, it is the responsibility of the client to > set the target table's schema properly before hand, which may or may not > include arbitrary fields, and then send the data. > > Also, if this requirement is for the log analytics solution work, as we've > discussed before, there should be a whole new remote API for that, and that > API can do these operations inside the server, using the OSGi services, and > not the original DAS REST API. So those operations will happen > automatically while keeping the remote log related API clean. > > Cheers, > Anjana. > > On Tue, Dec 1, 2015 at 5:13 PM, Malith Dhanushka <[email protected]> wrote: > >> Hi Folks, >> >> Currently indexing arbitrary fields is being achieved by dynamically >> updating analytics table schema through analytics REST API. This is not an >> accurate solution for a frequently updating schema. So the ideal solution >> would be to have a flag in data bridge event sink configuration to >> enable/disable indexing for all arbitrary fields. >> >> WDUT? >> >> Thanks, >> Malith >> -- >> Malith Dhanushka >> Senior Software Engineer - Data Technologies >> *WSO2, Inc. : wso2.com <http://wso2.com/>* >> *Mobile* : +94 716 506 693 >> > > > > -- > *Anjana Fernando* > Senior Technical Lead > WSO2 Inc. | http://wso2.com > lean . enterprise . middleware > -- Malith Dhanushka Senior Software Engineer - Data Technologies *WSO2, Inc. : wso2.com <http://wso2.com/>* *Mobile* : +94 716 506 693
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
