The main configuration point of analytics in BAM is its Hive scripts. At the moment we only support Hive language. But Hive has many drawbacks. For example there is no generic way to execute a SQL script on a RDBMS table in a Hive command. We had that requirement to execute the "DROP TABLE" query in Hive. Though there was a work around it was not a proper solution.
On the other hand AFAIK there is no proper listening mechanism for finish of execution of a Hive script. Such a technique is important to notify and/or pass data to another part of the WSO2 stack or call another part of the system. (e.g., invoke a proxy service in ESB after a certain Hive code line is executed) This kind of push modal is required for BAM when BAM is used for alerting scenarios. Yes, CEP is used for alerting but BAM can be used for not-urgent alerting. For example send an e-mail to marketing team after the sales has dropped to a certain level during one month. This can be done with BAM. Data push modal is required for reducing the latency as much as possible. I have come across another requirement for Class Analyzer / UDF with XML processing. Though simple XML can be processed with existing UDFs provided with Hive, XML strings with namespaces are not supported. We should add our own UDF for Hive level SOAP message processing. And also, already we pack a set of Class analyzers for IP to geo-location mapping and for deleting Cassandra rows for archiving purposes. Though they are packaged with BAM we do not explicitly mention about their existence in docs, though they are very useful for the users. I think it is the time to upgrade Hive script API we provide, with the new Class Analyzers / UDFs we have introduced. And also we should implement the Class Analyzers / UDFs for the commonly used scenarios such as the requirements I have mentioned earlier. This will improve the user experience by improving the userbility of Hive for broad range of solutions. And finally it will make the BAM analyzer similar to the Mediation Sequence in ESB which will improve the integration capability. Here are some of the Class analyzers / UDFs I think that will be useful for packing with BAM. 1. Executing a generic SQL statement (as described in the above example) 2. Web service / REST API calling (for invoking AS or ESB) 3. Java class execution - Similar to Jaggery Package host object. This should make it possible call any Java class in the OSGi environment 4. Thrift call execution - will help to pump data back to CEP and then take actions inside CEP 5. Registry value getting and setting - This will help a lot to share global values between Hive scripts while execution. 6. Hive script invoking - This will enable to asynchronously invoke another Hive script during execution of the original script. 7. Conditioning and iterating - will make it possible to stop/continue the script sequence by conditioning and will possible to execute a script as a for loop or while loop. There may be other useful features as well for defining the API. WDYT? * Maninda Edirisooriya* Software Engineer *WSO2, Inc. *lean.enterprise.middleware. *Blog* : http://maninda.blogspot.com/ *Phone* : +94 777603226
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
