Hi all, This is to explain how ML components will be integrated to DAS from DAS 3.1.0 release.
There are 2 modes - Standalone mode - ML components will share the Spark context instance created by the DAS components. Advised to be used for development and testing environments. - Clustered mode - ML components and DAS components will have separate Spark contexts. This means that the Spark cluster will be separated out and the consumed by the 2 components separately. Important points to note: - In the clustered approach, there will be a clear resource separation for the 2 components. - ML components will be disabled by default and can be enabled by passing a java env variable - Care should be taken when allocating resources, since there will be 2 separate spark contexts in the cluster, and the ML instance needs to be taken out from the analyzer cluster Future developments: This approach will be changed later on, so that even in the clustered mode, ML components have the option of using the analytics spark context. Would like to know your inputs regarding this Best -- *Niranda Perera* Software Engineer, WSO2 Inc. Mobile: +94-71-554-8430 Twitter: @n1r44 <https://twitter.com/N1R44> https://pythagoreanscript.wordpress.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
