Hi Niranda, i'll have a look.
Thanks and Regards. On Tue, Aug 11, 2015 at 7:13 AM, Niranda Perera <[email protected]> wrote: > Hi all, > > we have implemented a custom Spark JDBC connector to be used in the Carbon > environment. > > this enables the following > > 1. Now, temporary tables can be created in the Spark environment by > specifying an analytics datasource (configured by the > analytics-datasources.xml) and a table > 2. Spark uses "SELECT 1 FROM $table LIMIT 1" query to check the > existence of a table and the LIMIT query is not provided by all dbs. With > the new connector, this query can be provided with as a config. (this > config is still WIP) > 3. Adding new spark dialects related for various dbs (WIP) > > the idea is to test this for the following dbs > > - mysql > - h2 > - mssql > - oracle > - postgres > - db2 > > I have loosely tested the connector with MySQL, and I would like the APIM > team to use it with the API usage stats use-case, and provide us some > feedback. > > this connector can be accessed as follows. (docs are still not updated. I > will do that ASAP) > > create temporary table <temp_table> using CarbonJDBC options (dataSource > "<datasource name>", tableName "<table name>"); > > select * from <temp_table> > > insert into/overwrite table <temp_table> <some select statement> > > known limitations > > 1. when creating a temp table, it should already be created in the > underlying datasource > 2. "insert overwrite table" deletes the existing table and creates it > again > > > would be very grateful if you could use this connector in your current > JDBC use cases and provide us with feedback. > > best > -- > *Niranda Perera* > Software Engineer, WSO2 Inc. > Mobile: +94-71-554-8430 > Twitter: @n1r44 <https://twitter.com/N1R44> > https://pythagoreanscript.wordpress.com/ > -- Rukshan Chathuranga. Software Engineer. WSO2, Inc.
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
