Hi all,
As part of the HBase analytics datasource implementation for DAS 3.0, we
have come up with $subject which is envisioned to offer a standardised way
to specify connectivity parameters for a remote Hadoop-based instance in a
Carbon datasource configuration.
The datasource reader will expect the configuration to be specified in a
similar format which is used for standard Apache Commons Configuration [1],
as used by both HDFS and HBase. An example datasource definition would look
like:
<datasource>
<name>WSO2_ANALYTICS_FS_DB_HDFS</name>
<description>The datasource used for analytics file system</description>
<jndiConfig>
<name>jdbc/WSO2HDFSDB</name>
</jndiConfig>
<definition type="HADOOP">
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/dfs/data</value>
</property>
<property>
<name>fs.hdfs.impl</name>
<value>org.apache.hadoop.hdfs.DistributedFileSystem</value>
</property>
<property>
<name>fs.file.impl</name>
<value>org.apache.hadoop.fs.LocalFileSystem</value>
</property>
</configuration>
</definition>
</datasource>
The definition type for the above is set as "HADOOP". The datasource reader
implementation is currently hosted at [2], and would be merged with the
carbon-data git repo once reviewed.
Appreciate your thought and suggestions.
Thanks,
Gokul.
[1] http://commons.apache.org/proper/commons-configuration/
[2]
https://github.com/gokulbs/carbon-data/tree/master/components/data-sources/org.wso2.carbon.datasource.reader.hadoop
--
Balakrishnan Gokulakrishnan
Senior Software Engineer,
WSO2, Inc. http://wso2.com
Mob: +94 77 593 5789 | +1 650 272 9927
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
