A few documentation points
Project: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/commit/f64af570 Tree: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/tree/f64af570 Diff: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/diff/f64af570 Branch: refs/heads/master Commit: f64af570ed8f5dbed8a3bb942f753b0d656642a2 Parents: 1445d12 Author: Riccardo V. Vincelli <[email protected]> Authored: Tue Feb 9 13:10:13 2016 +0100 Committer: Riccardo V. Vincelli <[email protected]> Committed: Tue Feb 9 13:18:26 2016 +0100 ---------------------------------------------------------------------- docs/deployment-security.md | 2 +- docs/dev-connectors.md | 17 ++++++++++++++++- docs/dev-custom-serializer.md | 13 +++++++++++++ 3 files changed, 30 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-gearpump/blob/f64af570/docs/deployment-security.md ---------------------------------------------------------------------- diff --git a/docs/deployment-security.md b/docs/deployment-security.md index 331aeba..4be1346 100644 --- a/docs/deployment-security.md +++ b/docs/deployment-security.md @@ -55,7 +55,7 @@ provided for the gearpump-hbase connector. Specifically, the `UserConfig` object ```scala val principal = "gearpump/[email protected]" -val keytabContent = Files.toByteArray(new File("path_to_keytab_file)) +val keytabContent = Files.toByteArray(new File("path_to_keytab_file")) val appConfig = UserConfig.empty .withString("gearpump.kerberos.principal", principal) .withBytes("gearpump.keytab.file", keytabContent) http://git-wip-us.apache.org/repos/asf/incubator-gearpump/blob/f64af570/docs/dev-connectors.md ---------------------------------------------------------------------- diff --git a/docs/dev-connectors.md b/docs/dev-connectors.md index 458956b..a9983fc 100644 --- a/docs/dev-connectors.md +++ b/docs/dev-connectors.md @@ -104,7 +104,7 @@ To use `HBaseSink` in your application, you first need to add the `gearpump-exte To connect to HBase, you need to provide following info: - the HBase configuration to tell which HBase service to connect - - the table name + - the table name (you must create the table yourself, see the [HBase documentation](https://hbase.apache.org/book.html)) Then, you can use `HBaseSink` in your application: @@ -123,6 +123,21 @@ Then, you can use `HBaseSink` in your application: You can tune the connection to HBase via the HBase configuration passed in. If not passed, Gearpump will try to check local classpath to find a valid HBase configuration (`hbase-site.xml`). +Attention, due to the issue discussed [here](http://stackoverflow.com/questions/24456484/hbase-managed-zookeeper-suddenly-trying-to-connect-to-localhost-instead-of-zooke) you may need to create additional configuration for your HBase sink: + +```scala + def hadoopConfig = { + val conf = new Configuration() + conf.set("hbase.zookeeper.quorum", "zookeeperHost") + conf.set("hbase.zookeeper.property.clientPort", "2181") + conf + } +``` + +```scala + val sink = HBaseSink(UserConfig.empty, tableName, hadoopConfig) +``` + ## How to implement your own `DataSource` To implement your own `DataSource`, you need to implement two things: http://git-wip-us.apache.org/repos/asf/incubator-gearpump/blob/f64af570/docs/dev-custom-serializer.md ---------------------------------------------------------------------- diff --git a/docs/dev-custom-serializer.md b/docs/dev-custom-serializer.md index a35162b..ca0a332 100644 --- a/docs/dev-custom-serializer.md +++ b/docs/dev-custom-serializer.md @@ -29,6 +29,19 @@ When you decide that you want to define a custom serializer, you can do this in Please note that Gearpump shaded the original Kryo dependency. The package name ```com.esotericsoftware``` was relocated to ```io.gearpump.esotericsoftware```. So in the following customization, you should import corresponding shaded classes, the example code will show that part. +In general you should use the shaded version of a library whenever possible in order to avoid binary incompatibilities, eg don't use: + +```scala + import com.google.common.io.Files +``` + +but rather + +```scala + import io.gearpump.google.common.io.Files +``` + + ##### System Level Serializer If the serializer is widely used, you can define a global serializer which is available to all applications(or worker or master) in the system.
