Repository: incubator-unomi Updated Branches: refs/heads/master 7396d1871 -> 061ee51e0
Extensions : Router : Update README Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/061ee51e Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/061ee51e Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/061ee51e Branch: refs/heads/master Commit: 061ee51e001be5244f105e580de6a64781e29f1d Parents: 7396d18 Author: Abdelkader Midani <amid...@apache.org> Authored: Tue Jul 4 14:15:31 2017 +0200 Committer: Abdelkader Midani <amid...@apache.org> Committed: Tue Jul 4 14:15:54 2017 +0200 ---------------------------------------------------------------------- extensions/router/README.md | 82 ++++++++++++++++---- .../resources/OSGI-INF/blueprint/blueprint.xml | 4 +- .../main/resources/org.apache.unomi.router.cfg | 2 +- 3 files changed, 70 insertions(+), 18 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/061ee51e/extensions/router/README.md ---------------------------------------------------------------------- diff --git a/extensions/router/README.md b/extensions/router/README.md index 0d382d4..a927075 100644 --- a/extensions/router/README.md +++ b/extensions/router/README.md @@ -22,11 +22,20 @@ Unomi Router Unomi Router Extension a Karaf Feature that provide an Enterprise Application Integration tool. It is optional so you must configure it and install it in Karaf, and can be used for Machine - Machine or Human - Machine integration with Unomi. Mainly Unomi Router Extension aim to make it easy to import third party applications/platforms profiles into Unomi. -This extension is implemented using Apache Camel routes and is using Apache Kafka to buffer import process and make it failsafe. +This extension is implemented using Apache Camel routes and can use Apache Kafka to buffer import process and make it failsafe. ## Getting started 1. Configure your Unomi Router: - In the `etc/org.apache.unomi.router.cfg` file, you might want to update the following settings: + In the `etc/org.apache.unomi.router.cfg` file, first of all you need to decide if you want to use Apache Kafka to support huge amount of imported data + or just use the default configuration (Without broker) you might want to update the following settings: + + Configuration type + >`#Configuration Type values {'nobroker', 'kafka'}` + + >`router.config.type=nobroker` + + Change to 'kafka' and uncomment settings below to switch your configuration. + Kafka settings >`#Kafka settings` @@ -34,26 +43,23 @@ This extension is implemented using Apache Camel routes and is using Apache Kafk >`kafka.port=9092` - >`kafka.import.topic=camel-deposit` + >`kafka.import.topic=import-deposit` >`kafka.import.groupId=unomi-import-group` - Kafka host and port with the topic name and the groupId ti which the topic is assigned + >`kafka.export.topic=export-deposit` + + >`kafka.export.groupId=unomi-export-group` + + Kafka host and port with the topic name and the groupId to which the topic is assigned >`#Import One Shot upload directory` - >`import.oneshot.uploadDir=/tmp/unomi_oneshot_import_configs/` + >`import.oneshot.uploadDir=${karaf.data}/tmp/unomi_oneshot_import_configs/` Path to the folder where unomi should stock file imported for a oneshot processing - -2. Deploy into Apache Unomi using the following commands from the Apache Karaf shell: - ```sh - $ feature:repo-add mvn:org.apache.unomi/unomi-router-karaf-feature/${version}/xml/features - $ feature:install unomi-router-karaf-feature - ``` - -3. Send your import configuration: +2. Send your import configuration: An import configuration is nothing else than a simple JSON to describe how you want to import your data (Profiles). To create/update an import configuration @@ -64,7 +70,7 @@ This extension is implemented using Apache Camel routes and is using Apache Kafk "itemId": "f57f1f86-97bf-4ba0-b4e4-7d5e77e7c0bd", "itemType": "importConfig", "scope": "integration", - "name": "Test Recurrent", + "name": "Recurrent import", "description": "Just test recurrent import", "configType": "recurrent", "properties": { @@ -82,7 +88,7 @@ This extension is implemented using Apache Camel routes and is using Apache Kafk } ``` - Omit the `itemId` when creating new entry, `configType` can be '**recurrent**' for file/ftp/network path polling or '**oneshot**' for one time import. + Omit the `itemId` when creating new entry, `configType` can be '**recurrent**' for file/ftp/network path polling or '**oneshot**' for one time import (in case of oneshot configuration, omit the properties.source attribute). The `properties.source` attribute is an Apache Camel endpoint uri (See http://camel.apache.org/uris.html for more details). Unomi Router is designed to use **File** and **FTP** Camel components. @@ -106,8 +112,54 @@ This extension is implemented using Apache Camel routes and is using Apache Kafk First multipart with the name '**importConfigId**' is the importConfiguration to use to import the file, second one with the name '**file**' is the file to import. + 2. Send your export configuration: + + An export configuration is nothing else than a simple JSON to describe how you want to export your data (Profiles). + To create/update an export configuration + + `POST /cxs/exportConfiguration` + ```json + { + "itemId": "0e59d271-f5a4-497f-8646-0c8c66602278", + "itemType": "exportConfig", + "scope": "integration", + "name": "Test Recurrent", + "description": "Just test recurrent export", + "configType": "recurrent", + "properties": { + "destination": "{file/ftp}://{path}?fileName=profiles-export-${date:now:yyyyMMddHHmm}.csv", + "period": "1m", + "mapping": { + "0": "firstName", + "1": "lastName", + ... + } + }, + "active": true + } + ``` + Omit the `itemId` when creating new entry, `configType` can be '**recurrent**' for file/ftp/network path polling or '**oneshot**' for one time export (in case of oneshot configuration, omit the properties.destination + and properties.period attributes). + + The `properties.destination` attribute is an Apache Camel endpoint uri (See http://camel.apache.org/uris.html for more details). Unomi Router is designed to use **File** and **FTP**/**FTPS**/**SFTP** Camel components. + + File and ftp URI format: + + `file://directoryName[?options]` + See http://camel.apache.org/file2.html for more details. + + `ftp://[username@]hostname[:port]/directoryname[?options]` + `sftp://[username@]hostname[:port]/directoryname[?options]` + `ftps://[username@]hostname[:port]/directoryname[?options]` + + See http://camel.apache.org/ftp.html for more details. + + Concerning oneshot export configuration using the previously described service will create the export configuration, to return the generated file to download you need to call: + + `POST /cxs/importConfiguration/oneshot` + `Content-Type : application/json` \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/061ee51e/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml ---------------------------------------------------------------------- diff --git a/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml b/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml index 7ea578b..29df601 100644 --- a/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml +++ b/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml @@ -25,7 +25,7 @@ <cm:property-placeholder persistent-id="org.apache.unomi.router" update-strategy="reload"> <cm:default-properties> - <cm:property name="config.type" value="nobroker"/> + <cm:property name="router.config.type" value="nobroker"/> <cm:property name="config.allowedEndpoints" value="file,ftp"/> <cm:property name="kafka.host" value="localhost"/> <cm:property name="kafka.port" value="9092"/> @@ -90,7 +90,7 @@ <bean id="camelContext" class="org.apache.unomi.router.core.context.RouterCamelContext" init-method="initCamelContext" destroy-method="preDestroy"> - <property name="configType" value="${config.type}"/> + <property name="configType" value="${router.config.type}"/> <property name="allowedEndpoints" value="${config.allowedEndpoints}"/> <property name="uploadDir" value="${import.oneshot.uploadDir}"/> <property name="execHistorySize" value="${executionsHistory.size}"/> http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/061ee51e/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg ---------------------------------------------------------------------- diff --git a/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg b/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg index 22d7496..34c80ad 100644 --- a/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg +++ b/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg @@ -16,7 +16,7 @@ # #Configuration Type values {'nobroker', 'kafka'} -import.config.type=nobroker +router.config.type=nobroker #Uncomment and update Kafka settings to use Kafka as a broker