-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28197/
-----------------------------------------------------------
Review request for Ambari, Jaimin Jetly, Mahadev Konar, Srimanth Gunturi, and
Sid Wagle.
Bugs: AMBARI-8379
https://issues.apache.org/jira/browse/AMBARI-8379
Repository: ambari
Description
-------
Summary of changes needed and implications
changes in falcon/conf/startup.properties:
1) For Falcon lineage to work below changes are needed on Ambari deployed
configs for Falcon
Implications : Lineage feature of Falcon is blocked without these changes
MetadataMappingService is required for lineage:
```
*.application.services=org.apache.falcon.security.AuthenticationInitializationService,\
org.apache.falcon.workflow.WorkflowJobEndNotificationService,\
org.apache.falcon.service.ProcessSubscriberService,\
org.apache.falcon.entity.store.ConfigurationStore,\
org.apache.falcon.rerun.service.RetryService,\
org.apache.falcon.rerun.service.LateRunService,\
org.apache.falcon.service.LogCleanupService,\
org.apache.falcon.metadata.MetadataMappingService
```
```
######### Graph Database Properties #########
# Graph implementation
*.falcon.graph.blueprints.graph=com.thinkaurelius.titan.core.TitanFactory
# Graph Storage
*.falcon.graph.storage.directory=/grid/0/data/lineage/graphdb
*.falcon.graph.storage.backend=berkeleyje
*.falcon.graph.serialize.path=/grid/0/data/lineage
*.falcon.graph.preserve.history=false
```
The properties that need a directory in the local file system should detect a
mount point that is available on the Falcon server, and the python scripts need
to crease these directories when the Falcon service is configured.
The folder permission are,
owner:falcon user
group:hadoop
permissions:775
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/package/scripts/falcon.py
37dd213
ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/package/scripts/params.py
2316326
ambari-server/src/main/resources/stacks/HDP/2.2/services/FALCON/configuration/falcon-startup.properties.xml
ec8b9c7
ambari-web/app/data/HDP2/site_properties.js e9b8234
ambari-web/app/models/service_config.js 255a679
Diff: https://reviews.apache.org/r/28197/diff/
Testing
-------
Ran ambari-server unit tests, all of which passed.
OK
----------------------------------------------------------------------
Total run:664
Total errors:0
Total failures:0
OK
Also used the Install Wizard to deploy an HDP 2.2 cluster with 2 nodes, one of
which was the Falcon server, and the other the client.
The falcon-startup.properties contains the correct values.
http://c6411.ambari.apache.org:8080/api/v1/clusters/dev/configurations?type=falcon-startup.properties&tag=version1
* *.application.services:
"org.apache.falcon.security.AuthenticationInitializationService,\
org.apache.falcon.workflow.WorkflowJobEndNotificationService, \
org.apache.falcon.service.ProcessSubscriberService,\
org.apache.falcon.entity.store.ConfigurationStore,\
org.apache.falcon.rerun.service.RetryService,\
org.apache.falcon.rerun.service.LateRunService,\
org.apache.falcon.service.LogCleanupService,\
org.apache.falcon.metadata.MetadataMappingService",
* *.falcon.graph.blueprints.graph: "com.thinkaurelius.titan.core.TitanFactory",
* *.falcon.graph.preserve.history: "false",
* *.falcon.graph.serialize.path: "/hadoop/falcon/data/lineage",
* *.falcon.graph.storage.backend: "berkeleyje",
* *.falcon.graph.storage.directory: "/hadoop/falcon/data/lineage/graphdb",
Further, the permissions are correct,
ls -la /hadoop/falcon/data/
drwxrwxr-x 3 falcon hadoop 4096 Nov 18 23:54 lineage
ls -la /hadoop/falcon/data/lineage/
drwxrwxr-x 2 falcon hadoop 4096 Nov 19 00:03 graphdb
Thanks,
Alejandro Fernandez
