> On Feb. 24, 2018, 3:06 a.m., Apoorv Naik wrote: > > common/src/main/java/org/apache/atlas/service/Services.java > > Line 52 (original), 61 (patched) > > <https://reviews.apache.org/r/65779/diff/1/?file=1963965#file1963965line62> > > > > Wouldn't it be easier to add another state called MIGRATION in > > ServiceState enum and the Service.start() would be a no-op when Atlas is in > > migration state. > > Ashutosh Mestry wrote: > ServiceState has MIGRATION as new enum. Don't fully understand this > comment.
Every implementation of Service interface should check the current ServiceState and only do stuff when it's an ACTIVE instance and not in migration mode. eg. KafkaNotification and NotificationHookConsumer start can be no-op during migration, HBaseAuditRepository might be needed to generate audits. - Apoorv ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65779/#review198242 ----------------------------------------------------------- On Feb. 24, 2018, 5:08 p.m., Ashutosh Mestry wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65779/ > ----------------------------------------------------------- > > (Updated Feb. 24, 2018, 5:08 p.m.) > > > Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Ruchi Solani. > > > Bugs: ATLAS-2460 > https://issues.apache.org/jira/browse/ATLAS-2460 > > > Repository: atlas > > > Description > ------- > > **Background** > This implementation deals with the 'import into' part of the data migration > process. > > It assumes: > - Export from older cluster is done. > - Generated file has been moved to newer cluster. > > **Implementation** > > During _Atlas_ server startup, the configuration parameter is checked, if > that parameter exists, all services except _DataMigrationService_ is started. > Migration is started. Atlas server is available in _MIGRATION_ mode. It > processes REST calls made only to the _AdminResource_. > > Here's are the udpates: > - New configuration parameter has been added: > _atlas.migration.mode.filename=<name of the file to be imported>_ > This configuration parameter is set by Ambari as part of its migration > orchestration. > - _DataMigrationService_: New service that performs async migration as soon > as Atlas server starts up. > - _ImportProgressTracker_: Added to be able to query progress of import. > - _ImportService_: Modified to send updates to _ImportProgressTracker_. > - _AdminResource.getStatus()_ Now supplies additional status about migration. > - _ServiceState_ Modified to carry additional status _MIGRATION_. This status > is set by looking at the configuration parameter above. > - _AtlasEntityStoreV1_ modified to block sending out notifications during > migration. > - _Services_ modified for special handling of _DataMigrationService_. > > > **CURL** > Check status using: > ``` > curl -X GET -u admin:admin -H "Content-Type: application/json" -H > "Cache-Control: no-cache" http://localhost:21000/api/atlas/admin/status > ``` > > > Diffs > ----- > > common/src/main/java/org/apache/atlas/AtlasConstants.java f5de1df3 > common/src/main/java/org/apache/atlas/service/Services.java 1267dc92 > intg/src/main/java/org/apache/atlas/model/impexp/AtlasExportResult.java > 1ea961d8 > intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportResult.java > b97cbb3f > > repository/src/main/java/org/apache/atlas/repository/impexp/DataMigrationService.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/impexp/ImportProgressTracker.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java > c976c593 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java > bf417dd0 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java > af310954 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java > 0ba06b94 > > repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java > 07524d08 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java > f4b70531 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java > 8de978c8 > webapp/src/main/java/org/apache/atlas/Atlas.java 7cf6e3ea > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 86485fba > webapp/src/main/java/org/apache/atlas/web/service/ServiceState.java > 3fe8d18c > webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java > fd96fc3d > > > Diff: https://reviews.apache.org/r/65779/diff/2/ > > > Testing > ------- > > **Unit tests** > None added. > > **Functional tests** > Steps to test: > - Place the exported file say _export-1.zip_ at _/root_ > - Add to _Atlas_ Ambari's customer property: > _atlas.migration.mode.filename=/root/export-1.zip_ > - Ambari will prompt for a restart. Restart Atlas. > - On the server view the progress in the logs using: _tail -f > /var/log/atlas/application.log_ > - Use the CURL call mentioned above and view the status and the progress of > the import. > > > Thanks, > > Ashutosh Mestry > >
