Till Westmann has uploaded a new change for review. https://asterix-gerrit.ics.uci.edu/1902
Change subject: PLEASE EDIT to provide a meaningful commit message! ...................................................................... PLEASE EDIT to provide a meaningful commit message! The following commits from your working branch will be included: commit d50601ad3b118c8c6bbbb300adc2bae826cd58d4 Author: Abdullah Alamoudi <[email protected]> Date: Tue Jul 25 14:42:04 2017 -0700 [ASTERIXDB-1992][ING] Suspend/Resume active entities - user model changes: wait for completion only returns when the entity becomes inactive. - storage format changes: no - interface changes: 01) Introduce: IMetadataLockManager for entity locking. IActiveNotificationHandler for handling active events. IRetryPolicy for recovery of failed active jobs. IActiveEntityController for controlling active entities. 02) IJobLifecycleListener.notifyJobFinish now passes the JobStatus and Exceptions. 03) IActiveEntityEventsListener.isActive() returns true, if entity is active, false otherwise. 04) IActiveEntityEventsListener.unregister() removes the listener upon entity deletion. 05) IActiveEntityEventsListener.getJobFailure() returns the job failure if the entity is in a failed state. 06) IStatementExecutor.getComponentProvider() returns the storage component provider. 07) IStatementExecutor.getApplicationContext() returns the application context. 08) IMetadataManager.upsertEntity to perfrom metadata entities' upsert operations. 09) IMetadataNode.upsertEntity to perfrom metadata entities' upsert operations 10) ICcApplicationContext.getMetadataLockManager() returns the lock manger. 11) ICcApplicationContext.getClusterStateManager() returns the cluster state manager. details: - Starting and stopping of active entities now go through their listeners rather than having the listener indicating active entity and not having it indicating inactive entity. This facilitates suspend/resume operations of long running active jobs for the sake of DDL operations or topology changes. - Unit tests for the vast majority of code paths and different possible scenarios have been added. commit d8192749dc18d974a49431d049c886bf09b3006a Author: Yingyi Bu <[email protected]> Date: Mon Jul 24 23:46:39 2017 -0700 [DOC] Reorganize data model documentations. - user model changes: no - storage format changes: no - interface changes: no Details: - Re-organize data model documentations; - Align results in the function doc with JSON; - Fix a few minor issues in the function doc. Reviewed-on: https://asterix-gerrit.ics.uci.edu/1898 Sonar-Qube: Jenkins <[email protected]> Integration-Tests: Jenkins <[email protected]> Reviewed-by: Dmitry Lychagin <[email protected]> Tested-by: Jenkins <[email protected]> BAD: Jenkins <[email protected]> Change-Id: I08ada040b4de088916ab82ffdfd837fb2d5ab857 --- M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveEvent.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveJobNotificationHandler.java D asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveLifecycleListener.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActiveRuntimeManager.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/ActivityState.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/CountRetryPolicy.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/CountRetryPolicyFactory.java R asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventSubscriber.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveEntityEventsListener.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IActiveNotificationHandler.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IRetryPolicy.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/IRetryPolicyFactory.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/InfiniteRetryPolicy.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/InfiniteRetryPolicyFactory.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/NoRetryPolicyFactory.java A asterixdb/asterix-active/src/main/java/org/apache/asterix/active/SingleThreadEventProcessor.java M asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActivePartitionMessage.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/InvertedIndexPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/AbstractLangTranslator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ActiveStatsApiServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/RebalanceApiServlet.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/ActiveEntityEventsListener.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/ActiveNotificationHandler.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/FeedEventsListener.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/DefaultStatementExecutorFactory.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/GlobalRecoveryManager.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/RebalanceUtil.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorApiServletTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/Action.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/ActiveEventsListenerTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/ActiveStatsTest.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/Actor.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/DummyFeedEventsListener.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/TestClusterControllerActor.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/TestEventsListener.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/TestNodeControllerActor.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/TestUserActor.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IClusterEventsSubscriber.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IMetadataLockManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/cluster/IGlobalRecoveryManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/ICcApplicationContext.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/AsterixException.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/CompilationException.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/MetadataException.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/RuntimeDataException.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/metadata/IDataset.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/metadata/IMetadataLock.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/metadata/LockList.java M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties M asterixdb/asterix-doc/pom.xml M asterixdb/asterix-doc/src/main/markdown/builtins/11_type.md M asterixdb/asterix-doc/src/main/markdown/builtins/12_misc.md M asterixdb/asterix-doc/src/main/markdown/builtins/13_conditional.md M asterixdb/asterix-doc/src/main/markdown/builtins/1_numeric_common.md M asterixdb/asterix-doc/src/main/markdown/builtins/9_aggregate_sql.md A asterixdb/asterix-doc/src/main/markdown/datamodel/datamodel_composite.md A asterixdb/asterix-doc/src/main/markdown/datamodel/datamodel_header.md A asterixdb/asterix-doc/src/main/markdown/datamodel/datamodel_incomplete.md A asterixdb/asterix-doc/src/main/markdown/datamodel/datamodel_primitive_common.md A asterixdb/asterix-doc/src/main/markdown/datamodel/datamodel_primitive_delta.md D asterixdb/asterix-doc/src/site/markdown/datamodel.md M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/api/IDataSourceAdapter.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedTupleForwarder.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataset/adapter/FeedAdapter.java D asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/management/ActiveEntityEventsListener.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/runtime/AdapterExecutor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/AbstractSubscriber.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/NoOpSubscriber.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/StatsSubscriber.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/feed/watch/WaitForStateSubscriber.java M asterixdb/asterix-lang-aql/src/main/java/org/apache/asterix/lang/aql/statement/SubscribeFeedStatement.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/statement/ConnectFeedStatement.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataCache.java D asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataException.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataManager.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IActiveEntityController.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataEntityTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataManager.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IMetadataNode.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/api/IValueExtractor.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataManagerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/BuiltinTypeMap.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Dataset.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/FeedConnection.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entities/Index.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/CompactionPolicyTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasourceAdapterTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatatypeTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DataverseTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/ExternalFileTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/FeedConnectionTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/FeedPolicyTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/FeedTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/FunctionTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/IndexTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/LibraryTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/NodeGroupTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/NodeTupleTranslator.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/BuiltinFeedPolicies.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/functions/ExternalFunctionCompilerUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/DatasetLock.java D asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/IMetadataLock.java D asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/LockList.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLock.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/lock/MetadataLockManager.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/DatasetUtil.java A asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataLockUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/MetadataUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SplitsAndConstraintsUtil.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/valueextractors/MetadataEntityValueExtractor.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/valueextractors/NestedDatatypeNameValueExtractor.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/valueextractors/TupleCopyValueExtractor.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslatorTest.java M asterixdb/asterix-metadata/src/test/java/org/apache/asterix/metadata/entitytupletranslators/IndexTupleTranslatorTest.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/CcApplicationContext.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java M asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/datagen/AdmDataGen.java M asterixdb/asterix-tools/src/test/java/org/apache/asterix/tools/translator/ADGenDmlTranslator.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/HyracksDataException.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/job/IJobLifecycleListener.java M hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivityOperatorNodePushable.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/application/CCServiceContext.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/cluster/NodeManager.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/job/JobManager.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/RegisterNodeWork.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/test/java/org/apache/hyracks/control/cc/cluster/NodeManagerTest.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/utils/ExceptionUtils.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Task.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PipelinedPartition.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/NotifyTaskFailureWork.java M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/StartTasksWork.java M hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/utils/FrameDebugUtils.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/collectors/NonDeterministicChannelReader.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/AbstractMultiNCIntegrationTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/JobFailureTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/LocalityAwareConnectorTest.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/util/ExceptionOnCreatePushRuntimeOperatorDescriptor.java 155 files changed, 6,593 insertions(+), 2,258 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/02/1902/1 -- To view, visit https://asterix-gerrit.ics.uci.edu/1902 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I08ada040b4de088916ab82ffdfd837fb2d5ab857 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Till Westmann <[email protected]>
