Change in asterixdb[master]: Cleanup FileSplit and FileReference
abdullah alamoudi has submitted this change and it was merged. Change subject: Cleanup FileSplit and FileReference .. Cleanup FileSplit and FileReference This change gives FileSplit and FileReference specific meaning to avoid confusion of an absolute vs relative, local vs global, inside an IO device vs outside IO devices. In addition, it enables better abstraction of global partitions and delegate the responsibility of choosing which partition goes to which IO device to the IO Manager through the introduction of FileDeviceComputer In details: Previously, the LocalResource in Hyracks had partition (storage partition) and there is no such thing in Hyracks. This scope leak is bad. In addition The local resource had a name and a path. they were always the same and so the name was removed. The storage partition was instead moved to asterixdb implementation of the serialized object in the local resource. With all of these changes, the cluster controller (compiler) only needs to know about partitions and relative paths. It doesn't need to worry about heterogenous Node setups and different io device configurations. For File assignment to IO devices, a new interface (IFileDeviceComputer) was introduced which can be overriden by applications to have their own strategy for distributing files among IO devices. Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Reviewed-on: https://asterix-gerrit.ics.uci.edu/1352 Tested-by: JenkinsIntegration-Tests: Jenkins Reviewed-by: abdullah alamoudi --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalIndexingOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/DatasetOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryBTreeOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryInvertedIndexOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryRTreeOperationsHelper.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java M asterixdb/asterix-common/pom.xml M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixFileMapManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInvertedIndexInsertDeleteOperatorDescriptor.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IResourceFactory.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java R asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ResourceFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/indexing/ExternalFileIndexAccessor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/AbstractExternalDatasetIndexesOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesAbortOperatorDescriptor.java M
Change in asterixdb[master]: Cleanup FileSplit and FileReference
abdullah alamoudi has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 5: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 5: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1170/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 5: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1170/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 5: WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN: * asterixdb * hyracks-fullstack PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES! -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 5: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3356/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Hello Till Westmann, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1352 to look at the new patch set (#5). Change subject: Cleanup FileSplit and FileReference .. Cleanup FileSplit and FileReference This change gives FileSplit and FileReference specific meaning to avoid confusion of an absolute vs relative, local vs global, inside an IO device vs outside IO devices. In addition, it enables better abstraction of global partitions and delegate the responsibility of choosing which partition goes to which IO device to the IO Manager through the introduction of FileDeviceComputer In details: Previously, the LocalResource in Hyracks had partition (storage partition) and there is no such thing in Hyracks. This scope leak is bad. In addition The local resource had a name and a path. they were always the same and so the name was removed. The storage partition was instead moved to asterixdb implementation of the serialized object in the local resource. With all of these changes, the cluster controller (compiler) only needs to know about partitions and relative paths. It doesn't need to worry about heterogenous Node setups and different io device configurations. For File assignment to IO devices, a new interface (IFileDeviceComputer) was introduced which can be overriden by applications to have their own strategy for distributing files among IO devices. Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalIndexingOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/DatasetOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryBTreeOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryInvertedIndexOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryRTreeOperationsHelper.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java M asterixdb/asterix-common/pom.xml M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixFileMapManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInvertedIndexInsertDeleteOperatorDescriptor.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IResourceFactory.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java R asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ResourceFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/indexing/ExternalFileIndexAccessor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/AbstractExternalDatasetIndexesOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesAbortOperatorDescriptor.java M
Change in asterixdb[master]: Cleanup FileSplit and FileReference
abdullah alamoudi has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 4: Going over the comments and addressing them in the next patch... -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-Reviewer: abdullah alamoudi Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Till Westmann has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 4: Code-Review+2 (7 comments) Need to fix more things in the next few days. https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java: Line 184: partition.put("relative", relative); Seems that we're changing an external interface here. Who is using this interface? Do we change the code as well? https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java File asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java: Line 173: fileMapManager = new AsterixFileMapManager(ioManager); Why does the AsterixFileMapManager need the ioMananger? https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java File asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java: Line 159: element1.put("relative", true); Ok, this modifies the test that is supposed to validate that the interface does not change ... https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java File asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java: Line 55: if (next == device) { Use equals instead of == ? Seems more robust wrt change in the code base. If this is performance critical we might want to reconsider the linear search. https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java File asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java: Line 169: validateDatasetLifecycleManagerState(); Indentation seems wrong. https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java File asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java: Line 33: public abstract class Resource implements Serializable { That's a very generic name. What is it really about? Can we add a descriptive comment? https://asterix-gerrit.ics.uci.edu/#/c/1352/4/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java File asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java: Line 40: import org.apache.hyracks.api.io.FileSplit; Why do we need to move this? -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: Yes
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 4: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1153/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 4: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1153/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 4: WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN: * asterixdb * hyracks-fullstack PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES! -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 4: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3334/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 4 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1352 to look at the new patch set (#4). Change subject: Cleanup FileSplit and FileReference .. Cleanup FileSplit and FileReference This change gives FileSplit and FileReference specific meaning to avoid confusion of an absolute vs relative, local vs global, inside an IO device vs outside IO devices. In addition, it enables better abstraction of global partitions and delegate the responsibility of choosing which partition goes to which IO device to the IO Manager through the introduction of FileDeviceComputer In details: Previously, the LocalResource in Hyracks had partition (storage partition) and there is no such thing in Hyracks. This scope leak is bad. In addition The local resource had a name and a path. they were always the same and so the name was removed. The storage partition was instead moved to asterixdb implementation of the serialized object in the local resource. With all of these changes, the cluster controller (compiler) only needs to know about partitions and relative paths. It doesn't need to worry about heterogenous Node setups and different io device configurations. For File assignment to IO devices, a new interface (IFileDeviceComputer) was introduced which can be overriden by applications to have their own strategy for distributing files among IO devices. Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalIndexingOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/DatasetOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryBTreeOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryInvertedIndexOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryRTreeOperationsHelper.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixFileMapManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInvertedIndexInsertDeleteOperatorDescriptor.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IResourceFactory.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java R asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ResourceFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/indexing/ExternalFileIndexAccessor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/AbstractExternalDatasetIndexesOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesAbortOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesRecoverOperatorDescriptor.java M
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1150/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1150/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: Integration-Tests-1 Integration Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1148/ : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1148/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1147/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN: * asterixdb * hyracks-fullstack PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES! -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 3: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3331/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1352 to look at the new patch set (#3). Change subject: Cleanup FileSplit and FileReference .. Cleanup FileSplit and FileReference This change gives FileSplit and FileReference specific meaning to avoid confusion of an absolute vs relative, local vs global, inside an IO device vs outside IO devices. In addition, it enables better abstraction of global partitions and delegate the responsibility of choosing which partition goes to which IO device to the IO Manager through the introduction of FileDeviceComputer Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalIndexingOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/DatasetOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryBTreeOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryInvertedIndexOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryRTreeOperationsHelper.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java A asterixdb/asterix-app/storage/partition_0/Metadata/Dataset_idx_Dataset/.metadata A asterixdb/asterix-app/storage/partition_0/Metadata/Dataverse_idx_Dataverse/.metadata M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixFileMapManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInvertedIndexInsertDeleteOperatorDescriptor.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IResourceFactory.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java R asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ResourceFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/indexing/ExternalFileIndexAccessor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/AbstractExternalDatasetIndexesOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesAbortOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesRecoverOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalIndexBulkModifyOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java M
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 2: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1140/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 2: WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN: * asterixdb * hyracks-fullstack PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES! -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 2: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3320/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 2 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1352 to look at the new patch set (#2). Change subject: Cleanup FileSplit and FileReference .. Cleanup FileSplit and FileReference This change gives FileSplit and FileReference specific meaning to avoid confusion of an absolute vs relative, local vs global, inside an IO device vs outside IO devices. In addition, it enables better abstraction of global partitions and delegate the responsibility of choosing which partition goes to which IO device to the IO Manager through the introduction of FileDeviceComputer Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ConnectorAPIServlet.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalIndexingOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/FeedOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/AsterixNCAppRuntimeContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/DatasetOperations.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryBTreeOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryInvertedIndexOperationsHelper.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/file/SecondaryRTreeOperationsHelper.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorAPIServletTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/app/bootstrap/TestNodeController.java A asterixdb/asterix-app/storage/partition_0/Metadata/Dataset_idx_Dataset/.metadata A asterixdb/asterix-app/storage/partition_0/Metadata/Dataverse_idx_Dataverse/.metadata M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixFileMapManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/AsterixVirtualBufferCacheProvider.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInvertedIndexInsertDeleteOperatorDescriptor.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IResourceFactory.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/Resource.java R asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ResourceFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/GenericAdapterFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/indexing/ExternalFileIndexAccessor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/stream/factory/LocalFSInputStreamFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/AbstractExternalDatasetIndexesOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalBTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesAbortOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalDatasetIndexesRecoverOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalIndexBulkModifyOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorDescriptor.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalRTreeSearchOperatorNodePushable.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/FeedUtils.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java M
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 1: Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/1138/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 1: Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/3313/ -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No
Change in asterixdb[master]: Cleanup FileSplit and FileReference
Jenkins has posted comments on this change. Change subject: Cleanup FileSplit and FileReference .. Patch Set 1: WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN: * asterixdb * hyracks-fullstack PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES! -- To view, visit https://asterix-gerrit.ics.uci.edu/1352 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61 Gerrit-PatchSet: 1 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: abdullah alamoudiGerrit-Reviewer: Jenkins Gerrit-HasComments: No