Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
>From Ali Alsuliman : Attention is currently required from: Wail Alkowaileet, Ali Alsuliman, Michael Blow. Ali Alsuliman has removed a vote from this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Removed Verified-1 by Jenkins -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Ali Alsuliman Gerrit-Attention: Michael Blow Gerrit-MessageType: deleteVote
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
>From Ali Alsuliman : Attention is currently required from: Wail Alkowaileet, Michael Blow. Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Patch Set 1: Verified+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Michael Blow Gerrit-Comment-Date: Thu, 13 Jun 2024 01:07:07 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
Attention is currently required from: Wail Alkowaileet, Ali Alsuliman, Michael Blow. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Patch Set 1: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/KjB8geLFohquP7Xj9 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Ali Alsuliman Gerrit-Attention: Michael Blow Gerrit-Comment-Date: Thu, 13 Jun 2024 00:55:41 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][RT] Hint-less interval joins - user model changes: yes - s...
>From Jenkins : Attention is currently required from: Edward Coberly. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18372 ) Change subject: [NO ISSUE][RT] Hint-less interval joins - user model changes: yes - storage format changes: no - interface changes: no Details: Support interval joins without using a hint .. Patch Set 1: Integration-Tests-1 Integration Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/771/ : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18372 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: If553cdc5632a6caa6db5dfd2829a653aa7b2bcb9 Gerrit-Change-Number: 18372 Gerrit-PatchSet: 1 Gerrit-Owner: Edward Coberly Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Edward Coberly Gerrit-Comment-Date: Thu, 13 Jun 2024 00:51:16 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE] unregister shutdown hook on CC stop
>From Jenkins : Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18371 ) Change subject: [NO ISSUE] unregister shutdown hook on CC stop .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/770/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18371 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I48846ab8560ce0da64e74647de1849554d44dc6e Gerrit-Change-Number: 18371 Gerrit-PatchSet: 1 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Comment-Date: Thu, 13 Jun 2024 00:39:57 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][RT] Hint-less interval joins - user model changes: yes - s...
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18372 ) Change subject: [NO ISSUE][RT] Hint-less interval joins - user model changes: yes - storage format changes: no - interface changes: no Details: Support interval joins without using a hint .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/RY6b99hQXiVhvXfW6 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18372 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: If553cdc5632a6caa6db5dfd2829a653aa7b2bcb9 Gerrit-Change-Number: 18372 Gerrit-PatchSet: 1 Gerrit-Owner: Edward Coberly Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Thu, 13 Jun 2024 00:26:21 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE] unregister shutdown hook on CC stop
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18371 ) Change subject: [NO ISSUE] unregister shutdown hook on CC stop .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/HeT7Kknb5joKg5w88 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18371 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I48846ab8560ce0da64e74647de1849554d44dc6e Gerrit-Change-Number: 18371 Gerrit-PatchSet: 1 Gerrit-Owner: Peeyush Gupta Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 23:33:23 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
>From Murtadha Hubail : Attention is currently required from: Wail Alkowaileet, Ali Alsuliman, Michael Blow. Murtadha Hubail has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Ali Alsuliman Gerrit-Attention: Michael Blow Gerrit-Comment-Date: Wed, 12 Jun 2024 23:30:07 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
>From Ali Alsuliman : Attention is currently required from: Murtadha Hubail, Wail Alkowaileet, Michael Blow. Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Patch Set 1: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Murtadha Hubail Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Michael Blow Gerrit-Comment-Date: Wed, 12 Jun 2024 23:28:17 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE] unregister shutdown hook on CC stop
>From Peeyush Gupta :
Peeyush Gupta has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18371 )
Change subject: [NO ISSUE] unregister shutdown hook on CC stop
..
[NO ISSUE] unregister shutdown hook on CC stop
Change-Id: I48846ab8560ce0da64e74647de1849554d44dc6e
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
1 file changed, 10 insertions(+), 0 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/71/18371/1
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
index 8cd2487..5921e6a 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
@@ -296,6 +296,7 @@
@Override
public void stop() throws Exception {
LOGGER.info("Stopping Asterix cluster controller");
+super.stop();
appCtx.getClusterStateManager().setState(SHUTTING_DOWN);
((ActiveNotificationHandler)
appCtx.getActiveNotificationHandler()).stop();
AsterixStateProxy.unregisterRemoteObject();
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18371
To unsubscribe, or for help writing mail filters, visit
https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I48846ab8560ce0da64e74647de1849554d44dc6e
Gerrit-Change-Number: 18371
Gerrit-PatchSet: 1
Gerrit-Owner: Peeyush Gupta
Gerrit-MessageType: newchange
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
Attention is currently required from: Wail Alkowaileet. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 ) Change subject: [NO ISSUE] Add required --add-opens .. Patch Set 3: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/N6JzfaNw4VM1F33Q8 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 Gerrit-Change-Number: 18369 Gerrit-PatchSet: 3 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Wed, 12 Jun 2024 23:18:52 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
>From Jenkins : Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/769/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Comment-Date: Wed, 12 Jun 2024 23:06:13 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3428][OTH] Improve storage logging
>From Jenkins : Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18368 ) Change subject: [ASTERIXDB-3428][OTH] Improve storage logging .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/768/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18368 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I8b4e92585dee55a7f6a24342fb8d1a6390449c1c Gerrit-Change-Number: 18368 Gerrit-PatchSet: 2 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 22:41:54 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/dsbARpgoKTnJtr1u8 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 Gerrit-Change-Number: 18370 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 22:23:52 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size
>From Ali Alsuliman : Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18370 ) Change subject: [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size .. [ASTERIXDB-3423][CONF] Add compiler property for COPY TO buffer size - user model changes: no - storage format changes: no - interface changes: no Details: - Add property 'COMPILER_COPY_TO_WRITE_BUFFER_SIZE' to configure COPY TO write buffer size. Default is 8MB with a minimum of 5MB. - Enforce a minimum of 5MB for 'CLOUD_WRITE_BUFFER_SIZE'. - Remove property 'COMPILER_ENABLE_DB_RESOLUTION'. Change-Id: I50bebc4b8b683889855cb5dd048ab27d7193ff76 --- M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/writer/AbstractCloudExternalFileWriterFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CloudProperties.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/provider/ExternalWriterProvider.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/CompilerProperties.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3ClientConfig.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/writer/S3ExternalFileWriterFactory.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/writer/ExternalFileWriterConfiguration.java 12 files changed, 54 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/18370/1 diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java index 8cd2487..f19dea6 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java @@ -73,7 +73,6 @@ import org.apache.asterix.common.cluster.IGlobalTxManager; import org.apache.asterix.common.config.AsterixExtension; import org.apache.asterix.common.config.CloudProperties; -import org.apache.asterix.common.config.CompilerProperties; import org.apache.asterix.common.config.ExtensionProperties; import org.apache.asterix.common.config.ExternalProperties; import org.apache.asterix.common.config.GlobalConfig; @@ -171,10 +170,8 @@ new ReplicationProperties(PropertiesAccessor.getInstance(ccServiceCtx.getAppConfig())); INcLifecycleCoordinator lifecycleCoordinator = createNcLifeCycleCoordinator(repProp.isReplicationEnabled()); componentProvider = new StorageComponentProvider(); -boolean isDbResolutionEnabled = - ccServiceCtx.getAppConfig().getBoolean(CompilerProperties.Option.COMPILER_ENABLE_DB_RESOLUTION); boolean cloudDeployment = ccServiceCtx.getAppConfig().getBoolean(CLOUD_DEPLOYMENT); -boolean useDatabaseResolution = cloudDeployment && isDbResolutionEnabled; +boolean useDatabaseResolution = cloudDeployment; INamespaceResolver namespaceResolver = createNamespaceResolver(useDatabaseResolution); INamespacePathResolver namespacePathResolver = new NamespacePathResolver(useDatabaseResolution); ccExtensionManager = new CCExtensionManager(new ArrayList<>(getExtensions()), namespaceResolver, ccServiceCtx); diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java index d02dc4f..0ff8796 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java @@ -61,7 +61,6 @@ import org.apache.asterix.common.api.IPropertiesFactory; import org.apache.asterix.common.api.IReceptionistFactory; import org.apache.asterix.common.config.AsterixExtension; -import org.apache.asterix.common.config.CompilerProperties; import org.apache.asterix.common.config.ExtensionProperties; import org.apache.asterix.common.config.ExternalProperties; import org.apache.asterix.common.config.GlobalConfig; @@ -160,10 +159,8 @@ } MetadataBuiltinFunctions.init();
Change in asterixdb[master]: [ASTERIXDB-3428][OTH] Improve storage logging
>From Wail Alkowaileet : Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18368 to look at the new patch set (#2). Change subject: [ASTERIXDB-3428][OTH] Improve storage logging .. [ASTERIXDB-3428][OTH] Improve storage logging - user model changes: no - storage format changes: no - interface changes: no Details: - Storage log events will be in storage_debug.log - Reduce schema logging on query - Several improvements when logging on merge/flush Change-Id: I8b4e92585dee55a7f6a24342fb8d1a6390449c1c --- M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/LazyCloudIOManager.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/IoOperationExecutor.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/AtomicLSMIOOperationCallback.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractLSMMemoryComponent.java M hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/impls/AbstractTreeIndex.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/util/CloudFileUtil.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/NoOpIoOperation.java M asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationOperation.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/LSMHarness.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/ChainedLSMDiskComponentBulkLoader.java M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common/test/IoSchedulerTest.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/filesystem/PhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractAsynchronousScheduler.java M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/util/SchemaStringBuilderVisitor.java M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreePageWriteCallbackTest.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractIoOperation.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/TracedIOOperation.java M hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/storage/am/common/TreeIndexTestUtils.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractLSMIndex.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractLSMDiskComponent.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/LoadOperation.java M asterixdb/asterix-app/src/test/resources/log4j2-asterixdb-test.xml M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/operation/lsm/flush/FlushColumnMetadata.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/MergeOperation.java M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/operation/query/QueryColumnWithMetaMetadata.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/SynchronousScheduler.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/impls/LSMBTreeFileManager.java M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/operation/query/QueryColumnMetadata.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/lsm/LSMColumnBTreeWithBloomFilterDiskComponent.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/ParallelCacher.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIOOperation.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/HaltCallback.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/LSMIOOperationCallback.java M asterixdb/asterix-app/src/main/resources/cc.conf M
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
>From Jenkins : Attention is currently required from: Wail Alkowaileet. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 ) Change subject: [NO ISSUE] Add required --add-opens .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/767/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 Gerrit-Change-Number: 18369 Gerrit-PatchSet: 3 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Wed, 12 Jun 2024 21:21:40 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
Attention is currently required from: Wail Alkowaileet. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 ) Change subject: [NO ISSUE] Add required --add-opens .. Patch Set 3: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/yKZ8W4iNZxpWDK6H7 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 Gerrit-Change-Number: 18369 Gerrit-PatchSet: 3 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Wed, 12 Jun 2024 20:45:00 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Murtadha Hubail :
Murtadha Hubail has submitted this change. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 )
Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup
..
[NO ISSUE][TEST] Catch all failures on cc/nc startup
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Catch all nc/cc startup failures in AsterixHyracksIntegrationUtil.
Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363
Integration-Tests: Jenkins
Tested-by: Murtadha Hubail
Reviewed-by: Murtadha Hubail
Reviewed-by: Ali Alsuliman
Reviewed-by: Peeyush Gupta
---
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
1 file changed, 47 insertions(+), 13 deletions(-)
Approvals:
Murtadha Hubail: Looks good to me, but someone else must approve; Verified
Ali Alsuliman: Looks good to me, approved
Peeyush Gupta: Looks good to me, approved
Jenkins: Verified
Objections:
Anon. E. Moose #1000171: Violations found
diff --git
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
index 506e82a..e312b01 100644
---
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
+++
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
@@ -31,6 +31,7 @@
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.BiPredicate;
import java.util.stream.Stream;
@@ -61,7 +62,6 @@
import org.apache.hyracks.ipc.impl.HyracksConnection;
import org.apache.hyracks.storage.am.lsm.btree.impl.TestLsmBtreeLocalResource;
import org.apache.hyracks.test.support.TestUtils;
-import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.jetbrains.annotations.NotNull;
@@ -110,8 +110,8 @@
try {
integrationUtil.run(Boolean.getBoolean("cleanup.start"),
Boolean.getBoolean("cleanup.shutdown"),
getConfPath());
-} catch (Exception e) {
-LOGGER.fatal("Unexpected exception", e);
+} catch (Throwable t) {
+LOGGER.fatal("Unexpected exception", t);
System.exit(1);
}
}
@@ -155,19 +155,27 @@
}
opts.forEach(opt -> configManager.set(opt.getLeft(), opt.getRight()));
-cc.start();
+try {
+cc.start();
+} catch (Throwable t) {
+LOGGER.error("failed to start cc", t);
+throw t;
+}
// Starts ncs.
nodeNames = ccConfig.getConfigManager().getNodeNames();
List startupThreads = new ArrayList<>();
+AtomicBoolean ncFailedToStart = new AtomicBoolean(false);
for (NodeControllerService nc : nodeControllers) {
Thread ncStartThread = new Thread("IntegrationUtil-" + nc.getId())
{
@Override
public void run() {
try {
nc.start();
-} catch (Exception e) {
-LOGGER.log(Level.ERROR, e.getMessage(), e);
+LOGGER.info("started node {}", nc.getId());
+} catch (Throwable t) {
+LOGGER.error("failed to start node {}", nc.getId(), t);
+ncFailedToStart.set(true);
}
}
};
@@ -178,11 +186,14 @@
for (Thread thread : startupThreads) {
thread.join();
}
+if (ncFailedToStart.get()) {
+throw new Exception("some node failed to start");
+}
// Wait until cluster becomes active
((ICcApplicationContext)
cc.getApplicationContext()).getClusterStateManager().waitForState(ClusterState.ACTIVE);
hcc = new HyracksConnection(cc.getConfig().getClientListenAddress(),
cc.getConfig().getClientListenPort(),
cc.getNetworkSecurityManager().getSocketChannelFactory());
-this.ncs = nodeControllers.toArray(new
NodeControllerService[nodeControllers.size()]);
+this.ncs = nodeControllers.toArray(new NodeControllerService[0]);
}
@NotNull
@@ -297,8 +308,8 @@
public void run() {
try {
nodeControllerService.stop();
-} catch (Exception e) {
-e.printStackTrace();
+} catch (Throwable t) {
+LOGGER.error("failed to stop node {}",
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
>From Wail Alkowaileet : Attention is currently required from: Wail Alkowaileet. Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 to look at the new patch set (#3). Change subject: [NO ISSUE] Add required --add-opens .. [NO ISSUE] Add required --add-opens Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 --- M asterixdb/asterix-app/pom.xml M asterixdb/pom.xml M asterixdb/asterix-app/src/test/resources/cc-cloud-storage.conf 3 files changed, 25 insertions(+), 4 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/69/18369/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 Gerrit-Change-Number: 18369 Gerrit-PatchSet: 3 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Wail Alkowaileet Gerrit-MessageType: newpatchset
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Murtadha Hubail : Murtadha Hubail has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Verified+1 Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 20:30:52 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Peeyush Gupta : Attention is currently required from: Murtadha Hubail. Peeyush Gupta has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Peeyush Gupta Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 20:31:00 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Ali Alsuliman : Attention is currently required from: Murtadha Hubail. Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 20:30:59 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Murtadha Hubail : Attention is currently required from: Murtadha Hubail. Murtadha Hubail has removed a vote from this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Removed Verified-1 by Jenkins -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Murtadha Hubail Gerrit-MessageType: deleteVote
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
>From Wail Alkowaileet : Wail Alkowaileet has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 ) Change subject: [NO ISSUE] Add required --add-opens .. [NO ISSUE] Add required --add-opens Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 --- M asterixdb/asterix-app/pom.xml M asterixdb/pom.xml 2 files changed, 23 insertions(+), 2 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/69/18369/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 Gerrit-Change-Number: 18369 Gerrit-PatchSet: 2 Gerrit-Owner: Wail Alkowaileet Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-MessageType: newpatchset
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 ) Change subject: [NO ISSUE] Add required --add-opens .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/DePMrdNF63HPPBvk9 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743 Gerrit-Change-Number: 18369 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 20:21:08 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE] Add required --add-opens
>From Wail Alkowaileet :
Wail Alkowaileet has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369 )
Change subject: [NO ISSUE] Add required --add-opens
..
[NO ISSUE] Add required --add-opens
Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743
---
M asterixdb/asterix-app/pom.xml
1 file changed, 16 insertions(+), 1 deletion(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/69/18369/1
diff --git a/asterixdb/asterix-app/pom.xml b/asterixdb/asterix-app/pom.xml
index 914a625..61391c1 100644
--- a/asterixdb/asterix-app/pom.xml
+++ b/asterixdb/asterix-app/pom.xml
@@ -532,7 +532,13 @@
-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=${debug.suspend.flag}
-Xloggc:"${project.build.directory}/surefire-reports/SqlppExecutionTest-%p-gc.log"
-XX:+PrintGC -XX:+PrintGCDateStamps -XX:GCLogFileSize=10M
-
--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
--add-opens=java.management/sun.management=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
+
--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
+--add-opens=java.management/sun.management=ALL-UNNAMED
+--add-opens=java.base/java.lang=ALL-UNNAMED
+--add-opens=java.base/java.nio=ALL-UNNAMED
+--add-opens=java.base/java.util=ALL-UNNAMED
+--add-opens=java.base/java.io=ALL-UNNAMED
+--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18369
To unsubscribe, or for help writing mail filters, visit
https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: Idd59b3d9b3a5d3a94a367a8b50fbe55fc9f36743
Gerrit-Change-Number: 18369
Gerrit-PatchSet: 1
Gerrit-Owner: Wail Alkowaileet
Gerrit-MessageType: newchange
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
Attention is currently required from: Ritik Raj. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. Patch Set 3: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/QUvdq2ss2cNFERxf8 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 3 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-Comment-Date: Wed, 12 Jun 2024 20:00:39 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3428][OTH] Improve storage logging
>From Jenkins : Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18368 ) Change subject: [ASTERIXDB-3428][OTH] Improve storage logging .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/764/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18368 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I8b4e92585dee55a7f6a24342fb8d1a6390449c1c Gerrit-Change-Number: 18368 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 19:02:07 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
>From Jenkins : Attention is currently required from: Ritik Raj. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/763/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 3 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-Comment-Date: Wed, 12 Jun 2024 18:23:09 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
Attention is currently required from: Ritik Raj. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. Patch Set 3: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/ER9MMMdLodkKiRAf7 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 3 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-Comment-Date: Wed, 12 Jun 2024 17:20:36 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
>From Ritik Raj : Attention is currently required from: Ritik Raj. Hello Jenkins, Anon. E. Moose #1000171, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 to look at the new patch set (#3). Change subject: WIP: Merging pages ranges to reduce the network call .. WIP: Merging pages ranges to reduce the network call Details: Merging page ranges to reduce the number of network calls to object storage, with an aim to reduce the fetching of unwanted pages. Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 --- M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/MergedPageRanges.java A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/test/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/MergedPageRagesTest.java 4 files changed, 260 insertions(+), 4 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/67/18367/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 3 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-MessageType: newpatchset
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
Attention is currently required from: Ritik Raj. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. Patch Set 2: -Contrib Analytics Compatibility Compilation Successful https://cbjenkins.page.link/bQb1H2knKib3oanm7 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 2 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-Comment-Date: Wed, 12 Jun 2024 16:32:46 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Jenkins : Attention is currently required from: Murtadha Hubail. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/762/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 15:11:18 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: Testing socket timeout
Attention is currently required from: Hussain Towaileb. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 ) Change subject: Testing socket timeout .. Patch Set 2: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/38yr59KY1jxYMUvD9 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic9d82dc3c1e358efe9c196aa322824349a0e0ef4 Gerrit-Change-Number: 18349 Gerrit-PatchSet: 2 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Hussain Towaileb Gerrit-Comment-Date: Wed, 12 Jun 2024 15:09:17 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
>From Jenkins : Attention is currently required from: Ritik Raj. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. Patch Set 2: Integration-Tests-1 Integration Tests Failed https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/761/ : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 2 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-Comment-Date: Wed, 12 Jun 2024 13:47:44 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
Attention is currently required from: Ritik Raj. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. Patch Set 2: Contrib-2 Analytics Compatibility Compilation Failed https://cbjenkins.page.link/mVYG2TwdmLHPr7Hf6 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 2 Gerrit-Owner: Ritik Raj Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Ritik Raj Gerrit-Comment-Date: Wed, 12 Jun 2024 13:26:31 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: Testing socket timeout
>From Jenkins : Attention is currently required from: Hussain Towaileb. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 ) Change subject: Testing socket timeout .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/759/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic9d82dc3c1e358efe9c196aa322824349a0e0ef4 Gerrit-Change-Number: 18349 Gerrit-PatchSet: 2 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Hussain Towaileb Gerrit-Comment-Date: Wed, 12 Jun 2024 13:15:53 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
>From Ritik Raj : Ritik Raj has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 ) Change subject: WIP: Merging pages ranges to reduce the network call .. WIP: Merging pages ranges to reduce the network call Details: Merging page ranges to reduce the number of network calls to object storage, with an aim to reduce the fetching of unwanted pages. Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 --- M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/MergedPageRanges.java A hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/test/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/MergedPageRagesTest.java 4 files changed, 260 insertions(+), 4 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/67/18367/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17 Gerrit-Change-Number: 18367 Gerrit-PatchSet: 2 Gerrit-Owner: Ritik Raj Gerrit-CC: Jenkins Gerrit-MessageType: newpatchset
Change in asterixdb[master]: WIP: Merging pages ranges to reduce the network call
>From Ritik Raj :
Ritik Raj has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18367 )
Change subject: WIP: Merging pages ranges to reduce the network call
..
WIP: Merging pages ranges to reduce the network call
Details:
Merging page ranges to reduce the number of
network calls to object storage, with an aim
to reduce the fetching of unwanted pages.
Change-Id: I343abcc5d2c2f1e65e8566e2a346e0a66da11d17
---
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
A
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/MergedPageRanges.java
A
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/test/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/MergedPageRagesTest.java
4 files changed, 260 insertions(+), 4 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/67/18367/1
diff --git
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
index fb3019f..9ea47c1 100644
---
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
+++
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
@@ -38,6 +38,7 @@
import
org.apache.hyracks.storage.am.lsm.btree.column.cloud.buffercache.IColumnReadContext;
import
org.apache.hyracks.storage.am.lsm.btree.column.cloud.sweep.ColumnSweepLockInfo;
import
org.apache.hyracks.storage.am.lsm.btree.column.impls.btree.ColumnBTreeReadLeafFrame;
+import org.apache.hyracks.storage.common.arraylist.IntArrayList;
import org.apache.hyracks.storage.common.buffercache.BufferCacheHeaderHelper;
import org.apache.hyracks.storage.common.buffercache.CachedPage;
import org.apache.hyracks.storage.common.buffercache.IBufferCache;
@@ -49,9 +50,12 @@
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import it.unimi.dsi.fastutil.Pair;
+
@NotThreadSafe
public final class CloudColumnReadContext implements IColumnReadContext {
private static final Logger LOGGER = LogManager.getLogger();
+public static final Integer MAXIMUM_RESULTANT_RANGES = 2;
private final ColumnProjectorType operation;
private final IPhysicalDrive drive;
private final BitSet plan;
@@ -60,6 +64,7 @@
private final CloudMegaPageReadContext columnCtx;
private final List pinnedPages;
private final BitSet projectedColumns;
+private final MergedPageRanges mergedPageRanges;
public CloudColumnReadContext(IColumnProjectionInfo projectionInfo,
IPhysicalDrive drive, BitSet plan) {
this.operation = projectionInfo.getProjectorType();
@@ -70,6 +75,7 @@
columnCtx = new CloudMegaPageReadContext(operation, columnRanges,
drive);
pinnedPages = new ArrayList<>();
projectedColumns = new BitSet();
+mergedPageRanges = new MergedPageRanges(MAXIMUM_RESULTANT_RANGES);
if (operation == QUERY || operation == MODIFY) {
for (int i = 0; i < projectionInfo.getNumberOfProjectedColumns();
i++) {
int columnIndex = projectionInfo.getColumnIndex(i);
@@ -157,6 +163,10 @@
int[] columnsOrder = columnRanges.getColumnsOrder();
int i = 0;
int columnIndex = columnsOrder[i];
+
+IntArrayList pageRanges = new IntArrayList(40, 40);
+int rangeIndex = 0;
+int lastPageIdx = -1;
while (columnIndex > -1) {
if (columnIndex < columnRanges.getNumberOfPrimaryKeys()) {
columnIndex = columnsOrder[++i];
@@ -164,8 +174,12 @@
}
int firstPageIdx =
columnRanges.getColumnStartPageIndex(columnIndex);
+// not the first range
+if (lastPageIdx != -1) {
+mergedPageRanges.addGap(firstPageIdx - lastPageIdx,
rangeIndex);
+}
// last page of the column
-int lastPageIdx = firstPageIdx +
columnRanges.getColumnNumberOfPages(columnIndex) - 1;
+lastPageIdx = firstPageIdx +
columnRanges.getColumnNumberOfPages(columnIndex) - 1;
Change in asterixdb[master]: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient
>From Ali Alsuliman : Attention is currently required from: Murtadha Hubail, Wail Alkowaileet. Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 ) Change subject: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient .. Patch Set 2: Verified+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf Gerrit-Change-Number: 18364 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Murtadha Hubail Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Wed, 12 Jun 2024 12:23:04 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient
>From Ali Alsuliman :
Ali Alsuliman has submitted this change. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 )
Change subject: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient
..
[NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient
- user model changes: no
- storage format changes: no
- interface changes: yes
Details:
- Expose the object requests counts from IRequestProfiler.
Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364
Reviewed-by: Hussain Towaileb
Tested-by: Ali Alsuliman
---
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/IRequestProfiler.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/NoOpRequestProfiler.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
6 files changed, 120 insertions(+), 0 deletions(-)
Approvals:
Hussain Towaileb: Looks good to me, approved
Ali Alsuliman: Verified
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
index 89f3c81..209245b 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
@@ -25,6 +25,7 @@
import java.util.Set;
import org.apache.asterix.cloud.IWriteBufferProvider;
+import org.apache.asterix.cloud.clients.profiler.IRequestProfiler;
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.api.io.FileReference;
import org.apache.hyracks.control.nc.io.IOManager;
@@ -42,6 +43,11 @@
int getWriteBufferSize();
/**
+ * @return the requests profiler
+ */
+IRequestProfiler getProfiler();
+
+/**
* Creates a cloud buffered writer
*
* @param bucket bucket to write to
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
index 2eae455..5ce1f43 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
@@ -105,6 +105,11 @@
}
@Override
+public IRequestProfiler getProfiler() {
+return profiler;
+}
+
+@Override
public ICloudWriter createWriter(String bucket, String path,
IWriteBufferProvider bufferProvider) {
ICloudBufferedWriter bufferedWriter = new S3BufferedWriter(s3Client,
profiler, guardian, bucket, path);
return new CloudResettableInputStream(bufferedWriter, bufferProvider);
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
index accf3c9..2874b4e 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
@@ -91,6 +91,11 @@
}
@Override
+public IRequestProfiler getProfiler() {
+return profiler;
+}
+
+@Override
public ICloudWriter createWriter(String bucket, String path,
IWriteBufferProvider bufferProvider) {
return new GCSWriter(bucket, path, gcsClient, profiler);
}
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
index 624395b..3fc378c 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
@@ -95,6 +95,41 @@
log();
}
+@Override
+public long objectsListCount() {
+return listObjectsCounter.get();
+}
+
+@Override
+public long objectGetCount() {
+return getObjectCounter.get();
+}
+
+@Override
+public long objectWriteCount() {
+return writeObjectCounter.get();
+}
+
+@Override
+public long objectDeleteCount() {
+
Change in asterixdb[master]: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient
>From Hussain Towaileb : Attention is currently required from: Murtadha Hubail, Wail Alkowaileet, Ali Alsuliman. Hussain Towaileb has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 ) Change subject: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf Gerrit-Change-Number: 18364 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Murtadha Hubail Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Ali Alsuliman Gerrit-Comment-Date: Wed, 12 Jun 2024 12:22:46 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient
>From Ali Alsuliman : Attention is currently required from: Murtadha Hubail, Wail Alkowaileet, Ali Alsuliman, Hussain Towaileb. Hello Murtadha Hubail, Wail Alkowaileet, Jenkins, Hussain Towaileb, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 to look at the new patch set (#2). Change subject: [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient .. [NO ISSUE][OTH] Expose IRequestProfiler from ICloudClient - user model changes: no - storage format changes: no - interface changes: yes Details: - Expose the object requests counts from IRequestProfiler. Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf --- M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/IRequestProfiler.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/NoOpRequestProfiler.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java 6 files changed, 117 insertions(+), 0 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/64/18364/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf Gerrit-Change-Number: 18364 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Murtadha Hubail Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Ali Alsuliman Gerrit-Attention: Hussain Towaileb Gerrit-MessageType: newpatchset
Change in asterixdb[master]: Testing socket timeout
Attention is currently required from: Hussain Towaileb. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 ) Change subject: Testing socket timeout .. Patch Set 2: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/V9ygnffT8ukqo2HW9 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic9d82dc3c1e358efe9c196aa322824349a0e0ef4 Gerrit-Change-Number: 18349 Gerrit-PatchSet: 2 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Hussain Towaileb Gerrit-Comment-Date: Wed, 12 Jun 2024 12:12:24 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: Testing socket timeout
>From Hussain Towaileb : Attention is currently required from: Hussain Towaileb. Hello Jenkins, Anon. E. Moose #1000171, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 to look at the new patch set (#2). Change subject: Testing socket timeout .. Testing socket timeout Change-Id: Ic9d82dc3c1e358efe9c196aa322824349a0e0ef4 --- M asterixdb/asterix-external-data/pom.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/aws/s3/S3Utils.java M asterixdb/pom.xml M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/AwsS3InputStream.java 4 files changed, 37 insertions(+), 0 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/49/18349/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18349 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic9d82dc3c1e358efe9c196aa322824349a0e0ef4 Gerrit-Change-Number: 18349 Gerrit-PatchSet: 2 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Hussain Towaileb Gerrit-MessageType: newpatchset
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Jenkins : Attention is currently required from: Savyasach Reddy. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Patch Set 5: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/758/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 5 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-Comment-Date: Wed, 12 Jun 2024 11:20:36 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Savyasach Reddy : Attention is currently required from: Savyasach Reddy. Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 to look at the new patch set (#5). Change subject: Handle Exceptions while reading Avro data format .. Handle Exceptions while reading Avro data format Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractDataParser.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/hdfs/parquet/converter/ParquetConverterContext.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AvroDataParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/ParserContext.java M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties 7 files changed, 54 insertions(+), 29 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/66/18366/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 5 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-MessageType: newpatchset
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Jenkins : Attention is currently required from: Savyasach Reddy. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Patch Set 4: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/757/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 4 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-Comment-Date: Wed, 12 Jun 2024 09:56:37 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
Attention is currently required from: Savyasach Reddy. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Patch Set 4: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/9mZDP2HhfQtRZgxT8 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 4 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-Comment-Date: Wed, 12 Jun 2024 09:26:06 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Savyasach Reddy : Attention is currently required from: Savyasach Reddy. Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 to look at the new patch set (#4). Change subject: Handle Exceptions while reading Avro data format .. Handle Exceptions while reading Avro data format Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 --- M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AvroDataParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties 4 files changed, 42 insertions(+), 17 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/66/18366/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 4 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-MessageType: newpatchset
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Jenkins : Attention is currently required from: Savyasach Reddy. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/756/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 3 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-Comment-Date: Wed, 12 Jun 2024 08:45:32 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3427][COMP] CBO not picking array index
Attention is currently required from: Vijay Sarathy. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 ) Change subject: [ASTERIXDB-3427][COMP] CBO not picking array index .. Patch Set 2: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/Qq6qCFWDA7y5p5fc8 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1 Gerrit-Change-Number: 18365 Gerrit-PatchSet: 2 Gerrit-Owner: Vijay Sarathy Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Vijay Sarathy Gerrit-Comment-Date: Wed, 12 Jun 2024 08:06:26 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
Attention is currently required from: Savyasach Reddy. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Patch Set 3: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/YGQZPp7TdcA1yMjB6 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 3 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-Comment-Date: Wed, 12 Jun 2024 07:35:42 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Savyasach Reddy : Attention is currently required from: Savyasach Reddy. Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 to look at the new patch set (#3). Change subject: Handle Exceptions while reading Avro data format .. Handle Exceptions while reading Avro data format Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AvroDataParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java 2 files changed, 43 insertions(+), 17 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/66/18366/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 3 Gerrit-Owner: Savyasach Reddy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Savyasach Reddy Gerrit-MessageType: newpatchset
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Savyasach Reddy : Savyasach Reddy has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Handle Exceptions while reading Avro data format Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AvroDataParser.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java 2 files changed, 41 insertions(+), 17 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/66/18366/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 2 Gerrit-Owner: Savyasach Reddy Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-MessageType: newpatchset
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 ) Change subject: Handle Exceptions while reading Avro data format .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/T4zjZswszWmUCmcd6 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93 Gerrit-Change-Number: 18366 Gerrit-PatchSet: 1 Gerrit-Owner: Savyasach Reddy Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 07:07:56 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: Handle Exceptions while reading Avro data format
>From Savyasach Reddy :
Savyasach Reddy has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18366 )
Change subject: Handle Exceptions while reading Avro data format
..
Handle Exceptions while reading Avro data format
Change-Id: I84405ce8142a887c341e2f8d82a68d58e287dd93
---
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AvroDataParser.java
M
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java
2 files changed, 41 insertions(+), 17 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/66/18366/1
diff --git
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java
index 3f92f00..b6d7798 100644
---
a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java
+++
b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/stream/AvroRecordReader.java
@@ -20,6 +20,7 @@
import static
org.apache.asterix.external.util.ExternalDataConstants.EMPTY_STRING;
import static
org.apache.asterix.external.util.ExternalDataConstants.KEY_REDACT_WARNINGS;
+import static org.apache.hyracks.api.util.ExceptionUtils.getMessageOrToString;
import java.io.IOException;
import java.util.Arrays;
@@ -37,6 +38,7 @@
import org.apache.asterix.external.util.ExternalDataConstants;
import org.apache.asterix.external.util.ExternalDataUtils;
import org.apache.asterix.external.util.IFeedLogManager;
+import org.apache.avro.AvroRuntimeException;
import org.apache.avro.InvalidAvroMagicException;
import org.apache.avro.file.DataFileStream;
import org.apache.avro.generic.GenericDatumReader;
@@ -55,7 +57,7 @@
private static final List recordReaderFormats =
Collections.unmodifiableList(Arrays.asList(ExternalDataConstants.FORMAT_AVRO));
-public AvroRecordReader(AsterixInputStream inputStream, Map config) throws IOException {
+public AvroRecordReader(AsterixInputStream inputStream, Map config) throws HyracksDataException {
record = new
org.apache.asterix.external.input.record.GenericRecord<>();
this.inputStream = new DiscretizedMultipleInputStream(inputStream);
done = false;
@@ -69,11 +71,13 @@
}
@Override
-public void close() throws IOException {
+public void close() throws HyracksDataException {
try {
if (!done) {
inputStream.close();
}
+} catch (IOException e) {
+throw HyracksDataException.create(e);
} finally {
done = true;
}
@@ -91,21 +95,31 @@
}
@Override
-public IRawRecord next() throws IOException {
-avroRecord = dataFileStream.next(avroRecord);
-record.set(avroRecord);
-return record;
+public IRawRecord next() throws HyracksDataException {
+try {
+avroRecord = dataFileStream.next(avroRecord);
+record.set(avroRecord);
+return record;
+} catch (AvroRuntimeException e) {
+throw new RuntimeDataException(ErrorCode.EXTERNAL_SOURCE_ERROR, e,
getMessageOrToString(e));
+} catch (IOException e) {
+throw HyracksDataException.create(e);
+}
}
@Override
-public boolean hasNext() throws IOException {
-if (dataFileStream == null) {
-return false;
+public boolean hasNext() throws HyracksDataException {
+try {
+if (dataFileStream == null) {
+return false;
+}
+if (dataFileStream.hasNext()) {
+return true;
+}
+return advance() && dataFileStream.hasNext();
+} catch (AvroRuntimeException e) {
+throw new RuntimeDataException(ErrorCode.EXTERNAL_SOURCE_ERROR, e,
getMessageOrToString(e));
}
-if (dataFileStream.hasNext()) {
-return true;
-}
-return advance() && dataFileStream.hasNext();
}
@Override
@@ -135,7 +149,7 @@
}
-private boolean advance() throws IOException {
+private boolean advance() throws HyracksDataException {
try {
if (inputStream.advance()) {
DatumReader datumReader = new
GenericDatumReader<>();
@@ -144,8 +158,9 @@
}
} catch (InvalidAvroMagicException e) {
throw new
RuntimeDataException(ErrorCode.RECORD_READER_MALFORMED_INPUT_STREAM, e);
+} catch (IOException e) {
+throw HyracksDataException.create(e);
}
return false;
}
-
}
diff --git
Change in asterixdb[master]: [ASTERIXDB-3427][COMP] CBO not picking array index
>From Jenkins : Attention is currently required from: Vijay Sarathy. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 ) Change subject: [ASTERIXDB-3427][COMP] CBO not picking array index .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/753/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1 Gerrit-Change-Number: 18365 Gerrit-PatchSet: 2 Gerrit-Owner: Vijay Sarathy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Vijay Sarathy Gerrit-Comment-Date: Wed, 12 Jun 2024 06:00:40 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3427][COMP] CBO not picking array index
Attention is currently required from: Vijay Sarathy. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 ) Change subject: [ASTERIXDB-3427][COMP] CBO not picking array index .. Patch Set 2: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/58AGq62K6Qrx3ZFm9 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1 Gerrit-Change-Number: 18365 Gerrit-PatchSet: 2 Gerrit-Owner: Vijay Sarathy Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Vijay Sarathy Gerrit-Comment-Date: Wed, 12 Jun 2024 05:30:33 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3427][COMP] CBO not picking array index
>From Vijay Sarathy : Vijay Sarathy has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 ) Change subject: [ASTERIXDB-3427][COMP] CBO not picking array index .. [ASTERIXDB-3427][COMP] CBO not picking array index Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/Stats.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/JoinNode.java 2 files changed, 57 insertions(+), 27 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/65/18365/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1 Gerrit-Change-Number: 18365 Gerrit-PatchSet: 2 Gerrit-Owner: Vijay Sarathy Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-MessageType: newpatchset
Change in asterixdb[master]: [ASTERIXDB-3427][COMP] CBO not picking array index`
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 ) Change subject: [ASTERIXDB-3427][COMP] CBO not picking array index` .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/koKSSb8rt8fWhf3x6 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1 Gerrit-Change-Number: 18365 Gerrit-PatchSet: 1 Gerrit-Owner: Vijay Sarathy Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Wed, 12 Jun 2024 05:16:40 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3427][COMP] CBO not picking array index`
>From Vijay Sarathy :
Vijay Sarathy has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18365 )
Change subject: [ASTERIXDB-3427][COMP] CBO not picking array index`
..
[ASTERIXDB-3427][COMP] CBO not picking array index`
Change-Id: Ic47a22556a6fb9468e46244062d281bea7eb5fa1
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/Stats.java
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/JoinNode.java
2 files changed, 57 insertions(+), 27 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/65/18365/1
diff --git
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/JoinNode.java
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/JoinNode.java
index e868fa3..61f4d22 100644
---
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/JoinNode.java
+++
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/cbo/JoinNode.java
@@ -92,6 +92,7 @@
protected double origCardinality; // without any selections
protected double cardinality;
protected double size; // avg size of whole document; available from the
sample
+protected double unnestFactor;
protected double diskProjectionSize; // what is coming out of the disk; in
case of row format, it is the entire document
// in case of columnar we need to add
sizes of individual fields.
protected double projectionSizeAfterScan; // excludes fields only used for
selections
@@ -168,6 +169,10 @@
return size;
}
+public double getUnnestFactor() {
+return unnestFactor;
+}
+
public void setLimitVal(int val) {
limitVal = val;
}
@@ -247,33 +252,23 @@
public void setCardsAndSizes(Index.SampleIndexDetails idxDetails,
ILogicalOperator leafInput)
throws AlgebricksException {
-double origDatasetCard, finalDatasetCard;
-finalDatasetCard = origDatasetCard = idxDetails.getSourceCardinality();
+double origDatasetCard, finalDatasetCard, sampleCard;
+unnestFactor = 1.0;
DataSourceScanOperator scanOp =
joinEnum.findDataSourceScanOperator(leafInput);
if (scanOp == null) {
return; // what happens to the cards and sizes then? this may
happen in case of in lists
}
-double sampleCard = Math.min(idxDetails.getSampleCardinalityTarget(),
origDatasetCard);
-if (sampleCard == 0) { // should not happen unless the original
dataset is empty
-sampleCard = 1; // we may have to make some adjustments to costs
when the sample returns very rows.
-
-IWarningCollector warningCollector =
joinEnum.optCtx.getWarningCollector();
-if (warningCollector.shouldWarn()) {
-warningCollector.warn(Warning.of(scanOp.getSourceLocation(),
-
org.apache.asterix.common.exceptions.ErrorCode.SAMPLE_HAS_ZERO_ROWS));
-}
-}
-
List> result;
-SelectOperator selop = (SelectOperator)
joinEnum.findASelectOp(leafInput);
-if (selop == null) { // add a SelectOperator with TRUE condition. The
code below becomes simpler with a select operator.
-selop = new SelectOperator(new
MutableObject<>(ConstantExpression.TRUE));
-ILogicalOperator op = selop;
+SelectOperator selOp = (SelectOperator)
joinEnum.findASelectOp(leafInput);
+if (selOp == null) { // add a SelectOperator with TRUE condition. The
code below becomes simpler with a select operator.
+selOp = new SelectOperator(new
MutableObject<>(ConstantExpression.TRUE));
+ILogicalOperator op = selOp;
op.getInputs().add(new MutableObject<>(leafInput));
leafInput = op;
}
+
ILogicalOperator parent =
joinEnum.findDataSourceScanOperatorParent(leafInput);
Mutable ref = new MutableObject<>(leafInput);
@@ -303,6 +298,29 @@
// if there is only one conjunct, I do not have to call the sampling
query during index selection!
// insert this in place of the scandatasourceOp.
parent.getInputs().get(0).setValue(deepCopyofScan);
+
+boolean unnest = joinEnum.findUnnestOp(selOp);
+if (unnest) {
+ILogicalExpression saveExpr = selOp.getCondition().getValue();
+sampleCard =
joinEnum.stats.computeUnnestedOriginalCardinality(selOp);
+selOp.getCondition().setValue(saveExpr); // restore the expression
+unnestFactor = sampleCard /
idxDetails.getSampleCardinalityTarget();
+finalDatasetCard = origDatasetCard =
idxDetails.getSourceCardinality() * unnestFactor;
+} else {
+finalDatasetCard = origDatasetCard =
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
Attention is currently required from: Murtadha Hubail. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/wkmkgURtXmTWok3J7 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 03:52:55 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Jenkins : Attention is currently required from: Murtadha Hubail. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/751/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 02:14:47 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: cloud requests
>From Jenkins : Attention is currently required from: Murtadha Hubail, Wail Alkowaileet, Hussain Towaileb. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 ) Change subject: WIP: cloud requests .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/750/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf Gerrit-Change-Number: 18364 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Murtadha Hubail Gerrit-Attention: Wail Alkowaileet Gerrit-Attention: Hussain Towaileb Gerrit-Comment-Date: Wed, 12 Jun 2024 01:52:42 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
Attention is currently required from: Murtadha Hubail. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. Patch Set 2: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/UnGz9XCEsoULFDye9 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Wed, 12 Jun 2024 01:17:33 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Murtadha Hubail : Murtadha Hubail has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 ) Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup .. [NO ISSUE][TEST] Catch all failures on cc/nc startup - user model changes: no - storage format changes: no - interface changes: no Details: - Catch all nc/cc startup failures in AsterixHyracksIntegrationUtil. Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 --- M asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java 1 file changed, 41 insertions(+), 13 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/63/18363/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020 Gerrit-Change-Number: 18363 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-CC: Jenkins Gerrit-MessageType: newpatchset
Change in asterixdb[master]: WIP: cloud requests
>From Ali Alsuliman :
Ali Alsuliman has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18364 )
Change subject: WIP: cloud requests
..
WIP: cloud requests
Change-Id: I6964af27bf66717661cb450de793ca7d06bb9eaf
---
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/IRequestProfiler.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/NoOpRequestProfiler.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
M
asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
6 files changed, 110 insertions(+), 0 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/64/18364/1
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
index 89f3c81..209245b 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/ICloudClient.java
@@ -25,6 +25,7 @@
import java.util.Set;
import org.apache.asterix.cloud.IWriteBufferProvider;
+import org.apache.asterix.cloud.clients.profiler.IRequestProfiler;
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.api.io.FileReference;
import org.apache.hyracks.control.nc.io.IOManager;
@@ -42,6 +43,11 @@
int getWriteBufferSize();
/**
+ * @return the requests profiler
+ */
+IRequestProfiler getProfiler();
+
+/**
* Creates a cloud buffered writer
*
* @param bucket bucket to write to
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
index 2eae455..5ce1f43 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java
@@ -105,6 +105,11 @@
}
@Override
+public IRequestProfiler getProfiler() {
+return profiler;
+}
+
+@Override
public ICloudWriter createWriter(String bucket, String path,
IWriteBufferProvider bufferProvider) {
ICloudBufferedWriter bufferedWriter = new S3BufferedWriter(s3Client,
profiler, guardian, bucket, path);
return new CloudResettableInputStream(bufferedWriter, bufferProvider);
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
index accf3c9..2874b4e 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java
@@ -91,6 +91,11 @@
}
@Override
+public IRequestProfiler getProfiler() {
+return profiler;
+}
+
+@Override
public ICloudWriter createWriter(String bucket, String path,
IWriteBufferProvider bufferProvider) {
return new GCSWriter(bucket, path, gcsClient, profiler);
}
diff --git
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
index 624395b..3fc378c 100644
---
a/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
+++
b/asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/profiler/CountRequestProfiler.java
@@ -95,6 +95,41 @@
log();
}
+@Override
+public long objectsListCount() {
+return listObjectsCounter.get();
+}
+
+@Override
+public long objectGetCount() {
+return getObjectCounter.get();
+}
+
+@Override
+public long objectWriteCount() {
+return writeObjectCounter.get();
+}
+
+@Override
+public long objectDeleteCount() {
+return deleteObjectCounter.get();
+}
+
+@Override
+public long objectCopyCount() {
+return copyObjectCounter.get();
+}
+
+@Override
+public long objectMultipartUploadCount() {
+return multipartUploadCounter.get();
+}
+
+@Override
+public long objectMultipartDownloadCount() {
+
Change in asterixdb[master]: [NO ISSUE][TEST] Catch all failures on cc/nc startup
>From Murtadha Hubail :
Murtadha Hubail has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18363 )
Change subject: [NO ISSUE][TEST] Catch all failures on cc/nc startup
..
[NO ISSUE][TEST] Catch all failures on cc/nc startup
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Catch all nc/cc startup failures in AsterixHyracksIntegrationUtil.
Change-Id: Ifcbe1a0dff7f3dbb7e6b258d43bcbd5a2deb7020
---
M
asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
1 file changed, 41 insertions(+), 12 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/63/18363/1
diff --git
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
index 506e82a..bfe180f 100644
---
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
+++
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java
@@ -31,6 +31,7 @@
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.BiPredicate;
import java.util.stream.Stream;
@@ -110,8 +111,8 @@
try {
integrationUtil.run(Boolean.getBoolean("cleanup.start"),
Boolean.getBoolean("cleanup.shutdown"),
getConfPath());
-} catch (Exception e) {
-LOGGER.fatal("Unexpected exception", e);
+} catch (Throwable t) {
+LOGGER.fatal("Unexpected exception", t);
System.exit(1);
}
}
@@ -155,19 +156,27 @@
}
opts.forEach(opt -> configManager.set(opt.getLeft(), opt.getRight()));
-cc.start();
+try {
+cc.start();
+} catch (Throwable t) {
+LOGGER.error("failed to start cc", t);
+throw t;
+}
// Starts ncs.
nodeNames = ccConfig.getConfigManager().getNodeNames();
List startupThreads = new ArrayList<>();
+AtomicBoolean ncFailedToStart = new AtomicBoolean(false);
for (NodeControllerService nc : nodeControllers) {
Thread ncStartThread = new Thread("IntegrationUtil-" + nc.getId())
{
@Override
public void run() {
try {
nc.start();
-} catch (Exception e) {
-LOGGER.log(Level.ERROR, e.getMessage(), e);
+LOGGER.info("started node {}", nc.getId());
+} catch (Throwable t) {
+LOGGER.error("failed to start node {}", nc.getId(), t);
+ncFailedToStart.set(true);
}
}
};
@@ -178,11 +187,14 @@
for (Thread thread : startupThreads) {
thread.join();
}
+if (ncFailedToStart.get()) {
+throw new Exception("some node failed to start");
+}
// Wait until cluster becomes active
((ICcApplicationContext)
cc.getApplicationContext()).getClusterStateManager().waitForState(ClusterState.ACTIVE);
hcc = new HyracksConnection(cc.getConfig().getClientListenAddress(),
cc.getConfig().getClientListenPort(),
cc.getNetworkSecurityManager().getSocketChannelFactory());
-this.ncs = nodeControllers.toArray(new
NodeControllerService[nodeControllers.size()]);
+this.ncs = nodeControllers.toArray(new NodeControllerService[0]);
}
@NotNull
@@ -297,8 +309,8 @@
public void run() {
try {
nodeControllerService.stop();
-} catch (Exception e) {
-e.printStackTrace();
+} catch (Throwable t) {
+LOGGER.error("failed to stop node {}",
nodeControllerService.getId(), t);
}
}
};
@@ -367,8 +379,8 @@
public void run() {
try {
deinit(cleanupOnShutdown);
-} catch (Exception e) {
-LOGGER.log(Level.WARN, "Unexpected exception on shutdown",
e);
+} catch (Throwable t) {
+LOGGER.warn("Unexpected exception on shutdown", t);
}
}
});
@@ -385,8 +397,8 @@
public void run() {
try {
deinit(cleanupOnShutdown);
-} catch (Exception e) {
-LOGGER.log(Level.WARN, "Unexpected exception on shutdown",
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
>From Peeyush Gupta :
Peeyush Gupta has submitted this change. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 )
Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
..
[ASTERIXDB-3426][TX] Error during recovery for atomic statements
- user model changes: no
- storage format changes: no
- interface changes: yes
Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362
Integration-Tests: Jenkins
Tested-by: Jenkins
Reviewed-by: Peeyush Gupta
Reviewed-by: Murtadha Hubail
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AtomicJobRollbackMessage.java
M
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/AtomicNoWALTransactionContext.java
M
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IIndexCheckpointManager.java
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
4 files changed, 24 insertions(+), 6 deletions(-)
Approvals:
Murtadha Hubail: Looks good to me, approved
Peeyush Gupta: Looks good to me, but someone else must approve
Jenkins: Verified; Verified
Objections:
Anon. E. Moose #1000171: Violations found
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AtomicJobRollbackMessage.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AtomicJobRollbackMessage.java
index e611d8f..2110bc8 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AtomicJobRollbackMessage.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AtomicJobRollbackMessage.java
@@ -71,7 +71,7 @@
if (checkpoint.getLastComponentId() == v.getMaxId()) {
LOGGER.info("Removing checkpoint for resource {} for
component id {}", k,
checkpoint.getLastComponentId());
-checkpointManager.deleteLatest(v.getMaxId(), 1);
+checkpointManager.deleteLatest(v.getMaxId());
}
}
} catch (Exception e) {
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
index 77a4f19..4ca9f7c 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
@@ -231,18 +231,18 @@
}
@Override
-public void deleteLatest(long latestId, int historyToDelete) {
+public void deleteLatest(long latestId) {
try {
final Collection checkpointFiles =
ioManager.list(indexPath, CHECKPOINT_FILE_FILTER);
if (!checkpointFiles.isEmpty()) {
for (FileReference checkpointFile : checkpointFiles) {
-if (getCheckpointIdFromFileName(checkpointFile) >
(latestId - historyToDelete)) {
+if (read(checkpointFile).getLastComponentId() == latestId)
{
ioManager.delete(checkpointFile);
}
}
}
} catch (Exception e) {
-LOGGER.warn(() -> "Couldn't delete history checkpoints at " +
indexPath, e);
+LOGGER.warn(() -> "Couldn't delete latest checkpoint file at " +
indexPath, e);
}
}
diff --git
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IIndexCheckpointManager.java
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IIndexCheckpointManager.java
index 75557ec..2cf613b 100644
---
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IIndexCheckpointManager.java
+++
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IIndexCheckpointManager.java
@@ -102,7 +102,7 @@
*/
void delete();
-void deleteLatest(long latestId, int historyToDelete);
+void deleteLatest(long latestId);
/**
* Gets the index last valid component sequence.
diff --git
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/AtomicNoWALTransactionContext.java
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/AtomicNoWALTransactionContext.java
index 151d9ef..a743bb1 100644
---
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/AtomicNoWALTransactionContext.java
+++
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/AtomicNoWALTransactionContext.java
@@ -198,7 +198,7 @@
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
Attention is currently required from: Peeyush Gupta, Ali Alsuliman. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements .. Patch Set 3: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/7g94kFXCwivM3CNH8 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 3 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Peeyush Gupta Gerrit-Attention: Peeyush Gupta Gerrit-Attention: Ali Alsuliman Gerrit-Comment-Date: Tue, 11 Jun 2024 18:42:29 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
>From Murtadha Hubail : Attention is currently required from: Peeyush Gupta, Ali Alsuliman. Murtadha Hubail has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements .. Patch Set 3: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 3 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Peeyush Gupta Gerrit-Attention: Peeyush Gupta Gerrit-Attention: Ali Alsuliman Gerrit-Comment-Date: Tue, 11 Jun 2024 18:39:02 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
>From Peeyush Gupta : Attention is currently required from: Murtadha Hubail, Ali Alsuliman. Peeyush Gupta has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements .. Patch Set 3: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 3 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Peeyush Gupta Gerrit-Attention: Murtadha Hubail Gerrit-Attention: Ali Alsuliman Gerrit-Comment-Date: Tue, 11 Jun 2024 17:25:46 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
>From Jenkins : Attention is currently required from: Peeyush Gupta. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/748/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 3 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-Comment-Date: Tue, 11 Jun 2024 16:33:13 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
Attention is currently required from: Peeyush Gupta. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements .. Patch Set 3: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/m8qbJ9jveJ6GX48u5 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 3 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-Comment-Date: Tue, 11 Jun 2024 16:01:39 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3426][TX] Error during recovery for atomic statements
>From Peeyush Gupta : Attention is currently required from: Peeyush Gupta. Hello Jenkins, Anon. E. Moose #1000171, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 to look at the new patch set (#3). Change subject: [ASTERIXDB-3426][TX] Error during recovery for atomic statements .. [ASTERIXDB-3426][TX] Error during recovery for atomic statements - user model changes: no - storage format changes: no - interface changes: yes Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AtomicJobRollbackMessage.java M asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/AtomicNoWALTransactionContext.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IIndexCheckpointManager.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java 4 files changed, 19 insertions(+), 6 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/62/18362/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 3 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-MessageType: newpatchset
Change in asterixdb[master]: WIP: fix recovery for atomic statements
Attention is currently required from: Peeyush Gupta. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: WIP: fix recovery for atomic statements .. Patch Set 2: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/RYDT8Lujz3aJucz86 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 2 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-Comment-Date: Tue, 11 Jun 2024 07:43:03 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: fix recovery for atomic statements
>From Jenkins : Attention is currently required from: Peeyush Gupta. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: WIP: fix recovery for atomic statements .. Patch Set 2: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/747/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 2 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-Comment-Date: Tue, 11 Jun 2024 06:09:30 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: fix recovery for atomic statements
Attention is currently required from: Peeyush Gupta. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: WIP: fix recovery for atomic statements .. Patch Set 2: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/wRrNM1qM4tkQZZQn6 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 2 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-Comment-Date: Tue, 11 Jun 2024 05:08:18 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: fix recovery for atomic statements
Attention is currently required from: Peeyush Gupta. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: WIP: fix recovery for atomic statements .. Patch Set 1: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/SMdxx3v33R3QFsxn7 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 1 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Peeyush Gupta Gerrit-Comment-Date: Tue, 11 Jun 2024 03:41:09 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching
>From Wail Alkowaileet : Wail Alkowaileet has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 ) Change subject: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching .. [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching - user model changes: no - storage format changes: no - interface changes: yes Details: - ASTERIXDB-3424: Disk cache sweeper can interfere with rebalance - ASTERIXDB-3425: Use allocated storage quota to determine whether or not to persist non-column pages - Other fixes: - Make punchHole() uninterruptible - Allow plan reevaluation despite the disk is being pressured or not - Log the page info when the buffer cache's page read operation fails - Fix API tests failures (ASTERIXDB-3423) - Ignore CloudStorageGCSTest Change-Id: Ida79a61940ed8944ed9fb44ac7d8815d865b73e6 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 Integration-Tests: Jenkins Reviewed-by: Murtadha Hubail Tested-by: Jenkins --- M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/sweeper/SweepContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/DiskCacheSweeperThread.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/sweep/ColumnSweepPlanner.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/filesystem/HolePuncherProvider.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/write/CloudColumnWriteContext.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/cloud_storage/CloudStorageGCSTest.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/sweeper/Sweeper.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/CloudColumnIndexDiskCacheManager.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/filesystem/IHolePuncher.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/filesystem/PhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/accessor/SelectiveCloudAccessor.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/test/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/sweep/ColumnSweepPlannerTest.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/CloudDiskCacheMonitoringAndPrefetchingService.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/IDiskCacheMonitoringService.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/IPhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/DummyPhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/NoOpDiskCacheMonitoringService.java 23 files changed, 234 insertions(+), 96 deletions(-) Approvals: Murtadha Hubail: Looks good to me, approved Jenkins: Verified; Verified diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java index 547bc8b..b47703a 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java @@ -29,6 +29,7 @@ import org.apache.hyracks.api.control.CcId; import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.service.IControllerService; +import org.apache.hyracks.storage.common.disk.IDiskCacheMonitoringService; import
Change in asterixdb[master]: WIP: fix recovery for atomic statements
>From Jenkins : Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: WIP: fix recovery for atomic statements .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/746/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 1 Gerrit-Owner: Peeyush Gupta Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Comment-Date: Tue, 11 Jun 2024 02:09:30 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching
>From Murtadha Hubail : Attention is currently required from: Wail Alkowaileet. Murtadha Hubail has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 ) Change subject: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching .. Patch Set 3: Code-Review+2 (1 comment) File hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/DiskCacheSweeperThread.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361/comment/80d6ddb7_e076fb4c PS3, Line 124: Thread.currentThread().interrupt(); Usually the while(true) is replaced with while(!Thread.currentThread().isInterrupted()) then on catching InterruptedException you do the Thread.currentThread().interrupt();. You can log at the end that the sweeting thread is exiting -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ida79a61940ed8944ed9fb44ac7d8815d865b73e6 Gerrit-Change-Number: 18361 Gerrit-PatchSet: 3 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Tue, 11 Jun 2024 01:37:52 + Gerrit-HasComments: Yes Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching
>From Jenkins : Attention is currently required from: Wail Alkowaileet. Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 ) Change subject: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching .. Patch Set 3: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/745/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ida79a61940ed8944ed9fb44ac7d8815d865b73e6 Gerrit-Change-Number: 18361 Gerrit-PatchSet: 3 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Tue, 11 Jun 2024 01:11:33 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: fix recovery for atomic statements
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 ) Change subject: WIP: fix recovery for atomic statements .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/pCSCqQ9Yqk2GMYsx9 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238 Gerrit-Change-Number: 18362 Gerrit-PatchSet: 1 Gerrit-Owner: Peeyush Gupta Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Tue, 11 Jun 2024 01:07:48 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: WIP: fix recovery for atomic statements
>From Peeyush Gupta :
Peeyush Gupta has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362 )
Change subject: WIP: fix recovery for atomic statements
..
WIP: fix recovery for atomic statements
Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238
---
M
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
1 file changed, 10 insertions(+), 1 deletion(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/62/18362/1
diff --git
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
index 77a4f19..9463c88 100644
---
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
+++
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/IndexCheckpointManager.java
@@ -236,7 +236,7 @@
final Collection checkpointFiles =
ioManager.list(indexPath, CHECKPOINT_FILE_FILTER);
if (!checkpointFiles.isEmpty()) {
for (FileReference checkpointFile : checkpointFiles) {
-if (getCheckpointIdFromFileName(checkpointFile) >
(latestId - historyToDelete)) {
+if (read(checkpointFile).getLastComponentId() > (latestId
- historyToDelete)) {
ioManager.delete(checkpointFile);
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18362
To unsubscribe, or for help writing mail filters, visit
https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I2a92e07b230f6a1d4aca2cacff8a568c29615238
Gerrit-Change-Number: 18362
Gerrit-PatchSet: 1
Gerrit-Owner: Peeyush Gupta
Gerrit-MessageType: newchange
Change in asterixdb[master]: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching
>From Wail Alkowaileet : Wail Alkowaileet has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 ) Change subject: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching .. [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching - user model changes: no - storage format changes: no - interface changes: yes Details: - ASTERIXDB-3424: Disk cache sweeper can interfere with rebalance - ASTERIXDB-3425: Use allocated storage quota to determine whether or not to persist non-column pages - Other fixes: - Make punchHole() uninterruptible - Allow plan reevaluation despite the disk is being pressured or not - Log the page info when the buffer cache's page read operation fails - Fix API tests failures (ASTERIXDB-3423) - Ignore CloudStorageGCSTest Change-Id: Ida79a61940ed8944ed9fb44ac7d8815d865b73e6 --- M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/sweeper/SweepContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/DiskCacheSweeperThread.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/sweep/ColumnSweepPlanner.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/filesystem/HolePuncherProvider.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/write/CloudColumnWriteContext.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/cloud_storage/CloudStorageGCSTest.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/sweeper/Sweeper.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/CloudColumnIndexDiskCacheManager.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/filesystem/IHolePuncher.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/filesystem/PhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/accessor/SelectiveCloudAccessor.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/test/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/sweep/ColumnSweepPlannerTest.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/CloudDiskCacheMonitoringAndPrefetchingService.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/IDiskCacheMonitoringService.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/IPhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/DummyPhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/NoOpDiskCacheMonitoringService.java 23 files changed, 231 insertions(+), 97 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/61/18361/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ida79a61940ed8944ed9fb44ac7d8815d865b73e6 Gerrit-Change-Number: 18361 Gerrit-PatchSet: 2 Gerrit-Owner: Wail Alkowaileet Gerrit-CC: Jenkins Gerrit-MessageType: newpatchset
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
>From Wail Alkowaileet :
Wail Alkowaileet has submitted this change. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 )
Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read
..
[ASTERIXDB-3421][STO] Fix column offset and length on read
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Because we read the length of a column (4-bytes),
the remaining number of bytes should account for
those four bytes before reading the remaining pages
Change-Id: I035311613d950bcc73332f82730464cc362b7728
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360
Integration-Tests: Jenkins
Tested-by: Wail Alkowaileet
Reviewed-by: Wail Alkowaileet
Reviewed-by: Murtadha Hubail
---
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/lsm/tuples/ColumnMultiBufferProvider.java
3 files changed, 33 insertions(+), 10 deletions(-)
Approvals:
Murtadha Hubail: Looks good to me, approved
Wail Alkowaileet: Looks good to me, but someone else must approve; Verified
Jenkins: Verified
Objections:
Anon. E. Moose #1000171: Violations found
diff --git
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
index 7623698..fb3019f 100644
---
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
+++
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
@@ -154,12 +154,12 @@
// TODO What if every other page is requested. That would do N/2
request, where N is the number of pages.
// TODO This should be optimized in a way that minimizes the number of
requests
-int[] columnsOrders = columnRanges.getColumnsOrder();
+int[] columnsOrder = columnRanges.getColumnsOrder();
int i = 0;
-int columnIndex = columnsOrders[i];
+int columnIndex = columnsOrder[i];
while (columnIndex > -1) {
if (columnIndex < columnRanges.getNumberOfPrimaryKeys()) {
-columnIndex = columnsOrders[++i];
+columnIndex = columnsOrder[++i];
continue;
}
@@ -168,7 +168,7 @@
int lastPageIdx = firstPageIdx +
columnRanges.getColumnNumberOfPages(columnIndex) - 1;
// Advance to the next column to check if it has contiguous pages
-columnIndex = columnsOrders[++i];
+columnIndex = columnsOrder[++i];
while (columnIndex > -1) {
int sharedPageCount = 0;
// Get the next column's start page ID
@@ -183,7 +183,7 @@
lastPageIdx +=
columnRanges.getColumnNumberOfPages(columnIndex) - sharedPageCount;
// Advance to the next column
-columnIndex = columnsOrders[++i];
+columnIndex = columnsOrder[++i];
}
if (lastPageIdx >= columnRanges.getTotalNumberOfPages()) {
diff --git
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
index 21d5ce7..f28b2fd 100644
---
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
+++
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
@@ -116,7 +116,7 @@
* - The operation is not a merge operation (the component will be
deleted anyway)
* - The disk has space
*
- * Note: 'emtpy' can be false while 'cloudOnly is true'. We cannot
read from disk as the page can be
+ * Note: 'empty' can be false while 'cloudOnly is
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
>From Murtadha Hubail : Attention is currently required from: Wail Alkowaileet. Murtadha Hubail has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 ) Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I035311613d950bcc73332f82730464cc362b7728 Gerrit-Change-Number: 18360 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Mon, 10 Jun 2024 23:37:11 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching
>From Wail Alkowaileet : Wail Alkowaileet has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18361 ) Change subject: [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching .. [MULTIPLE ISSUES][STO] Multiple fixes for cloud caching - user model changes: no - storage format changes: no - interface changes: yes Details: - ASTERIXDB-3424: Disk cache sweeper can interfere with rebalance - ASTERIXDB-3425: Use allocated storage quota to determine whether or not to persist non-column pages - Other fixes: - Make punchHole() uninterruptible - Allow plan reevaluation despite the disk is being pressured or not - Log the page info when the buffer cache's page read operation fails - Fix API tests failures (ASTERIXDB-3423) - Ignore CloudStorageGCSTest Change-Id: Ida79a61940ed8944ed9fb44ac7d8815d865b73e6 --- M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_full/cluster_state_1_full.1.regexadm M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/sweeper/SweepContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1/cluster_state_1.1.regexadm M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/DiskCacheSweeperThread.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/sweep/ColumnSweepPlanner.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/filesystem/HolePuncherProvider.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/write/CloudColumnWriteContext.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/cloud_storage/CloudStorageGCSTest.java M asterixdb/asterix-app/src/test/resources/runtimets/results/api/cluster_state_1_less/cluster_state_1_less.1.regexadm M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/sweeper/Sweeper.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/CloudColumnIndexDiskCacheManager.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/filesystem/IHolePuncher.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/filesystem/PhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/BufferCache.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/lazy/accessor/SelectiveCloudAccessor.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/test/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/sweep/ColumnSweepPlannerTest.java M hyracks-fullstack/hyracks/hyracks-cloud/src/main/java/org/apache/hyracks/cloud/cache/service/CloudDiskCacheMonitoringAndPrefetchingService.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/IDiskCacheMonitoringService.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/IPhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/DummyPhysicalDrive.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/disk/NoOpDiskCacheMonitoringService.java 23 files changed, 219 insertions(+), 88 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/61/18361/1 diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java index 547bc8b..236f126 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CloudToLocalStorageCachingTask.java @@ -29,6 +29,7 @@ import org.apache.hyracks.api.control.CcId; import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.service.IControllerService; +import org.apache.hyracks.storage.common.disk.IDiskCacheMonitoringService; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -55,6 +56,10 @@ INcApplicationContext
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
>From Wail Alkowaileet : Attention is currently required from: Murtadha Hubail. Wail Alkowaileet has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 ) Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read .. Patch Set 1: Verified+1 Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I035311613d950bcc73332f82730464cc362b7728 Gerrit-Change-Number: 18360 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Wail Alkowaileet Gerrit-Attention: Murtadha Hubail Gerrit-Comment-Date: Mon, 10 Jun 2024 22:17:50 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
>From Wail Alkowaileet : Attention is currently required from: Wail Alkowaileet. Wail Alkowaileet has removed a vote from this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 ) Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read .. Removed Verified-1 by Jenkins -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I035311613d950bcc73332f82730464cc362b7728 Gerrit-Change-Number: 18360 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Wail Alkowaileet Gerrit-MessageType: deleteVote
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
Attention is currently required from: Wail Alkowaileet. Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 ) Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read .. Patch Set 1: Contrib-2 Analytics Compatibility Tests Failed https://cbjenkins.page.link/6FVDpnTPwvr9n2nv5 : UNSTABLE -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I035311613d950bcc73332f82730464cc362b7728 Gerrit-Change-Number: 18360 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Wail Alkowaileet Gerrit-Comment-Date: Mon, 10 Jun 2024 21:44:46 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
>From Jenkins : Jenkins has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 ) Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read .. Patch Set 1: Integration-Tests+1 Integration Tests Successful https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-trigger/742/ : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I035311613d950bcc73332f82730464cc362b7728 Gerrit-Change-Number: 18360 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-Reviewer: Jenkins Gerrit-CC: Anon. E. Moose #1000171 Gerrit-Comment-Date: Mon, 10 Jun 2024 20:14:09 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
Anon. E. Moose #1000171 has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 ) Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read .. Patch Set 1: Analytics Compatibility Compilation Successful https://cbjenkins.page.link/SrHWf3foRPii3gW97 : SUCCESS -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I035311613d950bcc73332f82730464cc362b7728 Gerrit-Change-Number: 18360 Gerrit-PatchSet: 1 Gerrit-Owner: Wail Alkowaileet Gerrit-CC: Anon. E. Moose #1000171 Gerrit-CC: Jenkins Gerrit-Comment-Date: Mon, 10 Jun 2024 19:12:44 + Gerrit-HasComments: No Gerrit-Has-Labels: No Gerrit-MessageType: comment
Change in asterixdb[master]: [ASTERIXDB-3421][STO] Fix column offset and length on read
>From Wail Alkowaileet :
Wail Alkowaileet has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18360 )
Change subject: [ASTERIXDB-3421][STO] Fix column offset and length on read
..
[ASTERIXDB-3421][STO] Fix column offset and length on read
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Because we read the length of a column (4-bytes),
the remaining number of bytes should account for
those four bytes before reading the remaining pages
Change-Id: I035311613d950bcc73332f82730464cc362b7728
---
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
M
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/impls/lsm/tuples/ColumnMultiBufferProvider.java
3 files changed, 28 insertions(+), 10 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/60/18360/1
diff --git
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
index 7623698..fb3019f 100644
---
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
+++
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudColumnReadContext.java
@@ -154,12 +154,12 @@
// TODO What if every other page is requested. That would do N/2
request, where N is the number of pages.
// TODO This should be optimized in a way that minimizes the number of
requests
-int[] columnsOrders = columnRanges.getColumnsOrder();
+int[] columnsOrder = columnRanges.getColumnsOrder();
int i = 0;
-int columnIndex = columnsOrders[i];
+int columnIndex = columnsOrder[i];
while (columnIndex > -1) {
if (columnIndex < columnRanges.getNumberOfPrimaryKeys()) {
-columnIndex = columnsOrders[++i];
+columnIndex = columnsOrder[++i];
continue;
}
@@ -168,7 +168,7 @@
int lastPageIdx = firstPageIdx +
columnRanges.getColumnNumberOfPages(columnIndex) - 1;
// Advance to the next column to check if it has contiguous pages
-columnIndex = columnsOrders[++i];
+columnIndex = columnsOrder[++i];
while (columnIndex > -1) {
int sharedPageCount = 0;
// Get the next column's start page ID
@@ -183,7 +183,7 @@
lastPageIdx +=
columnRanges.getColumnNumberOfPages(columnIndex) - sharedPageCount;
// Advance to the next column
-columnIndex = columnsOrders[++i];
+columnIndex = columnsOrder[++i];
}
if (lastPageIdx >= columnRanges.getTotalNumberOfPages()) {
diff --git
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
index 21d5ce7..f28b2fd 100644
---
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
+++
b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree-column/src/main/java/org/apache/hyracks/storage/am/lsm/btree/column/cloud/buffercache/read/CloudMegaPageReadContext.java
@@ -116,7 +116,7 @@
* - The operation is not a merge operation (the component will be
deleted anyway)
* - The disk has space
*
- * Note: 'emtpy' can be false while 'cloudOnly is true'. We cannot
read from disk as the page can be
+ * Note: 'empty' can be false while 'cloudOnly is true'. We cannot
read from disk as the page can be
* evicted at any moment. In other words, the sweeper told us that
it is going to evict this page; hence
* 'cloudOnly' is true.
*/
diff --git
Change in asterixdb[trinity]: Merge branch 'gerrit/neo' into 'gerrit/trinity'
>From Michael Blow :
Michael Blow has submitted this change. (
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18359 )
Change subject: Merge branch 'gerrit/neo' into 'gerrit/trinity'
..
Merge branch 'gerrit/neo' into 'gerrit/trinity'
Change-Id: I67f246de4e36b19085525cb4a6afe5a4f9e92212
---
M
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
2 files changed, 14 insertions(+), 5 deletions(-)
Approvals:
Jenkins: Verified; Verified
Michael Blow: Looks good to me, approved
diff --git
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
index eb155fc..b6274d9 100644
---
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
+++
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
@@ -78,13 +78,8 @@
}
@Override
-<<< HEAD (f9681d Merge branch 'gerrit/neo' into 'gerrit/trinity')
public synchronized void notifyJobCreation(JobId jobId, JobSpecification
spec,
IJobCapacityController.JobSubmissionStatus status) throws
HyracksException {
-LOGGER.debug("{} notified of new job {}", getClass().getSimpleName(),
jobId);
-===
-public synchronized void notifyJobCreation(JobId jobId, JobSpecification
spec) throws HyracksException {
->>> BRANCH (3b07af Merge branch 'gerrit/stabilization-40cfb8705b' into
'gerrit/)
if (jobResultLocations.get(jobId) != null) {
throw HyracksDataException.create(ErrorCode.MORE_THAN_ONE_RESULT,
jobId);
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18359
To unsubscribe, or for help writing mail filters, visit
https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: trinity
Gerrit-Change-Id: I67f246de4e36b19085525cb4a6afe5a4f9e92212
Gerrit-Change-Number: 18359
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Blow
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Michael Blow
Gerrit-MessageType: merged
Change in asterixdb[trinity]: Merge branch 'gerrit/neo' into 'gerrit/trinity'
>From Michael Blow : Michael Blow has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18359 ) Change subject: Merge branch 'gerrit/neo' into 'gerrit/trinity' .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18359 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: trinity Gerrit-Change-Id: I67f246de4e36b19085525cb4a6afe5a4f9e92212 Gerrit-Change-Number: 18359 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Blow Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Comment-Date: Mon, 10 Jun 2024 18:25:00 + Gerrit-HasComments: No Gerrit-Has-Labels: Yes Gerrit-MessageType: comment
Change in asterixdb[trinity]: Merge branch 'gerrit/neo' into 'gerrit/trinity'
>From Michael Blow : Attention is currently required from: Michael Blow. Michael Blow has removed a vote from this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18359 ) Change subject: Merge branch 'gerrit/neo' into 'gerrit/trinity' .. Removed Contrib-2 by Unrecognized Gerrit Account 1000171 (1000171) -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18359 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: trinity Gerrit-Change-Id: I67f246de4e36b19085525cb4a6afe5a4f9e92212 Gerrit-Change-Number: 18359 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Blow Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Attention: Michael Blow Gerrit-MessageType: deleteVote
