>From Wail Alkowaileet <[email protected]>: Attention is currently required from: Savyasach Reddy. Wail Alkowaileet has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253 )
Change subject: Add GCS Cloud Client ...................................................................... Patch Set 6: (8 comments) File asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSBufferedWriter.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/d5a89840_c703fed7 PS6, Line 46: WriteChannel It looks GCS has a more convenient way to write than S3. Maybe we need to rethink this. For example, we can have an abstraction to CloudResettableInputStream. See upload() https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/b297dfbd_1d97a392 PS6, Line 60: readNBytes This will create a byte[] containing the data in 'stream'. That means a byte[] with 5MB length will be created each time we upload. We need to rethink this. I'll try to redesign our way to do cloud write. https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/73a688e6_d8b6a79e PS6, Line 102: partNumber Why you need this? https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/1e937bc1_fc5419c3 PS6, Line 104: setChunkSize what's the minimum/maximum for this? File asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSClientConfig.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/70d55235_43cf2df2 PS6, Line 35: DELETE_BATCH_SIZE Hmm that's 10x lower than S3. Did you verify that? File asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/8b9839a7_e6b4c628 PS6, Line 100: list Should that also include BlobListOption.fields(Storage.BlobField.SIZE)? https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/9b0214e3_46553759 PS6, Line 121: seek What's the cost of this seek? Just curious if that actually will bring anything from the cloud and then do seek. We've seen so much from S3 I become super suspicious 😊 https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253/comment/b836f018_20486ac3 PS6, Line 187: DELETE_BATCH_SIZE We probably should include a URL of the documentation that states the limit. -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18253 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I0b9cad99de2d32d1e672a3d396897807687685e8 Gerrit-Change-Number: 18253 Gerrit-PatchSet: 6 Gerrit-Owner: Savyasach Reddy <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-CC: Murtadha Hubail <[email protected]> Gerrit-CC: Wail Alkowaileet <[email protected]> Gerrit-Attention: Savyasach Reddy <[email protected]> Gerrit-Comment-Date: Thu, 25 Apr 2024 23:22:42 +0000 Gerrit-HasComments: Yes Gerrit-Has-Labels: No Gerrit-MessageType: comment
