>From Murtadha Hubail <[email protected]>: Murtadha Hubail has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19134 )
Change subject: [ASTERIXDB-3503][EXT] Enhanced Error Handling for Unsupported External Sources in case of delta table format. ...................................................................... [ASTERIXDB-3503][EXT] Enhanced Error Handling for Unsupported External Sources in case of delta table format. - user model changes: yes - storage format changes: no - interface changes: no Details: - Renamed the configurationBuilder method in AwsS3DeltaReaderFactory to applyConfiguration. - Added a validation check in the validateDeltaTableExists method to ensure the external source type is supported for delta table format. Ext-ref: MB-64314 Change-Id: I350ba8ceddff554c2b0407449ce00b336715f92d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19134 Integration-Tests: Jenkins <[email protected]> Tested-by: Jenkins <[email protected]> Reviewed-by: Ayush Tripathi <[email protected]> Reviewed-by: Murtadha Hubail <[email protected]> --- M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/delta/AwsS3DeltaReaderFactory.java M asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java 2 files changed, 33 insertions(+), 3 deletions(-) Approvals: Murtadha Hubail: Looks good to me, approved Ayush Tripathi: Looks good to me, but someone else must approve Jenkins: Verified; Verified Objections: Anon. E. Moose #1000171: Violations found diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/delta/AwsS3DeltaReaderFactory.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/delta/AwsS3DeltaReaderFactory.java index 8dc820b..d29cd40 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/delta/AwsS3DeltaReaderFactory.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/input/record/reader/aws/delta/AwsS3DeltaReaderFactory.java @@ -91,7 +91,7 @@ throws AlgebricksException, HyracksDataException { this.configuration = configuration; Configuration conf = new Configuration(); - configurationBuilder(configuration, conf); + applyConfiguration(configuration, conf); String tableMetadataPath = S3Constants.HADOOP_S3_PROTOCOL + "://" + configuration.get(ExternalDataConstants.CONTAINER_NAME_FIELD_NAME) + '/' + configuration.get(ExternalDataConstants.DEFINITION_FIELD_NAME); @@ -189,7 +189,7 @@ partitionWorkLoadsBasedOnSize.addAll(workloadQueue); } - public static void configurationBuilder(Map<String, String> configuration, Configuration conf) { + public static void applyConfiguration(Map<String, String> configuration, Configuration conf) { conf.set(S3Constants.HADOOP_ACCESS_KEY_ID, configuration.get(S3Constants.ACCESS_KEY_ID_FIELD_NAME)); conf.set(S3Constants.HADOOP_SECRET_ACCESS_KEY, configuration.get(S3Constants.SECRET_ACCESS_KEY_FIELD_NAME)); if (configuration.get(S3Constants.SESSION_TOKEN_FIELD_NAME) != null) { diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java index 19c1979..12ce22d 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java @@ -519,11 +519,16 @@ String tableMetadataPath = null; if (configuration.get(ExternalDataConstants.KEY_EXTERNAL_SOURCE_TYPE) .equals(ExternalDataConstants.KEY_ADAPTER_NAME_AWS_S3)) { - AwsS3DeltaReaderFactory.configurationBuilder(configuration, conf); + AwsS3DeltaReaderFactory.applyConfiguration(configuration, conf); tableMetadataPath = S3Constants.HADOOP_S3_PROTOCOL + "://" + configuration.get(ExternalDataConstants.CONTAINER_NAME_FIELD_NAME) + '/' + configuration.get(ExternalDataConstants.DEFINITION_FIELD_NAME); + } else { + throw new CompilationException(ErrorCode.EXTERNAL_SOURCE_ERROR, + "Delta format is not supported for the external source type: " + + configuration.get(ExternalDataConstants.KEY_EXTERNAL_SOURCE_TYPE)); } + Engine engine = DefaultEngine.create(conf); io.delta.kernel.Table table = io.delta.kernel.Table.forPath(engine, tableMetadataPath); try { -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19134 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: goldfish Gerrit-Change-Id: I350ba8ceddff554c2b0407449ce00b336715f92d Gerrit-Change-Number: 19134 Gerrit-PatchSet: 4 Gerrit-Owner: Ayush Tripathi <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ayush Tripathi <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-MessageType: merged
