rdblue commented on code in PR #5096:
URL: https://github.com/apache/iceberg/pull/5096#discussion_r902046602
##########
aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java:
##########
@@ -241,6 +246,52 @@ private List<String> deleteObjectsInBucket(String bucket,
Collection<String> obj
return Lists.newArrayList();
}
+ @Override
+ public Stream<FileInfo> listPrefix(String prefix) {
+ S3URI s3uri = new S3URI(prefix,
awsProperties.s3BucketToAccessPointMapping());
+
+ return internalListPrefix(s3uri.bucket(), s3uri.key()).stream()
+ .flatMap(r -> r.contents().stream())
+ .map(o -> new FileInfo(o.key(), o.size(), o.lastModified()));
Review Comment:
I think this needs to reconstruct a URI rather than returning just the key.
Otherwise, it isn't really a "location" as we think of them in Iceberg. For
example, when listing `"s3://bucket/prefix"`, this might produce
`[FileInfo("prefix/file1.parquet", 10, ...), ...]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
