saxenapranav commented on code in PR #6270:
URL: https://github.com/apache/hadoop/pull/6270#discussion_r1401789516
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:
##########
@@ -811,13 +813,17 @@ private AbfsInputStreamContext
populateAbfsInputStreamContext(
boolean bufferedPreadDisabled = options
.map(c -> c.getBoolean(FS_AZURE_BUFFERED_PREAD_DISABLE, false))
.orElse(false);
+ int footerReadBufferSize = options.map(c -> c.getInt(
+ AZURE_FOOTER_READ_BUFFER_SIZE, DEFAULT_FOOTER_READ_BUFFER_SIZE))
Review Comment:
In case if dev goes with builder route and have not given
fs.azure.footer.read.request.size in builder opts, it will take the default
value even though in config a value is defined. Better in place of default
value, have the abfsConfugration.getFooterReadBufferSize().
How can you verify this happens, change bit of your `testNumBackendCalls`
code:
````
private void testNumBackendCalls(boolean optimizeFooterRead)
throws Exception {
for (int i = 0; i <= 4; i++) {
for (int j = 0; j <= 2; j++) {
int fileSize = (int) Math.pow(2, i) * 256 * ONE_KB;
int footerReadBufferSize = (int) Math.pow(2, j) * 256 * ONE_KB;
final AzureBlobFileSystem fs = getFileSystem(optimizeFooterRead,
fileSize, footerReadBufferSize);
String fileName = methodName.getMethodName() + i;
byte[] fileContent = getRandomBytesArray(fileSize);
Path testFilePath = createFileWithContent(fs, fileName, fileContent);
int length = AbfsInputStream.FOOTER_SIZE;
FutureDataInputStreamBuilder builder = fs.openFile(testFilePath);
// builder.opt(ConfigurationKeys.AZURE_FOOTER_READ_BUFFER_SIZE,
// footerReadBufferSize);
try (FSDataInputStream iStream = builder.build().get()) {
......
```
Lets add this usecase as well in tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
