[
https://issues.apache.org/jira/browse/HADOOP-17404?focusedWorklogId=522566&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-522566
]
ASF GitHub Bot logged work on HADOOP-17404:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 10/Dec/20 05:54
Start Date: 10/Dec/20 05:54
Worklog Time Spent: 10m
Work Description: snvijaya edited a comment on pull request #2509:
URL: https://github.com/apache/hadoop/pull/2509#issuecomment-742230317
Test results from East US2 accounts:
HNS-OAuth
========================
[INFO] Results:
[INFO]
[INFO] Tests run: 90, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 460, Failures: 0, Errors: 0, Skipped: 68
[INFO] Results:
[INFO]
[WARNING] Tests run: 256, Failures: 0, Errors: 0, Skipped: 165
HNS-SharedKey
========================
[INFO] Results:
[INFO]
[INFO] Tests run: 90, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 460, Failures: 0, Errors: 0, Skipped: 24
[INFO] Results:
[INFO]
[WARNING] Tests run: 256, Failures: 0, Errors: 0, Skipped: 48
NonHNS-OAuth
========================
[INFO] Results:
[INFO]
[INFO] Tests run: 90, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 460, Failures: 0, Errors: 0, Skipped: 251
[INFO] Results:
[INFO]
[WARNING] Tests run: 256, Failures: 0, Errors: 0, Skipped: 165
NonHNS-SharedKey
========================
[INFO] Results:
[INFO]
[INFO] Tests run: 90, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 460, Failures: 0, Errors: 0, Skipped: 247
[INFO] Results:
[INFO]
[WARNING] Tests run: 256, Failures: 0, Errors: 0, Skipped: 48
HNS-AppendBlob
========================
[INFO] Results:
[INFO]
[INFO] Tests run: 90, Failures: 0, Errors: 0, Skipped: 0
[INFO] Results:
[INFO]
[WARNING] Tests run: 460, Failures: 0, Errors: 0, Skipped: 68
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]
ITestAzureBlobFileSystemE2EScale.testWriteHeavyBytesToFileAcrossThreads:77 »
TestTimedOut
[INFO]
[ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 189
Re-ran the failed testclass :
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
727.959 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
### Test result from Canary region With Optimization turned On:
(Server support is available only in Canary).
[INFO] Tests run: 90, Failures: 0, Errors: 0, Skipped: 0
[WARNING] Tests run: 460, Failures: 0, Errors: 0, Skipped: 68
[WARNING] Tests run: 256, Failures: 0, Errors: 0, Skipped: 141
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 522566)
Time Spent: 1h 20m (was: 1h 10m)
> ABFS: Piggyback flush on Append calls for short writes
> ------------------------------------------------------
>
> Key: HADOOP-17404
> URL: https://issues.apache.org/jira/browse/HADOOP-17404
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.0
> Reporter: Sneha Vijayarajan
> Assignee: Sneha Vijayarajan
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When Hflush or Hsync APIs are called, a call is made to store backend to
> commit the data that was appended.
> If the data size written by Hadoop app is small, i.e. data size :
> * before any of HFlush/HSync call is made or
> * between 2 HFlush/Hsync API calls
> is less than write buffer size, 2 separate calls, one for append and another
> for flush is made,
> Apps that do such small writes eventually end up with almost similar number
> of calls for flush and append.
> This PR enables Flush to be piggybacked onto append call for such short write
> scenarios.
>
> NOTE: The changes is guarded over a config, and is disabled by default until
> relevant supported changes is made available on all store production clusters.
> New Config added: fs.azure.write.enableappendwithflush
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
