[jira] [Commented] (YARN-10320) Replace FSDataInputStream#read with readFully in Log Aggregation

2020-07-23 Thread Bilwa S T (Jira)


[ 
https://issues.apache.org/jira/browse/YARN-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163817#comment-17163817
 ] 

Bilwa S T commented on YARN-10320:
--

Thanks for patch [~tanu.ajmera]

I think you need to replace read with readFully in below code too
{code:java}
 while ((len = in.read(buf)) != -1) {
  //If buffer contents within fileLength, write
  if (len < bytesLeft) {
outputStreamState.getOutputStream().write(buf, 0, len);
bytesLeft-=len;
  } else {
//else only write contents within fileLength, then exit early
outputStreamState.getOutputStream().write(buf, 0,
(int)bytesLeft);
break;
  }
}
{code}
 

> Replace FSDataInputStream#read with readFully in Log Aggregation
> 
>
> Key: YARN-10320
> URL: https://issues.apache.org/jira/browse/YARN-10320
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: log-aggregation
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-10320-001.patch, YARN-10320-002.patch
>
>
> Have observed Log Aggregation code has used FSDataInputStream#read instead of 
> readFully in multiple places like below. One of the place is fixed by 
> YARN-8106.
> This Jira targets to fix at all other places.
> LogAggregationIndexedFileController#loadUUIDFromLogFile
> {code}
>   byte[] b = new byte[uuid.length];
>   int actual = fsDataInputStream.read(b);
>   if (actual != uuid.length || Arrays.equals(b, uuid)) {
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org



[jira] [Commented] (YARN-10320) Replace FSDataInputStream#read with readFully in Log Aggregation

2020-06-30 Thread Hadoop QA (Jira)


[ 
https://issues.apache.org/jira/browse/YARN-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148497#comment-17148497
 ] 

Hadoop QA commented on YARN-10320:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  2m 
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
17m  6s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  1m 
51s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
48s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 20s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m 
52s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
32s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m 59s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: 
https://builds.apache.org/job/PreCommit-YARN-Build/26235/artifact/out/Dockerfile
 |
| JIRA Issue | YARN-10320 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13006730/YARN-10320-002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite 
unit shadedclient findbugs checkstyle |
| uname | Linux 81d9ae82fa42 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 
10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / cd188ea9f0e |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-YARN-Build/26235/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: 

[jira] [Commented] (YARN-10320) Replace FSDataInputStream#read with readFully in Log Aggregation

2020-06-30 Thread Hadoop QA (Jira)


[ 
https://issues.apache.org/jira/browse/YARN-10320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148335#comment-17148335
 ] 

Hadoop QA commented on YARN-10320:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m 13s{color} 
| {color:red} YARN-10320 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-10320 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13006724/YARN-10320-001.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-YARN-Build/26233/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |


This message was automatically generated.



> Replace FSDataInputStream#read with readFully in Log Aggregation
> 
>
> Key: YARN-10320
> URL: https://issues.apache.org/jira/browse/YARN-10320
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: log-aggregation
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Tanu Ajmera
>Priority: Major
> Attachments: YARN-10320-001.patch
>
>
> Have observed Log Aggregation code has used FSDataInputStream#read instead of 
> readFully in multiple places like below. One of the place is fixed by 
> YARN-8106.
> This Jira targets to fix at all other places.
> LogAggregationIndexedFileController#loadUUIDFromLogFile
> {code}
>   byte[] b = new byte[uuid.length];
>   int actual = fsDataInputStream.read(b);
>   if (actual != uuid.length || Arrays.equals(b, uuid)) {
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org