subject:"\[jira\] \[Commented\] \(HADOOP\-17144\) Update Hadoop's lz4 to v1.9.2"

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2021-04-01 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312973#comment-17312973
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

[~prasad-acit] Native lz4 code was replaced with lz4-java by HADOOP-17292. 
branch-3.3 already has it.

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2021-04-01 Thread Renukaprasad C (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312967#comment-17312967
 ] 

Renukaprasad C commented on HADOOP-17144:
-

Thanks [~hemanthboyina], Same can be backported to branch 3.3 as well ?

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-10-18 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216185#comment-17216185
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

thanks for the review [~iwasakims]

committed to trunk

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-10-11 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212106#comment-17212106
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

+1. The whitespace issue coming from original lz4.c should be left as is.

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-10-07 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209319#comment-17209319
 ] 

Hadoop QA commented on HADOOP-17144:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 33m 
26s{color} |  | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} |  | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} |  | {color:green} The patch does not contain any @author tags. 
{color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} |  | {color:green} The patch appears to include 1 new or modified 
test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
5s{color} |  | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
56s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
59s{color} |  | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 
32s{color} |  | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
46s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 21m  
4s{color} |  | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
39m 24s{color} |  | {color:green} branch has no errors when building and 
testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
28s{color} |  | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
5s{color} |  | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 33m 
28s{color} |  | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 35m 
55s{color} |  | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
21s{color} |  | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
23s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
18s{color} |  | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 20m 18s{color} | 
[/diff-compile-cc-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-HADOOP-Build/86/artifact/out/diff-compile-cc-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt]
 | {color:red} root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 18 new + 154 unchanged - 
9 fixed = 172 total (was 163) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 20m 
18s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m 
18s{color} |  | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 
35s{color} |  | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 17m 35s{color} | 
[/diff-compile-cc-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt|https://ci-hadoop.apache.org/job/PreCommit-HADOOP-Build/86/artifact/out/diff-compile-cc-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt]
 | {color:red} root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with 
JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 32 new + 140 
unchanged - 23 fixed = 172 total (was 163) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green}

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-10-06 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208798#comment-17208798
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

sorry for being late , thanks for the review [~iwasakims]

updated the patch , please review

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-09-17 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198109#comment-17198109
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

{quote}
we do have a test case similar to this scenario in 
TestCompressorDecompressor#testCompressorDecompressorWithExeedBufferLimit , 
modified the lz4 constructors to use default buffer size 
{quote}

[~hemanthboyina] Interestingly the test case does not cover the intended 
scenario. I filed HADOOP-17270 for that. The 004 patch fails on the updated 
test case. Reverting the change below fixed the failure.

{noformat}
@@ -120,7 +121,7 @@ public synchronized void setInput(byte[] b, int off, int 
len) {
* consumed.
*/
   synchronized void setInputFromSavedData() {
-compressedDirectBufLen = Math.min(userBufLen, directBufferSize);
+compressedDirectBufLen = userBufLen;

 // Reinitialize lz4's input direct buffer
 compressedDirectBuf.rewind();
{noformat}


> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-09-13 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195051#comment-17195051
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

thanks for the comment [~iwasakims] , sorry for late response 
{quote}Adding a test case similar to 
TestLz4CompressorDecompressor#testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize
 for decompressor would make the point clear
{quote}
we do have a test case similar to this scenario in 
TestCompressorDecompressor#testCompressorDecompressorWithExeedBufferLimit , 
modified the lz4 constructors to use default buffer size , the compressor 
worked the same way as you have mentioned but decompressor didnt work the same 
as the lz4 decompressor api returned negative value for this scenario which is 
incorrect  

please correct me if i am missing something here

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-09-01 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188872#comment-17188872
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

{quote}
directBufferSize will be the size of raw data , so the compressedDirectBuf and 
UncompressedDirectBuf will have of same size
{quote}

The compressedDirectBuf and uncompressedDirectBuf are working buffer whose size 
can be set regardless of input data size. The buffer size is set to 
EFAULT_DIRECT_BUFFER_SIZE (= 64 * 1024) if the default constructor 
(Lz4Decompressor()) is used. Practically the userBufLen is greater than 
directBufferSize. Adding a test case similar to 
TestLz4CompressorDecompressor#testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize
 for decompressor would make the point clear.


> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-09-01 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188694#comment-17188694
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

{quote}Lz4Decompressor#setInputFromSavedData copies the part of the userBuf to 
compressedDirectBuf (working buffer) considering the capacit
{quote}
directBufferSize will be the size of raw data , so the compressedDirectBuf and 
UncompressedDirectBuf will have of same size
{code:java}
((ByteBuffer) compressedDirectBuf).put(userBuf, userBufOff,
compressedDirectBufLen); {code}

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-25 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184793#comment-17184793
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

{quote}user's compressed data input will be first kept in userBuf and in 
setInputFromSavedData userBuf will be put in compressedDirectBuf , as the 
compressed data length could be greater than source length we need to expanded 
the buffer
{quote}
The userBuf contains whole input data which is not processed in one path. 
Lz4Decompressor#setInputFromSavedData copies the part of the userBuf to 
compressedDirectBuf (working buffer) considering the capacity. User repeatedly 
calls {{decompress}} and {{needsInput}} until all data in the userBuf are 
consumed (then feeds new input by {{setInput}}). We can not expect the 
compressedDirectBuf can hold whole input at a time.

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-25 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184169#comment-17184169
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

thanks for the review [~iwasakims]
{quote}Since user of Lz4Decompressor provides already compressed data as input, 
we do not need to expand the internal buffer?
{quote}
user's compressed data input will be first kept in userBuf and in 
setInputFromSavedData userBuf will be put in compressedDirectBuf , as the 
compressed data length could be greater than source length  we need to  
expanded the buffer
{quote}This looks incorrect since the userBufLen could be greater than 
directBufferSize.
{quote}
In LZ4 decompressor  the userBufLen is nothing but the compressed data length , 
so as the compressed data length could be greater than source length , we need 
to set  compressedDirectBufLen with userBufLen 

LZ4 compression states that  Compression is guaranteed to succeed if 
'dstCapacity' >= LZ4_compressBound(srcSize)
{quote}cc and whitespace warnings should be addressed too

updating the calculation of the maxlength
{quote}
will update on the next patch

 

 

 

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-25 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183989#comment-17183989
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

{noformat}
@@ -185,7 +185,7 @@ void MeasureSingleFileLz4(const string & path, 
CompressResult & total, size_t bl
 size_t currentblocksize = std::min(data.length() - start, blockSize);
 uint64_t startTime = t.now();
 for (int i = 0; i < times; i++) {
-  int osize = LZ4_compress((char*)data.data() + start, outputBuffer, 
currentblocksize);
+  int osize = LZ4_compress_default((char*)data.data() + start, 
outputBuffer, currentblocksize, LZ4_compressBound(currentblocksize));
   result.compressedSize += osize;
   result.uncompressedSize += currentblocksize;
 }
{noformat}

There is no guarantee that LZ4_compressBound(currentblocksize) > 
sizeof(outputBuffer). How about updating the calculation of the maxlength?

{noformat}
  size_t maxlength = std::max((size_t)(blockSize * 1.005), blockSize + 8);
  char * outputBuffer = new char[maxlength];
{noformat}


> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-25 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183976#comment-17183976
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

Thanks for the update, [~hemanthboyina].

{noformat}
@@ -66,11 +66,12 @@
*/
   public Lz4Decompressor(int directBufferSize) {
 this.directBufferSize = directBufferSize;
-
-compressedDirectBuf = ByteBuffer.allocateDirect(directBufferSize);
+// is the size of destination buffer (which must be already allocated),
+// presumed an upper bound of decompressed size
+int dstCapacity = (directBufferSize) + ((directBufferSize) / 255) + 16;
+compressedDirectBuf = ByteBuffer.allocateDirect(dstCapacity);
 uncompressedDirectBuf = ByteBuffer.allocateDirect(directBufferSize);
 uncompressedDirectBuf.position(directBufferSize);
-
   }
{noformat}

Since user of Lz4Decompressor provides already compressed data as input, we do 
not need to expand the internal buffer?

{noformat}
   /**
@@ -120,7 +121,7 @@ public synchronized void setInput(byte[] b, int off, int 
len) {
* consumed.
*/
   synchronized void setInputFromSavedData() {
-compressedDirectBufLen = Math.min(userBufLen, directBufferSize);
+compressedDirectBufLen = userBufLen;
 
 // Reinitialize lz4's input direct buffer
 compressedDirectBuf.rewind();
{noformat}

This looks incorrect since the userBufLen could be greater than 
directBufferSize.

cc and whitespace warnings should be addressed too.

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-23 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182894#comment-17182894
 ] 

Hadoop QA commented on HADOOP-17144:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 
36s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
12s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m  
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
41m 37s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m 
23s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m 
48s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 40m 
35s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 43m 
42s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m 
17s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 29m 17s{color} | 
{color:red} root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 31 new + 140 unchanged - 
22 fixed = 171 total (was 162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 29m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 23m 
58s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 23m 58s{color} | 
{color:red} root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 with 
JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 generated 39 new + 132 
unchanged - 30 fixed = 171 total (was 162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 23m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 23m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m 
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} |

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-23 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182638#comment-17182638
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

thanks for the review [~iwasakims]

updated the patch  by fixing the comments , please review

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch, HADOOP-17144.004.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-19 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180603#comment-17180603
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

{noformat}
@@ -175,12 +175,13 @@ public void testDecompressorCompressAIOBException() {
   public void 
testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize() {
 int BYTES_SIZE = 1024 * 64 + 1;
 try {
-  Lz4Compressor compressor = new Lz4Compressor();
+  Lz4Compressor compressor = new Lz4Compressor(BYTES_SIZE);
{noformat}

The test name implies that the default constructor (Lz4Compressor()) must be 
tested. This should be passed without fix on test code side.


{noformat}
@@ -73,7 +73,7 @@ JNIEXPORT jint JNICALL 
Java_org_apache_hadoop_io_compress_lz4_Lz4Compressor_comp
 return (jint)0;
   }
 
-  compressed_direct_buf_len = LZ4_compress(uncompressed_bytes, 
compressed_bytes, uncompressed_direct_buf_len);
+  compressed_direct_buf_len = LZ4_compress_default(uncompressed_bytes, 
compressed_bytes, uncompressed_direct_buf_len, 
LZ4_compressBound(uncompressed_direct_buf_len));
{noformat}

Actual capacity of destination buffer must be given rather than calling 
LZ4_compressBound? Same can be applied to other invocation of LZ4_compress_*.


> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-18 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179503#comment-17179503
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

thanks for the comment [~iwasakims]

i have verified in CentOS 7, SUSE and in UBUNTU 

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-18 Thread Masatake Iwasaki (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179391#comment-17179391
 ] 

Masatake Iwasaki commented on HADOOP-17144:
---

[~hemanthboyina] I would like to test the patch on OS distribution other than 
Ubuntu (used for QA). I got no compilation error on CentOS 8 at least. Let me 
check the patch and test results then.

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-17 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178798#comment-17178798
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

[~aajisaka] [~iwasakims] can you please review the patch

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-11 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175444#comment-17175444
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

test failures were not related 

please review 

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, 
> HADOOP-17144.003.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-11 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175433#comment-17175433
 ] 

Hadoop QA commented on HADOOP-17144:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
48s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  3m 
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 33m 
 4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m 
10s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
56s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 24m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
45m 10s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
6s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m 
34s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue}  0m 
36s{color} | {color:blue} Used deprecated FindBugs config; considering 
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 45m 
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 23m 
52s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 23m 52s{color} | 
{color:red} root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 41 new + 130 unchanged - 
32 fixed = 171 total (was 162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 23m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 23m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 
18s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 21m 18s{color} | 
{color:red} root-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09 with JDK 
Private Build-1.8.0_252-8u252-b09-1~18.04-b09 generated 30 new + 141 unchanged 
- 21 fixed = 171 total (was 162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 21m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 21m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 
 6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 22m 
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-07 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173539#comment-17173539
 ] 

Hadoop QA commented on HADOOP-17144:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m 
20s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
40s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 20m  
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
36m 34s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
32s{color} | {color:green} trunk passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  
1s{color} | {color:green} trunk passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 31m  
4s{color} | {color:blue} Used deprecated FindBugs config; considering switching 
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 33m 
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m  
3s{color} | {color:green} the patch passed with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 19m  4s{color} | 
{color:red} root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 with JDK 
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 generated 46 new + 125 unchanged - 
37 fixed = 171 total (was 162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 19m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 19m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
35s{color} | {color:green} the patch passed with JDK Private 
Build-1.8.0_252-8u252-b09-1~18.04-b09 {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 16m 35s{color} | 
{color:red} root-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09 with JDK 
Private Build-1.8.0_252-8u252-b09-1~18.04-b09 generated 41 new + 130 unchanged 
- 32 fixed = 171 total (was 162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 16m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m 
48s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-08-06 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172544#comment-17172544
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

LZ4 compression states that  Compression is guaranteed to succeed if 
'dstCapacity' >= LZ4_compressBound(srcSize) , where LZ4_compressBound is
{code:java}
 LZ4_compressBound(isize) = (isize) + ((isize)/255) + 16){code}
 updated the patch as per the above rule and changed the buffer capacity in 
hadoop

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-07-30 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168281#comment-17168281
 ] 

Hadoop QA commented on HADOOP-17144:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  3m 
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 30m 
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 23m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
94m 55s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 16m 24s{color} | 
{color:red} root generated 58 new + 134 unchanged - 28 fixed = 192 total (was 
162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 16m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 17m 
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m 34s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}568m 12s{color} 
| {color:red} root in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  1m 
29s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}734m 28s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.yarn.applications.distributedshell.TestDistributedShell |
|   | 
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
|   | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
|   | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
|   | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
|   | hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR |
|   | hadoop.hdfs.server.balancer.TestBalancer |
|   | hadoop.tools.TestHdfsConfigFields |
|   | hadoop.io.compress.TestCompressorDecompressor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: 
https://ci-hadoop.apache.org/job/PreCommit-HADOOP-Build/9/artifact/out/Dockerfile
 |
| JIRA Issue | HADOOP-17144 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13008102/HADOOP-17144.001.patch
 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit golang |
| uname | Linux 0a2f3f627c3c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / cf4eb756085 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| cc |

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-07-30 Thread Hemanth Boyina (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167804#comment-17167804
 ] 

Hemanth Boyina commented on HADOOP-17144:
-

hi [~cyan4973] [~cnauroth] [~cmccabe] , i was trying to update LZ4 to v1.9.2 , 
i could find some incompatibility by using LZ4_compress in lz4compressor.c , so 
i tried with LZ4_compress_default  , still i could see some related test 
failures 

any suggestions ?

> Update Hadoop's lz4 to v1.9.2
> -
>
> Key: HADOOP-17144
> URL: https://issues.apache.org/jira/browse/HADOOP-17144
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HADOOP-17144.001.patch
>
>
> Update hadoop's native lz4 to v1.9.2 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

2020-07-22 Thread Hadoop QA (Jira)



[ 
https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17162512#comment-17162512
 ] 

Hadoop QA commented on HADOOP-17144:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
 4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 21m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
78m 34s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m 
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 17m 39s{color} | 
{color:red} root generated 55 new + 137 unchanged - 25 fixed = 192 total (was 
162) {color} |
| {color:green}+1{color} | {color:green} golang {color} | {color:green} 17m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m  
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
15m 11s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}621m 29s{color} 
| {color:red} root in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  1m 
29s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}778m 40s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
|   | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy 
|
|   | hadoop.yarn.applications.distributedshell.TestDistributedShell |
|   | hadoop.yarn.service.TestYarnNativeServices |
|   | hadoop.security.TestRaceWhenRelogin |
|   | hadoop.io.compress.TestCompressorDecompressor |
|   | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
|   | hadoop.hdfs.TestGetFileChecksum |
|   | hadoop.tools.TestHdfsConfigFields |
|   | hadoop.hdfs.server.namenode.TestFileTruncate |
|   | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
|   | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
|   | hadoop.yarn.sls.appmaster.TestAMSimulator |
|   | hadoop.yarn.sls.TestReservationSystemInvariants |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/17056/artifact/out/Dockerfile
 |
| JIRA Issue | HADOOP-17144 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/13008102/HADOOP-17144.001.patch
 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit golang |
| uname | Linux f1f43f99fd19

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2

27 matches

Site Navigation

Mail list logo

Footer information