[jira] [Commented] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782061#comment-16782061
]
BELUGA BEHR commented on HADOOP-16148:
--
Hey [~ste...@apache.org]
Since I did the original work and you took the time to look at it, thought I
might as well see it through. All green now. :)
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch, HADOOP-16148.2.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16148:
-
Attachment: HADOOP-16148.2.patch
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch, HADOOP-16148.2.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16148:
-
Status: Patch Available (was: Open)
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch, HADOOP-16148.2.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16148:
-
Status: Open (was: Patch Available)
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch, HADOOP-16148.2.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-16148:
Assignee: BELUGA BEHR
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16148:
-
Attachment: HADOOP-16148.1.patch
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16148) Cleanup LineReader Unit Test
[
https://issues.apache.org/jira/browse/HADOOP-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16148:
-
Status: Patch Available (was: Open)
> Cleanup LineReader Unit Test
>
>
> Key: HADOOP-16148
> URL: https://issues.apache.org/jira/browse/HADOOP-16148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-16148.1.patch
>
>
> I was trying to track down a bug and thought it might be coming from the
> {{LineReader}} class. It wasn't. However, I did clean up the unit test for
> this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16148) Cleanup LineReader Unit Test
BELUGA BEHR created HADOOP-16148:
Summary: Cleanup LineReader Unit Test
Key: HADOOP-16148
URL: https://issues.apache.org/jira/browse/HADOOP-16148
Project: Hadoop Common
Issue Type: Improvement
Components: common
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
I was trying to track down a bug and thought it might be coming from the
{{LineReader}} class. It wasn't. However, I did clean up the unit test for
this class a bit. I figured I might as well at least post the diff file here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762752#comment-16762752
]
BELUGA BEHR commented on HADOOP-16073:
--
Yup. It got a whole bunch of TimeoutExceptions and one:
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hdfs.server.namenode.TestReconstructStripedBlocks.testCountLiveReplicas(TestReconstructStripedBlocks.java:353)
{code}
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762345#comment-16762345
]
BELUGA BEHR edited comment on HADOOP-16073 at 2/7/19 4:22 AM:
--
Or, if you would like some examples of using it beyond consistency:
{code:java}
-secret = sb.toString().getBytes(Charset.forName("UTF-8"));
+secret = sb.toString().getBytes(StandardCharsets.UTF_8);
{code}
In these situations, Java does not have to load the UTF-8 Charset every call,
it can simply use the one that is already defined. Better performance.
{code:java}
- try {
-return new String(bb.array(), bb.arrayOffset() + bb.position(),
-bb.remaining(), "UTF-8");
- } catch (UnsupportedEncodingException e) {
-throw new IllegalCharsetNameException("UTF-8"); // won't happen.
- }
+ return new String(bb.array(), bb.arrayOffset() + bb.position(),
+ bb.remaining(), StandardCharsets.UTF_8);
{code}
Since the JDK is required to provide this implementation, there is no need to
catch "no such character set" exceptions. Leaner and quicker code. And as you
can see, the coder here know that this "won't happen" yet it needs to be coded
for and overhead on try/catch.
was (Author: belugabehr):
Or, if you would like some examples of using it beyond consistency:
{code:java}
-secret = sb.toString().getBytes(Charset.forName("UTF-8"));
+secret = sb.toString().getBytes(StandardCharsets.UTF_8);
{code}
In these situations, Java does not have to load the UTF-8 Charset every call,
it can simply use the one that is already defined. Better performance.
{code:java}
- try {
-return new String(bb.array(), bb.arrayOffset() + bb.position(),
-bb.remaining(), "UTF-8");
- } catch (UnsupportedEncodingException e) {
-throw new IllegalCharsetNameException("UTF-8"); // won't happen.
- }
+ return new String(bb.array(), bb.arrayOffset() + bb.position(),
+ bb.remaining(), StandardCharsets.UTF_8);
{code}
Since the JDK is required to provide this implementation, there is no need to
catch "no such character set" exceptions. Leaner and quicker code.
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762346#comment-16762346
]
BELUGA BEHR commented on HADOOP-16073:
--
[~elgoiri] Ha! Yes! As soon as I typed up that follow-up comment, I realized
what you meant. I was just about to correct myself. You are correct. I was
thinking you were talking about:
{code:java}
import com.google.common.base.Charsets;
{code}
It provides a similar capability to {{StandardCharsets}}
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762345#comment-16762345
]
BELUGA BEHR edited comment on HADOOP-16073 at 2/7/19 4:19 AM:
--
Or, if you would like some examples of using it beyond consistency:
{code:java}
-secret = sb.toString().getBytes(Charset.forName("UTF-8"));
+secret = sb.toString().getBytes(StandardCharsets.UTF_8);
{code}
In these situations, Java does not have to load the UTF-8 Charset every call,
it can simply use the one that is already defined. Better performance.
{code:java}
- try {
-return new String(bb.array(), bb.arrayOffset() + bb.position(),
-bb.remaining(), "UTF-8");
- } catch (UnsupportedEncodingException e) {
-throw new IllegalCharsetNameException("UTF-8"); // won't happen.
- }
+ return new String(bb.array(), bb.arrayOffset() + bb.position(),
+ bb.remaining(), StandardCharsets.UTF_8);
{code}
Since the JDK is required to provide this implementation, there is no need to
catch "no such character set" exceptions. Leaner and quicker code.
was (Author: belugabehr):
Or, if you would like some examples of using it beyond consistency:
{code:java}
-secret = sb.toString().getBytes(Charset.forName("UTF-8"));
+secret = sb.toString().getBytes(StandardCharsets.UTF_8);
{code}
In these situations, Java does not have to load the UTF-8 Charset every call,
it can simply use the one that is already defined.
{code:java}
- try {
-return new String(bb.array(), bb.arrayOffset() + bb.position(),
-bb.remaining(), "UTF-8");
- } catch (UnsupportedEncodingException e) {
-throw new IllegalCharsetNameException("UTF-8"); // won't happen.
- }
+ return new String(bb.array(), bb.arrayOffset() + bb.position(),
+ bb.remaining(), StandardCharsets.UTF_8);
{code}
Since the JDK is required to provide this implementation, there is no need to
catch "no such character set" exceptions. Leaner and quicker code.
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762345#comment-16762345
]
BELUGA BEHR commented on HADOOP-16073:
--
Or, if you would like some examples of using it beyond consistency:
{code:java}
-secret = sb.toString().getBytes(Charset.forName("UTF-8"));
+secret = sb.toString().getBytes(StandardCharsets.UTF_8);
{code}
In these situations, Java does not have to load the UTF-8 Charset every call,
it can simply use the one that is already defined.
{code:java}
- try {
-return new String(bb.array(), bb.arrayOffset() + bb.position(),
-bb.remaining(), "UTF-8");
- } catch (UnsupportedEncodingException e) {
-throw new IllegalCharsetNameException("UTF-8"); // won't happen.
- }
+ return new String(bb.array(), bb.arrayOffset() + bb.position(),
+ bb.remaining(), StandardCharsets.UTF_8);
{code}
Since the JDK is required to provide this implementation, there is no need to
catch "no such character set" exceptions. Leaner and quicker code.
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762339#comment-16762339
]
BELUGA BEHR edited comment on HADOOP-16073 at 2/7/19 4:08 AM:
--
[~elgoiri]
With all due respect, there is no such thing as "regular {{Charset}}". There
is a {{Charset}} class offered by Google's Guava library that existed to fill
this shortcoming of the JDK. However, now that the JDK includes this
{{StandardCharsets}} class, we should be using it consistently across the
project. One could see a situation where the 3rd party library deprecates and
removes this because the JDK has usurped it. Moving everything over would
isolate Hadoop from this and also encourage folks to use the JDK version moving
forward if that's all they see in the code base. There's also a minuscule
memory footprint savings only having to use one instance of the UTF-8
{{Charset}}.
https://google.github.io/guava/releases/19.0/api/docs/com/google/common/base/Charsets.html
was (Author: belugabehr):
[~elgoiri]
With all due respect, there is no such thing as "regular {{Charset}}". There
is a {{Charset}} class offered by Google's Guava library that existed to fill
this shortcoming of the JDK. However, now that the JDK includes this
{{StandardCharsets}} class, we should be using it consistently across the
project. One could see a situation where the 3rd party library deprecates and
removes this because the JDK has usurped it. Moving everything over would
isolate Hadoop from this and also encourage folks to use the JDK version moving
forward if that's all they see in the code base. There's also a minuscule
memory footprint savings only having to use one instance of the UTF-8
{{Charset}}.
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[
https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762339#comment-16762339
]
BELUGA BEHR commented on HADOOP-16073:
--
[~elgoiri]
With all due respect, there is no such thing as "regular {{Charset}}". There
is a {{Charset}} class offered by Google's Guava library that existed to fill
this shortcoming of the JDK. However, now that the JDK includes this
{{StandardCharsets}} class, we should be using it consistently across the
project. One could see a situation where the 3rd party library deprecates and
removes this because the JDK has usurped it. Moving everything over would
isolate Hadoop from this and also encourage folks to use the JDK version moving
forward if that's all they see in the code base. There's also a minuscule
memory footprint savings only having to use one instance of the UTF-8
{{Charset}}.
> Use JDK1.7 StandardCharsets
> ---
>
> Key: HADOOP-16073
> URL: https://issues.apache.org/jira/browse/HADOOP-16073
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Major
> Attachments: HADOOP-16073.1.patch
>
>
> Use Java 1.7
> [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html].
> Every JDK must now include support for several common charsets.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-16073: - Attachment: HADOOP-16073.1.patch > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Major > Attachments: HADOOP-16073.1.patch > > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-16073: - Status: Patch Available (was: Open) > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Major > Attachments: HADOOP-16073.1.patch > > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-16073: - Component/s: (was: tools) (was: streaming) > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Major > Attachments: HADOOP-16073.1.patch > > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-16073: - Status: Open (was: Patch Available) > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR reassigned HADOOP-16073: Assignee: BELUGA BEHR > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762207#comment-16762207 ] BELUGA BEHR commented on HADOOP-16073: -- [~goiri] [~giovanni.fumarola] Either of you gents able to take a look at this? > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761802#comment-16761802 ] BELUGA BEHR commented on HADOOP-16073: -- OK. I cleaned up my branch a little and rebased on the latest trunk. Please consider for inclusion into the project. Since this touches many files and has a high risk of conflicts with future changes, please review promptly. > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752870#comment-16752870 ] BELUGA BEHR commented on HADOOP-16073: -- https://github.com/apache/hadoop/pull/470 > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16073) Use JDK1.7 StandardCharsets
[ https://issues.apache.org/jira/browse/HADOOP-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-16073: - Summary: Use JDK1.7 StandardCharsets (was: Use JDK1.7 StandardCharset) > Use JDK1.7 StandardCharsets > --- > > Key: HADOOP-16073 > URL: https://issues.apache.org/jira/browse/HADOOP-16073 > Project: Hadoop Common > Issue Type: Improvement > Components: streaming, tools >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Priority: Major > > Use Java 1.7 > [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. > Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16073) Use JDK1.7 StandardCharset
BELUGA BEHR created HADOOP-16073: Summary: Use JDK1.7 StandardCharset Key: HADOOP-16073 URL: https://issues.apache.org/jira/browse/HADOOP-16073 Project: Hadoop Common Issue Type: Improvement Components: streaming, tools Affects Versions: 3.2.0 Reporter: BELUGA BEHR Use Java 1.7 [StandardCharsets|https://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html]. Every JDK must now include support for several common charsets. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16067) Incorrect Format Debug Statement KMSACLs
[
https://issues.apache.org/jira/browse/HADOOP-16067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16067:
-
Summary: Incorrect Format Debug Statement KMSACLs (was: Invalid Debug
Statement KMSACLs)
> Incorrect Format Debug Statement KMSACLs
>
>
> Key: HADOOP-16067
> URL: https://issues.apache.org/jira/browse/HADOOP-16067
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Priority: Trivial
> Labels: newbie, noob
>
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.debug("Checking user [{}] for: {}: {}" + ugi.getShortUserName(),
> opType.toString(), acl.getAclString());
> }
> {code}
> The logging message here is incorrect because the first variable is being
> concatenated to the string instead of being passed as an argument.
> {code:java}
> -- Notice the user name 'hdfs' at the end and the spare curly brackets
> 2019-01-23 13:27:45,244 DEBUG
> org.apache.hadoop.crypto.key.kms.server.KMSACLs: Checking user [GENERATE_EEK]
> for: hdfs supergroup: {}hdfs
> {code}
> [https://github.com/apache/hadoop/blob/a55d6bba71c81c1c4e9d8cd11f55c78f10a548b0/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSACLs.java#L313-L316]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16067) Invalid Debug Statement KMSACLs
BELUGA BEHR created HADOOP-16067:
Summary: Invalid Debug Statement KMSACLs
Key: HADOOP-16067
URL: https://issues.apache.org/jira/browse/HADOOP-16067
Project: Hadoop Common
Issue Type: Bug
Components: kms
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
{code:java}
if (LOG.isDebugEnabled()) {
LOG.debug("Checking user [{}] for: {}: {}" + ugi.getShortUserName(),
opType.toString(), acl.getAclString());
}
{code}
The logging message here is incorrect because the first variable is being
concatenated to the string instead of being passed as an argument.
{code:java}
-- Notice the user name 'hdfs' at the end and the spare curly brackets
2019-01-23 13:27:45,244 DEBUG org.apache.hadoop.crypto.key.kms.server.KMSACLs:
Checking user [GENERATE_EEK] for: hdfs supergroup: {}hdfs
{code}
[https://github.com/apache/hadoop/blob/a55d6bba71c81c1c4e9d8cd11f55c78f10a548b0/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSACLs.java#L313-L316]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16022) Increase Compression Buffer Sizes - Remove Magic Numbers
[
https://issues.apache.org/jira/browse/HADOOP-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736256#comment-16736256
]
BELUGA BEHR commented on HADOOP-16022:
--
[~ste...@apache.org] Thanks Steve for the interest.
I looked at the test failures and found the entire setup a bit wonky.
In particular...
[Here|https://github.com/apache/hadoop/blob/7b57f2f71fbaa5af4897309597cca70a95b04edd/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L659]
{code:java|title=TFile.java}
void finishDataBlock(boolean bForceFinish) throws IOException {
...
// exceeded the size limit, do the compression and finish the block
if (bForceFinish || blkAppender.getCompressedSize() >= sizeMinBlock) {
...
{code}
As I understand it:
The general flow of this code is that a bunch of small records are serialized
into bytes and written out to a stream. After a certain threshold of bytes from
the stream have been compressed, the stream is stopped, flushed, and written
out as a single block. Well, the current logic is a bit flawed I believe
because, as we can see here, the block size is based on the size of the
compressed bytes and not the total number of bytes written into the stream.
What is happening here is that as the bytes are written to the stream, they are
first buffered (into the {{BufferedInputStream}} I touched) before being passed
to the compressor. The bytes only make it to the compressor once the buffer has
filled and been forced to flush.
So, in the current implementation, 4K bytes are written to the
{{BufferedInputStream}}, the buffer is flushed, the bytes compressed, the
compressed size reported by {{getCompressedSize()}}, and then flushed out as a
block. When I changed the buffer to 8K, now twice the amount of data is being
buffered before compression and is written to each block. This is very
confusing to say the least... the number of blocks written out are dependent on
the arbitrary size of this {{BufferedInputStream}} returned by the
{{Compression}} class. That is very confusing and hard to test. The person
crafting the unit test must know how big this internal, non-configurable, write
buffer is in order to write an effective test. Also, if we use the default JDK
buffer size (as recommended), these tests may fail depending on the JDK
implementation. I think it is better to change the code to make blocks based on
the number of raw bytes written into the stream, not the number of bytes in its
compressed form. In this way, writing {{n}} bytes will always yield {{y}}
blocks, no matter how big the write buffer is.
Thoughts?
> Increase Compression Buffer Sizes - Remove Magic Numbers
>
>
> Key: HADOOP-16022
> URL: https://issues.apache.org/jira/browse/HADOOP-16022
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
>Affects Versions: 2.10.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-16022.1.patch
>
>
> {code:java|title=Compression.java}
> // data input buffer size to absorb small reads from application.
> private static final int DATA_IBUF_SIZE = 1 * 1024;
> // data output buffer size to absorb small writes from application.
> private static final int DATA_OBUF_SIZE = 4 * 1024;
> {code}
> There exists these hard coded buffer sizes in the Compression code. Instead,
> use the JVM default sizes, which, this day and age, are usually set for 8K.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16022) Increase Compression Buffer Sizes - Remove Magic Numbers
[
https://issues.apache.org/jira/browse/HADOOP-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16022:
-
Status: Patch Available (was: Open)
> Increase Compression Buffer Sizes - Remove Magic Numbers
>
>
> Key: HADOOP-16022
> URL: https://issues.apache.org/jira/browse/HADOOP-16022
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
>Affects Versions: 2.10.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-16022.1.patch
>
>
> {code:java|title=Compression.java}
> // data input buffer size to absorb small reads from application.
> private static final int DATA_IBUF_SIZE = 1 * 1024;
> // data output buffer size to absorb small writes from application.
> private static final int DATA_OBUF_SIZE = 4 * 1024;
> {code}
> There exists these hard coded buffer sizes in the Compression code. Instead,
> use the JVM default sizes, which, this day and age, are usually set for 8K.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16022) Increase Compression Buffer Sizes - Remove Magic Numbers
[
https://issues.apache.org/jira/browse/HADOOP-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-16022:
Assignee: BELUGA BEHR
> Increase Compression Buffer Sizes - Remove Magic Numbers
>
>
> Key: HADOOP-16022
> URL: https://issues.apache.org/jira/browse/HADOOP-16022
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
>Affects Versions: 2.10.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-16022.1.patch
>
>
> {code:java|title=Compression.java}
> // data input buffer size to absorb small reads from application.
> private static final int DATA_IBUF_SIZE = 1 * 1024;
> // data output buffer size to absorb small writes from application.
> private static final int DATA_OBUF_SIZE = 4 * 1024;
> {code}
> There exists these hard coded buffer sizes in the Compression code. Instead,
> use the JVM default sizes, which, this day and age, are usually set for 8K.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16022) Increase Compression Buffer Sizes - Remove Magic Numbers
[
https://issues.apache.org/jira/browse/HADOOP-16022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-16022:
-
Attachment: HADOOP-16022.1.patch
> Increase Compression Buffer Sizes - Remove Magic Numbers
>
>
> Key: HADOOP-16022
> URL: https://issues.apache.org/jira/browse/HADOOP-16022
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
>Affects Versions: 2.10.0, 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-16022.1.patch
>
>
> {code:java|title=Compression.java}
> // data input buffer size to absorb small reads from application.
> private static final int DATA_IBUF_SIZE = 1 * 1024;
> // data output buffer size to absorb small writes from application.
> private static final int DATA_OBUF_SIZE = 4 * 1024;
> {code}
> There exists these hard coded buffer sizes in the Compression code. Instead,
> use the JVM default sizes, which, this day and age, are usually set for 8K.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16022) Increase Compression Buffer Sizes - Remove Magic Numbers
BELUGA BEHR created HADOOP-16022:
Summary: Increase Compression Buffer Sizes - Remove Magic Numbers
Key: HADOOP-16022
URL: https://issues.apache.org/jira/browse/HADOOP-16022
Project: Hadoop Common
Issue Type: Improvement
Components: io
Affects Versions: 2.10.0, 3.2.0
Reporter: BELUGA BEHR
{code:java|title=Compression.java}
// data input buffer size to absorb small reads from application.
private static final int DATA_IBUF_SIZE = 1 * 1024;
// data output buffer size to absorb small writes from application.
private static final int DATA_OBUF_SIZE = 4 * 1024;
{code}
There exists these hard coded buffer sizes in the Compression code. Instead,
use the JVM default sizes, which, this day and age, are usually set for 8K.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15962) FileUtils Small Buffer Size
[
https://issues.apache.org/jira/browse/HADOOP-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15962:
-
Attachment: HADOOP-15962.2.patch
> FileUtils Small Buffer Size
> ---
>
> Key: HADOOP-15962
> URL: https://issues.apache.org/jira/browse/HADOOP-15962
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.3.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15962.1.patch, HADOOP-15962.2.patch
>
>
> Note sure if this code is even being used, but it implements a copy routing
> utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
> there are libraries for this stuff.
> {code:java|title=FileUtil.java}
> int count;
> byte data[] = new byte[2048];
> try (BufferedOutputStream outputStream = new BufferedOutputStream(
> new FileOutputStream(outputFile));) {
> while ((count = tis.read(data)) != -1) {
> outputStream.write(data, 0, count);
> }
> outputStream.flush();
> }
> {code}
> I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15962) FileUtils Small Buffer Size
[
https://issues.apache.org/jira/browse/HADOOP-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15962:
-
Status: Open (was: Patch Available)
> FileUtils Small Buffer Size
> ---
>
> Key: HADOOP-15962
> URL: https://issues.apache.org/jira/browse/HADOOP-15962
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.3.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15962.1.patch, HADOOP-15962.2.patch
>
>
> Note sure if this code is even being used, but it implements a copy routing
> utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
> there are libraries for this stuff.
> {code:java|title=FileUtil.java}
> int count;
> byte data[] = new byte[2048];
> try (BufferedOutputStream outputStream = new BufferedOutputStream(
> new FileOutputStream(outputFile));) {
> while ((count = tis.read(data)) != -1) {
> outputStream.write(data, 0, count);
> }
> outputStream.flush();
> }
> {code}
> I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15962) FileUtils Small Buffer Size
[
https://issues.apache.org/jira/browse/HADOOP-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15962:
-
Status: Patch Available (was: Open)
[~ste...@apache.org] Thanks for the review!
I have removed some changes for easier merging.
> FileUtils Small Buffer Size
> ---
>
> Key: HADOOP-15962
> URL: https://issues.apache.org/jira/browse/HADOOP-15962
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.3.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15962.1.patch, HADOOP-15962.2.patch
>
>
> Note sure if this code is even being used, but it implements a copy routing
> utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
> there are libraries for this stuff.
> {code:java|title=FileUtil.java}
> int count;
> byte data[] = new byte[2048];
> try (BufferedOutputStream outputStream = new BufferedOutputStream(
> new FileOutputStream(outputFile));) {
> while ((count = tis.read(data)) != -1) {
> outputStream.write(data, 0, count);
> }
> outputStream.flush();
> }
> {code}
> I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15962) FileUtils Small Buffer Size
[
https://issues.apache.org/jira/browse/HADOOP-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15962:
-
Status: Patch Available (was: Open)
> FileUtils Small Buffer Size
> ---
>
> Key: HADOOP-15962
> URL: https://issues.apache.org/jira/browse/HADOOP-15962
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.3.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15962.1.patch
>
>
> Note sure if this code is even being used, but it implements a copy routing
> utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
> there are libraries for this stuff.
> {code:java|title=FileUtil.java}
> int count;
> byte data[] = new byte[2048];
> try (BufferedOutputStream outputStream = new BufferedOutputStream(
> new FileOutputStream(outputFile));) {
> while ((count = tis.read(data)) != -1) {
> outputStream.write(data, 0, count);
> }
> outputStream.flush();
> }
> {code}
> I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15962) FileUtils Small Buffer Size
[
https://issues.apache.org/jira/browse/HADOOP-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15962:
-
Attachment: HADOOP-15962.1.patch
> FileUtils Small Buffer Size
> ---
>
> Key: HADOOP-15962
> URL: https://issues.apache.org/jira/browse/HADOOP-15962
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.3.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15962.1.patch
>
>
> Note sure if this code is even being used, but it implements a copy routing
> utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
> there are libraries for this stuff.
> {code:java|title=FileUtil.java}
> int count;
> byte data[] = new byte[2048];
> try (BufferedOutputStream outputStream = new BufferedOutputStream(
> new FileOutputStream(outputFile));) {
> while ((count = tis.read(data)) != -1) {
> outputStream.write(data, 0, count);
> }
> outputStream.flush();
> }
> {code}
> I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15962) FileUtils Small Buffer Size
BELUGA BEHR created HADOOP-15962:
Summary: FileUtils Small Buffer Size
Key: HADOOP-15962
URL: https://issues.apache.org/jira/browse/HADOOP-15962
Project: Hadoop Common
Issue Type: Improvement
Components: common
Affects Versions: 3.3.0
Reporter: BELUGA BEHR
Note sure if this code is even being used, but it implements a copy routing
utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
there are libraries for this stuff.
{code:java|title=FileUtil.java}
int count;
byte data[] = new byte[2048];
try (BufferedOutputStream outputStream = new BufferedOutputStream(
new FileOutputStream(outputFile));) {
while ((count = tis.read(data)) != -1) {
outputStream.write(data, 0, count);
}
outputStream.flush();
}
{code}
I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15962) FileUtils Small Buffer Size
[
https://issues.apache.org/jira/browse/HADOOP-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-15962:
Assignee: BELUGA BEHR
> FileUtils Small Buffer Size
> ---
>
> Key: HADOOP-15962
> URL: https://issues.apache.org/jira/browse/HADOOP-15962
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.3.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
>
> Note sure if this code is even being used, but it implements a copy routing
> utilizing a 2K buffer. Modern JVM uses 8K, but 4K should be minimum. Also,
> there are libraries for this stuff.
> {code:java|title=FileUtil.java}
> int count;
> byte data[] = new byte[2048];
> try (BufferedOutputStream outputStream = new BufferedOutputStream(
> new FileOutputStream(outputFile));) {
> while ((count = tis.read(data)) != -1) {
> outputStream.write(data, 0, count);
> }
> outputStream.flush();
> }
> {code}
> I also fixed a couple of check-style warnings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15836) Review of AccessControlList
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15836:
-
Status: Patch Available (was: Reopened)
OK, so I am providing a new patch with lessons learned from the failed unit
tests and from HADOOP-12640. The two unit tests for MAPREDUCE-7155 and
YARN-8928 pass locally.
I have made one functional change from how things are now:
{code:java}
// The space in the group list is acceptable.
// However, there is no test (and therefore ambiguity)
// about what to do if the user list has a space in it.
// There is only a test for a space in the group list
Iterator iter;
acl = new AccessControlList("drwho,joe tardis, users");
users = acl.getUsers();
assertEquals(users.size(), 2);
iter = users.iterator();
assertEquals(iter.next(), "drwho");
assertEquals(iter.next(), "joe");
groups = acl.getGroups();
assertEquals(groups.size(), 2);
iter = groups.iterator();
assertEquals(iter.next(), "tardis");
assertEquals(iter.next(), "users");
{code}
I have made both situations fail as an invalid format. To avoid any ambiguity,
the format should be strict: {{u1,u2 g1,g2}}
{code:java|title=Patch}
@Test(expected = IllegalArgumentException.class)
public void testSpaceInGroupString() {
// Proper format is: u1,u2 g1,g2
new AccessControlList("drwho,joe tardis, group2");
}
@Test(expected = IllegalArgumentException.class)
public void testSpaceInUSerString() {
// Proper format is: u1,u2 g1,g2
new AccessControlList("drwho, joe tardis,group2");
}
{code}
> Review of AccessControlList
> ---
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15836.1.patch, HADOOP-15836.2.patch,
> assertEqualACLStrings.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15836) Review of AccessControlList
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16705357#comment-16705357
]
BELUGA BEHR edited comment on HADOOP-15836 at 11/30/18 10:25 PM:
-
OK, so I am providing a new patch with lessons learned from the failed unit
tests and from HADOOP-12640. The two unit tests for MAPREDUCE-7155 and
YARN-8928 pass locally.
I have made one functional change from how things are now:
{code:java}
// The space in the group list is acceptable.
// However, there is no test (and therefore ambiguity)
// about what to do if the user list has a space in it.
// There is only a test for a space in the group list
Iterator iter;
acl = new AccessControlList("drwho,joe tardis, users");
users = acl.getUsers();
assertEquals(users.size(), 2);
iter = users.iterator();
assertEquals(iter.next(), "drwho");
assertEquals(iter.next(), "joe");
groups = acl.getGroups();
assertEquals(groups.size(), 2);
iter = groups.iterator();
assertEquals(iter.next(), "tardis");
assertEquals(iter.next(), "users");
{code}
I have made both situations fail as an invalid format. To avoid any ambiguity,
the format should be strict: {{u1,u2g1,g2}}
{code:java|title=Patch}
@Test(expected = IllegalArgumentException.class)
public void testSpaceInGroupString() {
// Proper format is: u1,u2 g1,g2
new AccessControlList("drwho,joe tardis, group2");
}
@Test(expected = IllegalArgumentException.class)
public void testSpaceInUSerString() {
// Proper format is: u1,u2 g1,g2
new AccessControlList("drwho, joe tardis,group2");
}
{code}
was (Author: belugabehr):
OK, so I am providing a new patch with lessons learned from the failed unit
tests and from HADOOP-12640. The two unit tests for MAPREDUCE-7155 and
YARN-8928 pass locally.
I have made one functional change from how things are now:
{code:java}
// The space in the group list is acceptable.
// However, there is no test (and therefore ambiguity)
// about what to do if the user list has a space in it.
// There is only a test for a space in the group list
Iterator iter;
acl = new AccessControlList("drwho,joe tardis, users");
users = acl.getUsers();
assertEquals(users.size(), 2);
iter = users.iterator();
assertEquals(iter.next(), "drwho");
assertEquals(iter.next(), "joe");
groups = acl.getGroups();
assertEquals(groups.size(), 2);
iter = groups.iterator();
assertEquals(iter.next(), "tardis");
assertEquals(iter.next(), "users");
{code}
I have made both situations fail as an invalid format. To avoid any ambiguity,
the format should be strict: {{u1,u2 g1,g2}}
{code:java|title=Patch}
@Test(expected = IllegalArgumentException.class)
public void testSpaceInGroupString() {
// Proper format is: u1,u2 g1,g2
new AccessControlList("drwho,joe tardis, group2");
}
@Test(expected = IllegalArgumentException.class)
public void testSpaceInUSerString() {
// Proper format is: u1,u2 g1,g2
new AccessControlList("drwho, joe tardis,group2");
}
{code}
> Review of AccessControlList
> ---
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15836.1.patch, HADOOP-15836.2.patch,
> assertEqualACLStrings.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15836) Review of AccessControlList
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15836:
-
Attachment: HADOOP-15836.2.patch
> Review of AccessControlList
> ---
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15836.1.patch, HADOOP-15836.2.patch,
> assertEqualACLStrings.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661648#comment-16661648
]
BELUGA BEHR commented on HADOOP-15836:
--
[~elgoiri]
Hello team. I will try this again. However, I had a bit of a spill today and
broke some fingers. I'll get back on this when I'm able.
> Review of AccessControlList
> ---
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15836.1.patch, assertEqualACLStrings.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660697#comment-16660697
]
BELUGA BEHR commented on HADOOP-15836:
--
As we've seen in the unit tests, this class is used to generate ACL strings:
bq. I think it might be more pleasant to a human operator if the list is
ordered alphabetically.
I proposed this change for that reason. Using a {{HashSet}}, the users and
groups are seemingly randomly ordered, which is mildly confusing since this
information is supplied trough a configuration. The output string will likely
not match the input string, so we can at least alphabetize it. People will
understand that ordering and it may be a little easier to troubleshoot. Thanks.
> Review of AccessControlList
> ---
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Status: Patch Available (was: Open) > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch, > HADOOP-12640.1.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Attachment: HADOOP-12640.1.patch > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch, > HADOOP-12640.1.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Attachment: (was: HADOOP-12640.1.patch) > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Status: Open (was: Patch Available) > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659109#comment-16659109
]
BELUGA BEHR commented on HADOOP-15836:
--
[~jlowe] Looking at those now.
> Review of AccessControlList
> ---
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Attachment: HADOOP-12640.1.patch > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch, > HADOOP-12640.1.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Status: Patch Available (was: Open) > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch, > HADOOP-12640.1.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Status: Open (was: Patch Available) > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.7.1 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-12640: - Affects Version/s: (was: 2.7.1) 3.2.0 > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657350#comment-16657350
]
BELUGA BEHR commented on HADOOP-15836:
--
I though about this previously. This information can be read from a config
file and loaded into a {{HashSet}} (as is currently implemented). If the
software needed to spit out the user/group names back to the screen, or back to
a file, the ordering of the output would almost certainly not be the same as
the input (just depends on how the stuff is ordered in the {{HashMap}}). So as
things stand, the software will most likely not build the output in the same
order as the input. Therefore, it appears that there is no constraints on how
the stuff is ordered. If we care about the output matching the input exactly,
that should be a new ticket. Since the order doesn't matter, I think it might
be more pleasant to a human operator if the list is ordered alphabetically.
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15760) Include Apache Commons Collections4
[ https://issues.apache.org/jira/browse/HADOOP-15760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657319#comment-16657319 ] BELUGA BEHR commented on HADOOP-15760: -- Slow and steady. > Include Apache Commons Collections4 > --- > > Key: HADOOP-15760 > URL: https://issues.apache.org/jira/browse/HADOOP-15760 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.10.0, 3.0.3 >Reporter: BELUGA BEHR >Priority: Major > Attachments: HADOOP-15760.1.patch > > > Please allow for use of Apache Commons Collections 4 library with the end > goal of migrating from Apache Commons Collections 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-12640) Code Review AccessControlList
[ https://issues.apache.org/jira/browse/HADOOP-12640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR reassigned HADOOP-12640: Assignee: BELUGA BEHR > Code Review AccessControlList > - > > Key: HADOOP-12640 > URL: https://issues.apache.org/jira/browse/HADOOP-12640 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.7.1 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: AccessControlList.patch, AccessControlList.patch > > > After some confusion of my own, in particular with > "mapreduce.job.acl-view-job," I have looked over the AccessControlList > implementation and cleaned it up and clarified a few points. > 1) I added tests to demonstrate the existing behavior of including an > asterisk in either the username or the group field, it overrides everything > and allows all access. > "user1,user2,user3 *" = all access > "* group1,group2" = all access > "* *" = all access > "* " = all access > " *" = all access > 2) General clean-up and simplification -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657309#comment-16657309
]
BELUGA BEHR commented on HADOOP-15836:
--
... the {{TreeSet}} enforces the user and group names to be ordered
alphabetically, I also changed the unit tests to verify that they are arranged
alphabetically.
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657307#comment-16657307
]
BELUGA BEHR commented on HADOOP-15836:
--
[~elgoiri] Good eyes. All things equal, I'd prefer a {{HashSet}}, however, the
unit tests are written in such a way that they care about the order of the user
and group names. The unit tests are written in such a way that they just so
happen to pass given today's implementation of a {{HashSet}} in the JDK.
However, these assumptions on order makes the tests very brittle and would
possibly fail with a JDK upgrade in which the order was changed based on a
change in the HashSet implementation. Since the unit tests are expecting a
certain order to be enforced, I used a {{TreeSet}} which keeps the keys in
order explicitly, no matter future changes to the JDK.
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656912#comment-16656912
]
BELUGA BEHR commented on HADOOP-15836:
--
[~elgoiri] You see what happens when you are too kind? :) Can you please take
a look at this one also?
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15852:
-
Status: Patch Available (was: Open)
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch, HADOOP-15852.2.patch
>
>
> My new mission is to remove instances of {{StringBuffer}} in favor of
> {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15852:
-
Status: Open (was: Patch Available)
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch, HADOOP-15852.2.patch
>
>
> My new mission is to remove instances of {{StringBuffer}} in favor of
> {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15852:
-
Attachment: HADOOP-15852.2.patch
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch, HADOOP-15852.2.patch
>
>
> My new mission is to remove instances of {{StringBuffer}} in favor of
> {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15854) AuthToken Use StringBuilder instead of StringBuffer
[
https://issues.apache.org/jira/browse/HADOOP-15854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-15854:
Assignee: BELUGA BEHR
> AuthToken Use StringBuilder instead of StringBuffer
> ---
>
> Key: HADOOP-15854
> URL: https://issues.apache.org/jira/browse/HADOOP-15854
> Project: Hadoop Common
> Issue Type: Improvement
> Components: auth
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-15854.1.patch
>
>
> Use {{StringBuilder}} instead of {{StringBuffer}} because {{StringBuilder}}
> is not synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15854) AuthToken Use StringBuilder instead of StringBuffer
[
https://issues.apache.org/jira/browse/HADOOP-15854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15854:
-
Attachment: HADOOP-15854.1.patch
> AuthToken Use StringBuilder instead of StringBuffer
> ---
>
> Key: HADOOP-15854
> URL: https://issues.apache.org/jira/browse/HADOOP-15854
> Project: Hadoop Common
> Issue Type: Improvement
> Components: auth
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-15854.1.patch
>
>
> Use {{StringBuilder}} instead of {{StringBuffer}} because {{StringBuilder}}
> is not synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15854) AuthToken Use StringBuilder instead of StringBuffer
[
https://issues.apache.org/jira/browse/HADOOP-15854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15854:
-
Status: Patch Available (was: Open)
> AuthToken Use StringBuilder instead of StringBuffer
> ---
>
> Key: HADOOP-15854
> URL: https://issues.apache.org/jira/browse/HADOOP-15854
> Project: Hadoop Common
> Issue Type: Improvement
> Components: auth
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-15854.1.patch
>
>
> Use {{StringBuilder}} instead of {{StringBuffer}} because {{StringBuilder}}
> is not synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15854) AuthToken Use StringBuilder instead of StringBuffer
BELUGA BEHR created HADOOP-15854:
Summary: AuthToken Use StringBuilder instead of StringBuffer
Key: HADOOP-15854
URL: https://issues.apache.org/jira/browse/HADOOP-15854
Project: Hadoop Common
Issue Type: Improvement
Components: auth
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
Attachments: HADOOP-15854.1.patch
Use {{StringBuilder}} instead of {{StringBuffer}} because {{StringBuilder}} is
not synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15852:
-
Description:
My new mission is to remove instances of {{StringBuffer}} in favor of
{{StringBuilder}}.
* Simplify Code
* Use Eclipse to generate hashcode/equals
* User StringBuilder instead of StringBuffer
was:
My new mission is to remove {{StringBuffer}}s in favor of {{StringBuilder}}.
* Simplify Code
* Use Eclipse to generate hashcode/equals
* User StringBuilder instead of StringBuffer
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch
>
>
> My new mission is to remove instances of {{StringBuffer}} in favor of
> {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-15852:
Assignee: BELUGA BEHR
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch
>
>
> My new mission is to remove {{StringBuffer}}s in favor of {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15852:
-
Status: Patch Available (was: Open)
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch
>
>
> My new mission is to remove {{StringBuffer}}s in favor of {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15852) QuotaUsage Review
[
https://issues.apache.org/jira/browse/HADOOP-15852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15852:
-
Attachment: HADOOP-15852.1.patch
> QuotaUsage Review
> -
>
> Key: HADOOP-15852
> URL: https://issues.apache.org/jira/browse/HADOOP-15852
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15852.1.patch
>
>
> My new mission is to remove {{StringBuffer}}s in favor of {{StringBuilder}}.
> * Simplify Code
> * Use Eclipse to generate hashcode/equals
> * User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15852) QuotaUsage Review
BELUGA BEHR created HADOOP-15852:
Summary: QuotaUsage Review
Key: HADOOP-15852
URL: https://issues.apache.org/jira/browse/HADOOP-15852
Project: Hadoop Common
Issue Type: Improvement
Components: common
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
Attachments: HADOOP-15852.1.patch
My new mission is to remove {{StringBuffer}}s in favor of {{StringBuilder}}.
* Simplify Code
* Use Eclipse to generate hashcode/equals
* User StringBuilder instead of StringBuffer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650331#comment-16650331
]
BELUGA BEHR commented on HADOOP-15828:
--
[~elgoiri] Please review :)
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch,
> HADOOP-15828.3.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15836:
-
Status: Patch Available (was: Open)
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15836:
-
Attachment: HADOOP-15836.1.patch
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15836.1.patch
>
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15836) Review of AccessControlList.java
BELUGA BEHR created HADOOP-15836:
Summary: Review of AccessControlList.java
Key: HADOOP-15836
URL: https://issues.apache.org/jira/browse/HADOOP-15836
Project: Hadoop Common
Issue Type: Improvement
Components: common, security
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
* Improve unit tests (expected / actual were backwards)
* Unit test expected elements to be in order but the class's return Collections
were unordered
* Formatting cleanup
* Removed superfluous white space
* Remove use of LinkedList
* Removed superfluous code
* Use {{unmodifiable}} Collections where JavaDoc states that caller must not
manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15836) Review of AccessControlList.java
[
https://issues.apache.org/jira/browse/HADOOP-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-15836:
Assignee: BELUGA BEHR
> Review of AccessControlList.java
>
>
> Key: HADOOP-15836
> URL: https://issues.apache.org/jira/browse/HADOOP-15836
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common, security
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
>
> * Improve unit tests (expected / actual were backwards)
> * Unit test expected elements to be in order but the class's return
> Collections were unordered
> * Formatting cleanup
> * Removed superfluous white space
> * Remove use of LinkedList
> * Removed superfluous code
> * Use {{unmodifiable}} Collections where JavaDoc states that caller must not
> manipulate the data structure
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Status: Patch Available (was: Open)
Fixed the unit test to more closely match the return value of the JDK. (JDK
does not return a null value for the affected test, but the test did).
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch,
> HADOOP-15828.3.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Attachment: HADOOP-1528.3.patch
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch,
> HADOOP-15828.3.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Attachment: (was: HADOOP-1528.3.patch)
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch,
> HADOOP-15828.3.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Attachment: HADOOP-15828.3.patch
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch,
> HADOOP-15828.3.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Status: Open (was: Patch Available)
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-15830) Server.java Prefer ArrayList
[
https://issues.apache.org/jira/browse/HADOOP-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642716#comment-16642716
]
BELUGA BEHR edited comment on HADOOP-15830 at 10/9/18 2:42 AM:
---
I just submitted another patch which I was a bit more aggressive on. It has
all the changes before, plus:
# Removed some instances of "log and throw" error handling. This is an
anti-pattern and should be avoided. (try.. catch... log... throw) Log or
throw; do not do both.
# Applied some code formatting to improve readability and check-style of
certain areas
# Removed a lot of dead white space
# Remove logging guards {{LOG.isDebugEnabled()}} in favor of SLF4j parameter
logging
# Removed many instances of logging containing
{{Thread.currentThread().getName()}} to record the thread name performing the
logging. Emitting the thread name can be configured with the logging framework
and does not need to be done explicitly by the caller.
Pick a patch that works for you :)
was (Author: belugabehr):
I just submitted another patch which I was a bit more aggressive on. It has
all the changes before, plus:
# Removed some instances of "log and throw" error handling. This is an
anti-pattern and should be avoided. (try.. catch... log... throw) Log or
throw; do not do both.
# Applied some code formatting to improve readability and check-style of
certain areas
# Removed a lot of dead white space
# Remove logging guards {{LOG.isDebugEnabled()}} in favor of SLF4j parameter
logging
# Removed many instances of logging containing
{{Thread.currentThread().getName()}} to record the thread name performing the
logging. Emitting the thread name can be configured with the logging framework
and does not need to be done explicitly by the caller.
> Server.java Prefer ArrayList
>
>
> Key: HADOOP-15830
> URL: https://issues.apache.org/jira/browse/HADOOP-15830
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15830.2.patch, HDFS-13969.1.patch
>
>
> * Prefer ArrayDeque over LinkedList (faster, less memory overhead)
> * Address this code:
> {code}
> //
> // Remove calls that have been pending in the responseQueue
> // for a long time.
> //
> private void doPurge(RpcCall call, long now) {
> LinkedList responseQueue = call.connection.responseQueue;
> synchronized (responseQueue) {
> Iterator iter = responseQueue.listIterator(0);
> while (iter.hasNext()) {
> call = iter.next();
> if (now > call.timestamp + PURGE_INTERVAL) {
> closeConnection(call.connection);
> break;
> }
> }
> }
> }
> {code}
> It says "Remove calls" (plural) but only one call will be removed because of
> the 'break' statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15830) Server.java Prefer ArrayList
[
https://issues.apache.org/jira/browse/HADOOP-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15830:
-
Status: Patch Available (was: Open)
I just submitted another patch which I was a bit more aggressive on. It has
all the changes before, plus:
# Removed some instances of "log and throw" error handling. This is an
anti-pattern and should be avoided. (try.. catch... log... throw) Log or
throw; do not do both.
# Applied some code formatting to improve readability and check-style of
certain areas
# Removed a lot of dead white space
# Remove logging guards {{LOG.isDebugEnabled()}} in favor of SLF4j parameter
logging
# Removed many instances of logging containing
{{Thread.currentThread().getName()}} to record the thread name performing the
logging. Emitting the thread name can be configured with the logging framework
and does not need to be done explicitly by the caller.
> Server.java Prefer ArrayList
>
>
> Key: HADOOP-15830
> URL: https://issues.apache.org/jira/browse/HADOOP-15830
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15830.2.patch, HDFS-13969.1.patch
>
>
> * Prefer ArrayDeque over LinkedList (faster, less memory overhead)
> * Address this code:
> {code}
> //
> // Remove calls that have been pending in the responseQueue
> // for a long time.
> //
> private void doPurge(RpcCall call, long now) {
> LinkedList responseQueue = call.connection.responseQueue;
> synchronized (responseQueue) {
> Iterator iter = responseQueue.listIterator(0);
> while (iter.hasNext()) {
> call = iter.next();
> if (now > call.timestamp + PURGE_INTERVAL) {
> closeConnection(call.connection);
> break;
> }
> }
> }
> }
> {code}
> It says "Remove calls" (plural) but only one call will be removed because of
> the 'break' statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15830) Server.java Prefer ArrayList
[
https://issues.apache.org/jira/browse/HADOOP-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15830:
-
Attachment: HADOOP-15830.2.patch
> Server.java Prefer ArrayList
>
>
> Key: HADOOP-15830
> URL: https://issues.apache.org/jira/browse/HADOOP-15830
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15830.2.patch, HDFS-13969.1.patch
>
>
> * Prefer ArrayDeque over LinkedList (faster, less memory overhead)
> * Address this code:
> {code}
> //
> // Remove calls that have been pending in the responseQueue
> // for a long time.
> //
> private void doPurge(RpcCall call, long now) {
> LinkedList responseQueue = call.connection.responseQueue;
> synchronized (responseQueue) {
> Iterator iter = responseQueue.listIterator(0);
> while (iter.hasNext()) {
> call = iter.next();
> if (now > call.timestamp + PURGE_INTERVAL) {
> closeConnection(call.connection);
> break;
> }
> }
> }
> }
> {code}
> It says "Remove calls" (plural) but only one call will be removed because of
> the 'break' statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15830) Server.java Prefer ArrayList
[
https://issues.apache.org/jira/browse/HADOOP-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15830:
-
Status: Open (was: Patch Available)
> Server.java Prefer ArrayList
>
>
> Key: HADOOP-15830
> URL: https://issues.apache.org/jira/browse/HADOOP-15830
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HDFS-13969.1.patch
>
>
> * Prefer ArrayDeque over LinkedList (faster, less memory overhead)
> * Address this code:
> {code}
> //
> // Remove calls that have been pending in the responseQueue
> // for a long time.
> //
> private void doPurge(RpcCall call, long now) {
> LinkedList responseQueue = call.connection.responseQueue;
> synchronized (responseQueue) {
> Iterator iter = responseQueue.listIterator(0);
> while (iter.hasNext()) {
> call = iter.next();
> if (now > call.timestamp + PURGE_INTERVAL) {
> closeConnection(call.connection);
> break;
> }
> }
> }
> }
> {code}
> It says "Remove calls" (plural) but only one call will be removed because of
> the 'break' statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Attachment: HADOOP-15828.2.patch
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Status: Patch Available (was: Open)
Fix compilation issue.
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Status: Open (was: Patch Available)
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch, HADOOP-15828.2.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15830) Server.java Prefer ArrayList
[
https://issues.apache.org/jira/browse/HADOOP-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642679#comment-16642679
]
BELUGA BEHR commented on HADOOP-15830:
--
[~elgoiri] Thanks for the look!
Ya, I think the assumption is that if an items in the queue are timestamped as
they are placed into the queue, so in essence, it is sorted. However, I'm not
always sure that is the case.
{code}
// Item goes on the front of the list
call.connection.responseQueue.addFirst(call);
if (inHandler) {
// timestamp is reset
call.timestamp = Time.now();
...
{code}
So in this case, it is actually possible that the item at the front of the list
has the newest timestamp in the queue. I'm not sure in practice if this
happens or if the purge can happen when this is the case, but it would cause
the purge loop to bump out immediately and leave expired calls in the queue.
Regardless, without a priority queue implementation, it seems best to not
assume order.
> Server.java Prefer ArrayList
>
>
> Key: HADOOP-15830
> URL: https://issues.apache.org/jira/browse/HADOOP-15830
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HDFS-13969.1.patch
>
>
> * Prefer ArrayDeque over LinkedList (faster, less memory overhead)
> * Address this code:
> {code}
> //
> // Remove calls that have been pending in the responseQueue
> // for a long time.
> //
> private void doPurge(RpcCall call, long now) {
> LinkedList responseQueue = call.connection.responseQueue;
> synchronized (responseQueue) {
> Iterator iter = responseQueue.listIterator(0);
> while (iter.hasNext()) {
> call = iter.next();
> if (now > call.timestamp + PURGE_INTERVAL) {
> closeConnection(call.connection);
> break;
> }
> }
> }
> }
> {code}
> It says "Remove calls" (plural) but only one call will be removed because of
> the 'break' statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Moved] (HADOOP-15830) Server.java Prefer ArrayList
[
https://issues.apache.org/jira/browse/HADOOP-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR moved HDFS-13969 to HADOOP-15830:
-
Affects Version/s: (was: 3.2.0)
3.2.0
Component/s: (was: ipc)
ipc
Key: HADOOP-15830 (was: HDFS-13969)
Project: Hadoop Common (was: Hadoop HDFS)
> Server.java Prefer ArrayList
>
>
> Key: HADOOP-15830
> URL: https://issues.apache.org/jira/browse/HADOOP-15830
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HDFS-13969.1.patch
>
>
> * Prefer ArrayDeque over LinkedList (faster, less memory overhead)
> * Address this code:
> {code}
> //
> // Remove calls that have been pending in the responseQueue
> // for a long time.
> //
> private void doPurge(RpcCall call, long now) {
> LinkedList responseQueue = call.connection.responseQueue;
> synchronized (responseQueue) {
> Iterator iter = responseQueue.listIterator(0);
> while (iter.hasNext()) {
> call = iter.next();
> if (now > call.timestamp + PURGE_INTERVAL) {
> closeConnection(call.connection);
> break;
> }
> }
> }
> }
> {code}
> It says "Remove calls" (plural) but only one call will be removed because of
> the 'break' statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Moved] (HADOOP-15829) Review of NetgroupCache
[ https://issues.apache.org/jira/browse/HADOOP-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR moved HDFS-13971 to HADOOP-15829: - Affects Version/s: (was: 3.2.0) 3.2.0 Component/s: (was: security) (was: hdfs) security Key: HADOOP-15829 (was: HDFS-13971) Project: Hadoop Common (was: Hadoop HDFS) > Review of NetgroupCache > --- > > Key: HADOOP-15829 > URL: https://issues.apache.org/jira/browse/HADOOP-15829 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.2.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Minor > Attachments: HDFS-13971.1.patch > > > * Simplify code and performance by using Guava Multimap -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR reassigned HADOOP-15828:
Assignee: BELUGA BEHR
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Attachment: HADOOP-15828.1.patch
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15828) Review of MachineList class
BELUGA BEHR created HADOOP-15828:
Summary: Review of MachineList class
Key: HADOOP-15828
URL: https://issues.apache.org/jira/browse/HADOOP-15828
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 3.2.0
Reporter: BELUGA BEHR
Attachments: HADOOP-15828.1.patch
Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
implementation and use empty collections instead of 'null' values, add logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15828) Review of MachineList class
[
https://issues.apache.org/jira/browse/HADOOP-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15828:
-
Status: Patch Available (was: Open)
> Review of MachineList class
> ---
>
> Key: HADOOP-15828
> URL: https://issues.apache.org/jira/browse/HADOOP-15828
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.2.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HADOOP-15828.1.patch
>
>
> Clean up and simplify class {{MachineList}}. Primarily, remove LinkedList
> implementation and use empty collections instead of 'null' values, add
> logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15760) Include Apache Commons Collections4
[ https://issues.apache.org/jira/browse/HADOOP-15760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-15760: - Description: Please allow for use of Apache Commons Collections 4 library with the end goal of migrating from Apache Commons Collections 3. (was: Please allow for use of Apache Commons Collections 4 library with the end goal of migrating from Commons Collects 3.) > Include Apache Commons Collections4 > --- > > Key: HADOOP-15760 > URL: https://issues.apache.org/jira/browse/HADOOP-15760 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.10.0, 3.0.3 >Reporter: BELUGA BEHR >Priority: Major > Attachments: HADOOP-15760.1.patch > > > Please allow for use of Apache Commons Collections 4 library with the end > goal of migrating from Apache Commons Collections 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15760) Include Apache Commons Collections4
[ https://issues.apache.org/jira/browse/HADOOP-15760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-15760: - Status: Patch Available (was: Open) > Include Apache Commons Collections4 > --- > > Key: HADOOP-15760 > URL: https://issues.apache.org/jira/browse/HADOOP-15760 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.3, 2.10.0 >Reporter: BELUGA BEHR >Priority: Major > Attachments: HADOOP-15760.1.patch > > > Please allow for use of Apache Commons Collections 4 library with the end > goal of migrating from Commons Collects 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15760) Include Apache Commons Collections4
BELUGA BEHR created HADOOP-15760: Summary: Include Apache Commons Collections4 Key: HADOOP-15760 URL: https://issues.apache.org/jira/browse/HADOOP-15760 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.3, 2.10.0 Reporter: BELUGA BEHR Attachments: HADOOP-15760.1.patch Please allow for use of Apache Commons Collections 4 library with the end goal of migrating from Commons Collects 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15760) Include Apache Commons Collections4
[ https://issues.apache.org/jira/browse/HADOOP-15760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated HADOOP-15760: - Attachment: HADOOP-15760.1.patch > Include Apache Commons Collections4 > --- > > Key: HADOOP-15760 > URL: https://issues.apache.org/jira/browse/HADOOP-15760 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.10.0, 3.0.3 >Reporter: BELUGA BEHR >Priority: Major > Attachments: HADOOP-15760.1.patch > > > Please allow for use of Apache Commons Collections 4 library with the end > goal of migrating from Commons Collects 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15148) Improve DataOutputByteBuffer
[
https://issues.apache.org/jira/browse/HADOOP-15148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HADOOP-15148:
-
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
The code that this improved has been removed from the project.
> Improve DataOutputByteBuffer
>
>
> Key: HADOOP-15148
> URL: https://issues.apache.org/jira/browse/HADOOP-15148
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HADOOP-15148.1.patch
>
>
> * Use ArrayDeque instead of LinkedList
> * Replace an ArrayList that was being used as a queue with ArrayDeque
> * Improve write single byte method to hard-code sizes and save time
> {quote}
> Resizable-array implementation of the Deque interface. Array deques have no
> capacity restrictions; they grow as necessary to support usage. They are not
> thread-safe; in the absence of external synchronization, they do not support
> concurrent access by multiple threads. Null elements are prohibited. This
> class is *likely to be* ... *faster than LinkedList when used as a queue.*
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
