[jira] [Commented] (HADOOP-18117) Add an option to preserve root directory permissions

2022-02-09 Thread Hongbing Wang (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-18117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489432#comment-17489432
 ] 

Hongbing Wang commented on HADOOP-18117:


Thanks for considering this issue. In fact, we also faced this problem and 
solved it in our internal version as -HADOOP-15211- pointed out. 

Some discussion:
1.  Is it better not to add extra parameters?
2. If add parameters, `updateRootDirectoryAttributes` may be too complicated.

 

> Add an option to preserve root directory permissions
> 
>
> Key: HADOOP-18117
> URL: https://issues.apache.org/jira/browse/HADOOP-18117
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: Mohanad Elsafty
>Priority: Minor
>
> As mentioned in https://issues.apache.org/jira/browse/HADOOP-15211
>  
> If *-update* or *-overwrite* is being passed when *distcp* used, the root 
> directory will be skipped in two occasions (CopyListing#doBuildListing & 
> CopyCommitter#preserveFileAttributesForDirectories), which will ignore root 
> directory's attributes.
>  
> We face the same issue when distcp huge data between clusters and it takes 
> too much effort to update root directories attributes manually.
>  
> From the earlier ticket it's obvious why this behaviour is there, but 
> sometime we need to enforce root directory update hence I will add a new 
> option for distcp to enable someone (who understands the need of this and 
> know what they are doing) to enforce the update of root directory's 
> attributes (permissions, ownership, ...)
>  
> It should be simple one, something like this
> {code:java}
> $ hadoop distcp -p -update -updateRootDirectoryAttributes /a/b/c /a/b/d {code}
> This behaviour is optional and will be *false* by default. (it should not 
> affect existing *distcp* users).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143785#comment-17143785
 ] 

Hongbing Wang commented on HADOOP-17087:


ok, I understand. I agree that no change is best. Thanks [~ayushtkn] for the 
patient guidance. 

> Add EC flag to stat commands
> 
>
> Key: HADOOP-17087
> URL: https://issues.apache.org/jira/browse/HADOOP-17087
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-17087.001.patch
>
>
> We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
> do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
> judge the ec file. 
> So I added ec flag to stat cli.
> old result: 
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt
> regular file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> regular file
> {code}
> new result:
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt 
> erasure coding file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> replica file
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143771#comment-17143771
 ] 

Hongbing Wang commented on HADOOP-17087:


Stat is a comprehensive description of the file. Maybe the ec flag should also 
be added in stat, I think. Do you [~ayushtkn] think it's necessary?

> Add EC flag to stat commands
> 
>
> Key: HADOOP-17087
> URL: https://issues.apache.org/jira/browse/HADOOP-17087
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-17087.001.patch
>
>
> We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
> do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
> judge the ec file. 
> So I added ec flag to stat cli.
> old result: 
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt
> regular file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> regular file
> {code}
> new result:
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt 
> erasure coding file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> replica file
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143769#comment-17143769
 ] 

Hongbing Wang commented on HADOOP-17087:


{quote}

Ls with -e option

{quote}

Sorry i didn't notice this way before. Yahh~ it's good. Thanks [~ayushtkn] 

> Add EC flag to stat commands
> 
>
> Key: HADOOP-17087
> URL: https://issues.apache.org/jira/browse/HADOOP-17087
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-17087.001.patch
>
>
> We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
> do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
> judge the ec file. 
> So I added ec flag to stat cli.
> old result: 
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt
> regular file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> regular file
> {code}
> new result:
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt 
> erasure coding file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> replica file
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongbing Wang updated HADOOP-17087:
---
Attachment: HADOOP-17087.001.patch

> Add EC flag to stat commands
> 
>
> Key: HADOOP-17087
> URL: https://issues.apache.org/jira/browse/HADOOP-17087
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-17087.001.patch
>
>
> We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
> do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
> judge the ec file. 
> So I added ec flag to stat cli.
> old result: 
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt
> regular file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> regular file
> {code}
> new result:
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt 
> erasure coding file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> replica file
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongbing Wang updated HADOOP-17087:
---
Description: 
We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
judge the ec file. 

So I added ec flag to stat cli.

old result: 
{code:java}
$ hadoop fs -stat "%F" /user/ec/ec.txt
regular file
$ hadoop fs -stat "%F" /user/rep/rep.txt 
regular file
{code}
new result:
{code:java}
$ hadoop fs -stat "%F" /user/ec/ec.txt 
erasure coding file
$ hadoop fs -stat "%F" /user/rep/rep.txt 
replica file
{code}
 

  was:
We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
judge the ec file. 

So I added ec flag to stat cli.

old result:

 
{code:java}
$ hadoop fs -stat "%F" /user/ec/ec.txt
regular file
$ hadoop fs -stat "%F" /user/rep/rep.txt 
regular file
{code}
new result:
{code:java}
$ hadoop fs -stat "%F" /user/ec/ec.txt 
erasure coding file
$ hadoop fs -stat "%F" /user/rep/rep.txt 
replica file
{code}
 


> Add EC flag to stat commands
> 
>
> Key: HADOOP-17087
> URL: https://issues.apache.org/jira/browse/HADOOP-17087
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Reporter: Hongbing Wang
>Priority: Major
>
> We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
> do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
> judge the ec file. 
> So I added ec flag to stat cli.
> old result: 
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt
> regular file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> regular file
> {code}
> new result:
> {code:java}
> $ hadoop fs -stat "%F" /user/ec/ec.txt 
> erasure coding file
> $ hadoop fs -stat "%F" /user/rep/rep.txt 
> replica file
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Created] (HADOOP-17087) Add EC flag to stat commands

2020-06-24 Thread Hongbing Wang (Jira)
Hongbing Wang created HADOOP-17087:
--

 Summary: Add EC flag to stat commands
 Key: HADOOP-17087
 URL: https://issues.apache.org/jira/browse/HADOOP-17087
 Project: Hadoop Common
  Issue Type: Improvement
  Components: common
Reporter: Hongbing Wang


We currently do not have a brief way to judge an ec file.  {{hdfs fsck}}  can 
do but shows too much information. Neither {{du}} nor {{ls}} can accurately 
judge the ec file. 

So I added ec flag to stat cli.

old result:

 
{code:java}
$ hadoop fs -stat "%F" /user/ec/ec.txt
regular file
$ hadoop fs -stat "%F" /user/rep/rep.txt 
regular file
{code}
new result:
{code:java}
$ hadoop fs -stat "%F" /user/ec/ec.txt 
erasure coding file
$ hadoop fs -stat "%F" /user/rep/rep.txt 
replica file
{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-21 Thread Hongbing Wang (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17088738#comment-17088738
 ] 

Hongbing Wang commented on HADOOP-16977:


We have disabled kerberos. The username can be passed to YARN by setting the 
env var HADOOP_USER_NAME in the submission service (tomcat) and it takes 
effect. But if we use multiple HADOOP_USER_NAME in the submission service to 
submit the corresponding distcp, the username set with 
`System.setProperty(HADOOP_USER_NAME, value)` will override the previous 
settings. So, this may be a problem. 

> in javaApi, UGI params should be overidden through FileSystem conf
> --
>
> Key: HADOOP-16977
> URL: https://issues.apache.org/jira/browse/HADOOP-16977
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.7.2, 3.2.0
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-16977.001.patch, HADOOP-16977.002.patch
>
>
> org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
> get the configure from the configuration files. Like below:
> {code:java}
> private static void ensureInitialized() {
>   if (conf == null) {
> synchronized(UserGroupInformation.class) {
>   if (conf == null) { // someone might have beat us
> initialize(new Configuration(), false);
>   }
> }
>   }
> }{code}
> So that, if FileSystem is created through FileSystem#get or 
> FileSystem#newInstance with conf, the conf values different from the 
> configuration files will not take effect in UserGroupInformation.  E.g:
> {code:java}
> Configuration conf = new Configuration();
> conf.set("k1","v1");
> conf.set("k2","v2");
> FileSystem fs = FileSystem.get(uri, conf);{code}
> "k1" or "k2" will not work in UserGroupInformation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-18 Thread Hongbing Wang (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086367#comment-17086367
 ] 

Hongbing Wang commented on HADOOP-16977:


Thank [~ste...@apache.org] , thank you for your detailed answers and 
suggestions. I think it should be fixed because our internal version uses 
`hadoop.user.name` for  authentication, but it cannot be passed to 
distcp.execute (). We can choose `System.setProperty(k,v)` to solve the 
problem, but system variables will affect Global. Now, I try to modify from the 
business code. Thank you once again!

> in javaApi, UGI params should be overidden through FileSystem conf
> --
>
> Key: HADOOP-16977
> URL: https://issues.apache.org/jira/browse/HADOOP-16977
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.7.2, 3.2.0
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-16977.001.patch, HADOOP-16977.002.patch
>
>
> org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
> get the configure from the configuration files. Like below:
> {code:java}
> private static void ensureInitialized() {
>   if (conf == null) {
> synchronized(UserGroupInformation.class) {
>   if (conf == null) { // someone might have beat us
> initialize(new Configuration(), false);
>   }
> }
>   }
> }{code}
> So that, if FileSystem is created through FileSystem#get or 
> FileSystem#newInstance with conf, the conf values different from the 
> configuration files will not take effect in UserGroupInformation.  E.g:
> {code:java}
> Configuration conf = new Configuration();
> conf.set("k1","v1");
> conf.set("k2","v2");
> FileSystem fs = FileSystem.get(uri, conf);{code}
> "k1" or "k2" will not work in UserGroupInformation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-14 Thread Hongbing Wang (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongbing Wang updated HADOOP-16977:
---
Attachment: HADOOP-16977.002.patch

> in javaApi, UGI params should be overidden through FileSystem conf
> --
>
> Key: HADOOP-16977
> URL: https://issues.apache.org/jira/browse/HADOOP-16977
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 2.7.2, 3.2.0
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-16977.001.patch, HADOOP-16977.002.patch
>
>
> org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
> get the configure from the configuration files. Like below:
> {code:java}
> private static void ensureInitialized() {
>   if (conf == null) {
> synchronized(UserGroupInformation.class) {
>   if (conf == null) { // someone might have beat us
> initialize(new Configuration(), false);
>   }
> }
>   }
> }{code}
> So that, if FileSystem is created through FileSystem#get or 
> FileSystem#newInstance with conf, the conf values different from the 
> configuration files will not take effect in UserGroupInformation.  E.g:
> {code:java}
> Configuration conf = new Configuration();
> conf.set("k1","v1");
> conf.set("k2","v2");
> FileSystem fs = FileSystem.get(uri, conf);{code}
> "k1" or "k2" will not work in UserGroupInformation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-12 Thread Hongbing Wang (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongbing Wang updated HADOOP-16977:
---
Attachment: HADOOP-16977.001.patch

> in javaApi, UGI params should be overidden through FileSystem conf
> --
>
> Key: HADOOP-16977
> URL: https://issues.apache.org/jira/browse/HADOOP-16977
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 2.7.2, 3.2.0
>Reporter: Hongbing Wang
>Priority: Major
> Attachments: HADOOP-16977.001.patch
>
>
> org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
> get the configure from the configuration files. Like below:
> {code:java}
> private static void ensureInitialized() {
>   if (conf == null) {
> synchronized(UserGroupInformation.class) {
>   if (conf == null) { // someone might have beat us
> initialize(new Configuration(), false);
>   }
> }
>   }
> }{code}
> So that, if FileSystem is created through FileSystem#get or 
> FileSystem#newInstance with conf, the conf values different from the 
> configuration files will not take effect in UserGroupInformation.  E.g:
> {code:java}
> Configuration conf = new Configuration();
> conf.set("k1","v1");
> conf.set("k2","v2");
> FileSystem fs = FileSystem.get(uri, conf);{code}
> "k1" or "k2" will not work in UserGroupInformation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-12 Thread Hongbing Wang (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongbing Wang updated HADOOP-16977:
---
Description: 
org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
get the configure from the configuration files. Like below:
{code:java}
private static void ensureInitialized() {
  if (conf == null) {
synchronized(UserGroupInformation.class) {
  if (conf == null) { // someone might have beat us
initialize(new Configuration(), false);
  }
}
  }
}{code}
So that, if FileSystem is created through FileSystem#get or 
FileSystem#newInstance with conf, the conf values different from the 
configuration files will not take effect in UserGroupInformation.  E.g:
{code:java}
Configuration conf = new Configuration();
conf.set("k1","v1");
conf.set("k2","v2");
FileSystem fs = FileSystem.get(uri, conf);{code}
"k1" or "k2" will not work in UserGroupInformation.

  was:
org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
get the configure from the configuration files. Like below:
{code:java}
private static void ensureInitialized() {
  if (conf == null) {
synchronized(UserGroupInformation.class) {
  if (conf == null) { // someone might have beat us
initialize(new Configuration(), false);
  }
}
  }
}{code}
So that, if FileSystem is created through FileSystem#get or 
FileSystem#newInstance with conf*, the conf values different from the 
configuration files will not take effect in UserGroupInformation.* E.g:
{code:java}
Configuration conf = new Configuration();
conf.set("k1","v1");
conf.set("k2","v2");
FileSystem fs = FileSystem.get(uri, conf);{code}
"k1" or "k2" will not work in UserGroupInformation.


> in javaApi, UGI params should be overidden through FileSystem conf
> --
>
> Key: HADOOP-16977
> URL: https://issues.apache.org/jira/browse/HADOOP-16977
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 2.7.2, 3.2.0
>Reporter: Hongbing Wang
>Priority: Major
>
> org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
> get the configure from the configuration files. Like below:
> {code:java}
> private static void ensureInitialized() {
>   if (conf == null) {
> synchronized(UserGroupInformation.class) {
>   if (conf == null) { // someone might have beat us
> initialize(new Configuration(), false);
>   }
> }
>   }
> }{code}
> So that, if FileSystem is created through FileSystem#get or 
> FileSystem#newInstance with conf, the conf values different from the 
> configuration files will not take effect in UserGroupInformation.  E.g:
> {code:java}
> Configuration conf = new Configuration();
> conf.set("k1","v1");
> conf.set("k2","v2");
> FileSystem fs = FileSystem.get(uri, conf);{code}
> "k1" or "k2" will not work in UserGroupInformation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Updated] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-12 Thread Hongbing Wang (Jira)


 [ 
https://issues.apache.org/jira/browse/HADOOP-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongbing Wang updated HADOOP-16977:
---
Description: 
org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
get the configure from the configuration files. Like below:
{code:java}
private static void ensureInitialized() {
  if (conf == null) {
synchronized(UserGroupInformation.class) {
  if (conf == null) { // someone might have beat us
initialize(new Configuration(), false);
  }
}
  }
}{code}
So that, if FileSystem is created through FileSystem#get or 
FileSystem#newInstance with conf*, the conf values different from the 
configuration files will not take effect in UserGroupInformation.* E.g:
{code:java}
Configuration conf = new Configuration();
conf.set("k1","v1");
conf.set("k2","v2");
FileSystem fs = FileSystem.get(uri, conf);{code}
"k1" or "k2" will not work in UserGroupInformation.

> in javaApi, UGI params should be overidden through FileSystem conf
> --
>
> Key: HADOOP-16977
> URL: https://issues.apache.org/jira/browse/HADOOP-16977
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 2.7.2, 3.2.0
>Reporter: Hongbing Wang
>Priority: Major
>
> org.apache.hadoop.security.UserGroupInformation#ensureInitialized,will always 
> get the configure from the configuration files. Like below:
> {code:java}
> private static void ensureInitialized() {
>   if (conf == null) {
> synchronized(UserGroupInformation.class) {
>   if (conf == null) { // someone might have beat us
> initialize(new Configuration(), false);
>   }
> }
>   }
> }{code}
> So that, if FileSystem is created through FileSystem#get or 
> FileSystem#newInstance with conf*, the conf values different from the 
> configuration files will not take effect in UserGroupInformation.* E.g:
> {code:java}
> Configuration conf = new Configuration();
> conf.set("k1","v1");
> conf.set("k2","v2");
> FileSystem fs = FileSystem.get(uri, conf);{code}
> "k1" or "k2" will not work in UserGroupInformation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Created] (HADOOP-16977) in javaApi, UGI params should be overidden through FileSystem conf

2020-04-12 Thread Hongbing Wang (Jira)
Hongbing Wang created HADOOP-16977:
--

 Summary: in javaApi, UGI params should be overidden through 
FileSystem conf
 Key: HADOOP-16977
 URL: https://issues.apache.org/jira/browse/HADOOP-16977
 Project: Hadoop Common
  Issue Type: Bug
  Components: common
Affects Versions: 3.2.0, 2.7.2
Reporter: Hongbing Wang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org