[jira] [Created] (HADOOP-17157) S3A rename operation not the same with HDFS
Jiajia Li created HADOOP-17157:
--
Summary: S3A rename operation not the same with HDFS
Key: HADOOP-17157
URL: https://issues.apache.org/jira/browse/HADOOP-17157
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Reporter: Jiajia Li
When I run the test ITestS3ADeleteManyFiles, I change the
[https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/scale/ITestS3ADeleteManyFiles.java#L97]
to
{code}
fs.mkdirs(finalDir);
{code}
So before rename operator, "finalParent/final" has been created.
But after the rename operation, all the files will be moved from
"srcParent/src" to "finalParent/final"
So this is not the same with the HDFS rename operation:
HDFS rename includes the calculation of the destination path. If the
destination exists and is a directory, the final destination of the rename
becomes the destination + the filename of the source path.
let dest = if (isDir(FS, src) and d != src) :
d + [filename(src)]
else :
d
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15072) Upgrade Apache Kerby version to 1.1.0
Jiajia Li created HADOOP-15072: -- Summary: Upgrade Apache Kerby version to 1.1.0 Key: HADOOP-15072 URL: https://issues.apache.org/jira/browse/HADOOP-15072 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Jiajia Li Assignee: Jiajia Li Apache Kerby 1.1.0 implements cross-realm support, and also includes a GSSAPI module. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14426) Upgrade Kerby version from 1.0.0-RC2 to 1.0.0
Jiajia Li created HADOOP-14426: -- Summary: Upgrade Kerby version from 1.0.0-RC2 to 1.0.0 Key: HADOOP-14426 URL: https://issues.apache.org/jira/browse/HADOOP-14426 Project: Hadoop Common Issue Type: Improvement Reporter: Jiajia Li Assignee: Jiajia Li Apache Kerby 1.0.0 with some bug fixes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13220) Ignore findbugs checking in MiniKdc#stop and add the kerby version hadoop-project/pom.xml
Jiajia Li created HADOOP-13220:
--
Summary: Ignore findbugs checking in MiniKdc#stop and add the
kerby version hadoop-project/pom.xml
Key: HADOOP-13220
URL: https://issues.apache.org/jira/browse/HADOOP-13220
Project: Hadoop Common
Issue Type: Improvement
Components: test
Reporter: Jiajia Li
This is a follow up jira from HADOOP-12911.
1. Now with the findbug warning：
{noformat}
org.apache.hadoop.minikdc.MiniKdc.stop() calls Thread.sleep() with a lock held
At MiniKdc.java:lock held At MiniKdc.java:[line 345]
{noformat}
As discussed in HADOOP-12911:
bq. Why was this committed with a findbugs errors rather than adding the
necessary plumbing in pom.xml to make it go away?
we will add the findbugsExcludeFile.xml and will get rid of this given
kerby-1.0.0-rc3 release.
2. Add the kerby version hadoop-project/pom.xml
bq. hadoop-project/pom.xml contains the dependencies of all libraries used in
all modules of hadoop, under dependencyManagement. Only here version will be
mentioned. All other Hadoop Modules will inherit hadoop-project, so all
submodules will use the same version. In submodule, version need not be
mentioned in pom.xml. This will make version management easier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-13015) Implement kinit command execution facility in Java by leveraging Apache Kerby
Jiajia Li created HADOOP-13015: -- Summary: Implement kinit command execution facility in Java by leveraging Apache Kerby Key: HADOOP-13015 URL: https://issues.apache.org/jira/browse/HADOOP-13015 Project: Hadoop Common Issue Type: Improvement Reporter: Jiajia Li Assignee: Jiajia Li The kinit command is configured and invoked for obtaining and renewing tickets in Hadoop. There will be some benefits to implement the facility by leveraging the Kerby KrbClient API: 1. Resolve the hard dependency to MIT Kerberos client package; 2. Pure in Java, no need to configure and set up any runnable environment, easy to support on various platforms, easy to maintain and test. This proposes to do the change with a prototype patch for the review and discussion. Welcome your feedback. Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-13014) Add the MiniKdcService in Hadoop
Jiajia Li created HADOOP-13014: -- Summary: Add the MiniKdcService in Hadoop Key: HADOOP-13014 URL: https://issues.apache.org/jira/browse/HADOOP-13014 Project: Hadoop Common Issue Type: Improvement Reporter: Jiajia Li Assignee: Jiajia Li As discussed in HADOOP-12911, Steve advised "Perhaps we could have a MiniKDC service, which the existing MiniKDC code instantiated on its existing lifecycle." -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-13013) Introduce Apache Kerby into Hadoop
Jiajia Li created HADOOP-13013: -- Summary: Introduce Apache Kerby into Hadoop Key: HADOOP-13013 URL: https://issues.apache.org/jira/browse/HADOOP-13013 Project: Hadoop Common Issue Type: Improvement Reporter: Jiajia Li Assignee: Jiajia Li As discussed in the mailing list, we’d like to introduce Apache Kerby into Hadoop. Apache Kerby is a Kerberos centric project and aims to provide a first Java Kerberos library that contains both client and server supports. The relevant features include: It supports full Kerberos encryption types aligned with both MIT KDC and MS AD; Client APIs to allow to login via password, credential cache, keytab file and etc.; Utilities for generate, operate and inspect keytab and credential cache files; A simple KDC server that borrows some ideas from Hadoop-MiniKDC and can be used in tests but with minimal overhead in external dependencies; A brand new token mechanism is provided, can be experimentally used, using it a JWT token can be used to exchange a TGT or service ticket; Anonymous PKINIT support, can be experientially used, as the first Java library that supports the Kerberos major extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12911) Upgrade Hadoop MiniKDC with Kerby
Jiajia Li created HADOOP-12911: -- Summary: Upgrade Hadoop MiniKDC with Kerby Key: HADOOP-12911 URL: https://issues.apache.org/jira/browse/HADOOP-12911 Project: Hadoop Common Issue Type: Improvement Components: test Reporter: Jiajia Li Assignee: Jiajia Li As discussed in the mailing list, we’d like to introduce Apache Kerby into Hadoop. Initially it’s good to start with upgrading Hadoop MiniKDC with Kerby offerings. Apache Kerby (https://github.com/apache/directory-kerby), as an Apache Directory sub project, is a Java Kerberos binding. It provides a SimpleKDC server that borrowed ideas from MiniKDC and implemented all the facilities existing in MiniKDC. Currently MiniKDC depends on the old Kerberos implementation in Directory Server project, but the implementation is stopped being maintained. Directory community has a plan to replace the implementation using Kerby. MiniKDC can use Kerby SimpleKDC directly to avoid depending on the full of Directory project. Kerby also provides nice identity backends such as the lightweight memory based one and the very simple json one for easy development and test environments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
