Arun Suresh created HADOOP-16260:
Summary: Allow Distcp to create a new tempTarget file per File
Key: HADOOP-16260
URL: https://issues.apache.org/jira/browse/HADOOP-16260
Project: Hadoop Common
Thanks for bringing this up Wangda. +1
Makes a lot of sense to have Submarine follow its own release cadence - for
all the reasons you outlined.
I would one up this proposal to ask why shouldn't we allow YARN to have its
own releases as well - but that is for a separate thread :)
Cheers
-Arun
On
ate.
>
> Which specific job you are looking? I looked at
> https://builds.apache.org/job/PreCommit-YARN-Build/ but can't seem to
> find any user specific auth.
>
> +Vinod
>
> On Jan 22, 2019, at 10:00 AM, Arun Suresh wrote:
>
> Hey Vinod.. Ping!
>
> Cheers
> -Aru
Hey Vinod.. Ping!
Cheers
-Arun
On Fri, Jan 18, 2019 at 9:46 AM Arun Suresh wrote:
> Hi Vinod
>
> Can you please help with this:
> https://issues.apache.org/jira/browse/INFRA-17673 ?
>
> Cheers
> -Arun
>
> On Wed, Jan 16, 2019, 12:53 PM Arun Suresh
>> Hi
>
N. Features in trunk and works
> >>> end-to-end.
> >>> * (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
> >>> * (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename
> >>> commits.
> >>> * (Suma): YARN-71
> Weiwei
>
> On 27 Jan 2018, 9:16 AM +0800, Wangda Tan <wheele...@gmail.com>, wrote:
> +1
>
> On Sat, Jan 27, 2018 at 2:05 AM, Chris Douglas <cdoug...@apache.org
> <mailto:cdoug...@apache.org>> wrote:
> +1 Looking forward to this. -C
>
> On Fri,
org>, wrote:
>>> +1.
>>>
>>> Thanks Arun.
>>>
>>> I did manual testing for check affinity and anti-affinity features with
>>> placement allocator. Also checked SLS to see any performance regression,
>>> and there are not much d
Hello yarn-dev@
Based on the positive feedback from the DISCUSS thread [1], I'd like to
start a formal vote to merge YARN-6592 [2] to trunk. The vote will run for
5 days, and will end Jan 31 7:30AM PDT.
This feature adds support for placing containers in YARN using rich
placement constraints.
Hello yarn-dev@
We feel that the YARN-6592 dev branch mostly in shape to be merged into
trunk. This branch adds support for placing containers in YARN using rich
placement constraints. For example, this can be used by applications to
co-locate containers on a node or rack (*affinity *constraint),
Hey Sammi,
Looks like your filter is private.
Cheers
-Arun
On Wed, Jan 10, 2018 at 12:28 AM, Chen, Sammi wrote:
> Hi All,
>
>
>
> We have successfully released Apache Hadoop 2.9.0 in Nov. 2017 [1] with 30
> new features, 407 improvements, 790 bug fixes new fixed issues
Another option - atleast for feature branches is to maybe squash merge -
this way we see it as a single commit ? Although we will loose the feature
branch history (I am ok with that though)
Cheers
-Arun
On Thu, Dec 14, 2017 at 11:32 AM, Eric Yang wrote:
> Thank you for
+1 (binding)
- Verified signatures of the source tarball.
- built from source - using the docker build environment.
- set up a pseudo-distributed test cluster.
- ran basic HDFS commands
- ran some basic MR jobs
Cheers
-Arun
On Tue, Dec 12, 2017 at 1:52 PM, Andrew Wang
; >
> > > o User limits are honored during In-queue preemption
> > >
> > > o Priorities are honored during In-queue preemption
> > >
> > > o Can kill applications from the command line
> > >
> > > o Users with different weights are assigned r
Hi all,
We are pleased to announce the release of Apache Hadoop 2.9.0. This is the
first release of Hadoop 2.9 line and will be the starting release for
Apache Hadoop 2.9.x line - it includes 30 New Features with 500+ subtasks,
407 Improvements, 790 Bug fixes new fixed issues since 2.8.2.
Please
are documented in BUILDING.txt
> but the options do not exist in 2.x releases.
> Filed HADOOP-15045 to fix this issue.
> I think this issue doesn't block the release.
>
> Thanks and regards,
> Akira
>
> On 2017/11/14 9:10, Arun Suresh wrote:
>
>> Hi Folks,
&
Hi Folks,
Apache Hadoop 2.9.0 is the first release of Hadoop 2.9 line and will be the
starting release for Apache Hadoop 2.9.x line - it includes 30 New Features
with 500+ subtasks, 407 Improvements, 790 Bug fixes new fixed issues since
2.8.2.
More information about the 2.9.0 release plan can be
Cancelling the RC2 to fix the LICENSE.txt issue.
Will be rolling out an RC3 shortly. Given that the delta is just the
LICENSE fix, we will be carrying over the votes from this thread as well.
Cheers
-Arun/Subru
On Mon, Nov 13, 2017 at 2:59 PM, Arun Suresh <asur...@apache.org> wrote:
>
confirm first? There may (or may not) be
> other similar instances.
>
> -Xiao
>
> On Mon, Nov 13, 2017 at 2:34 PM, Arun Suresh <asur...@apache.org> wrote:
>
> > Thanks for notifying Anu,
> > We've raised https://issues.apache.org/jira/browse/HADOOP-15036 to
> add
[
https://issues.apache.org/jira/browse/HADOOP-15036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh resolved HADOOP-15036.
--
Resolution: Fixed
Fix Version/s: 3.0.0
2.9.0
Thanks for the rev
Thanks for notifying Anu,
We've raised https://issues.apache.org/jira/browse/HADOOP-15036 to address
this and have included a patch.
Can you please take a look and possibly +1 if you are fine ?
Cheers
-Arun
On Mon, Nov 13, 2017 at 2:19 PM, Anu Engineer
wrote:
> -1
Arun Suresh created HADOOP-15036:
Summary: Update LICENSE.txt for HADOOP-14840
Key: HADOOP-15036
URL: https://issues.apache.org/jira/browse/HADOOP-15036
Project: Hadoop Common
Issue Type
t;> > org.apache.curator.framework.imps.CuratorTransactionImpl$2.c
>> all(CuratorTransactionImpl.java:129)
>> > at
>> >
>> > org.apache.curator.framework.imps.CuratorTransactionImpl$2.c
>> all(CuratorTransactionImpl.java:125)
>> >
Here is my +1 to start.
- Setup a small 4 node cluster
- Verified some basic HDFS commands
- Ran Pi / sleep jobs (with some mix of Opportunistic containers - both
distributed and centralized scheduling)
Cheers
-Arun
On Fri, Nov 3, 2017 at 4:38 PM, Arun Suresh <asur...@apache.org> wrote:
Thanks for starting this discussion VInod.
I agree (C) is a bad idea.
I would prefer (A) given that ATM, branch-2 is still very close to
branch-2.9 - and it is a good time to make a collective decision to lock
down commits to branch-2.
I think we should also clearly define what the 'bridging'
stead of
> the md files, for quicker perusal?
>
> Thanks
> +Vinod
>
> On Nov 3, 2017, at 3:50 PM, Arun Suresh <asur...@apache.org> wrote:
>
> Hi folks,
>
> Apache Hadoop 2.9.0 is the first stable release of Hadoop 2.9 line and
> will be the latest stable/pr
Hi folks,
Apache Hadoop 2.9.0 is the first stable release of Hadoop 2.9 line and
will be the latest stable/production release for Apache Hadoop - it
includes 30 New Features with 500+ subtasks, 407 Improvements, 787 Bug
fixes new fixed issues since 2.8.2 .
More information about the
+1 (binding)
Cheers
-Arun
On Nov 3, 2017 11:44 AM, "Chandni Singh" wrote:
> +1
> Thanks,
> Chandni
>
> From: Jian He
> Sent: Monday, October 30, 2017 1:49 PM
> To: yarn-...@hadoop.apache.org; Hdfs-dev;
Quick update folks.
We just cut branch-2.9.0, and set the mvn version to 2.9.0
We've also updated the version on branch-2.9 to 2.9.1-SNAPSHOT.
Expect an RC in the next 24 hrs :)
Cheers
-Arun/Subru
On Tue, Oct 31, 2017 at 5:24 PM, Arun Suresh <arun.sur...@gmail.com> wrote:
> H
the method/class in question has
actually not been touched in years.
@Junping, Kindly do take a look and let us know if something is amiss.
Thanks
-Arun/Subru
On Tue, Oct 31, 2017 at 1:47 PM, Arun Suresh <asur...@apache.org> wrote:
> Hi Junping / Wangda,
>
> We just ran jdiff on br
n to separate thread.
> Sure. I will reply jdiff in separated thread.
>
> Thanks,
>
> Junping
>
> 2017-10-31 13:44 GMT-07:00 Arun Suresh <asur...@apache.org>:
>
>> Hello folks
>>
>> We just cut branch-2.9 since all the critical/blocker issues are
PM, Arun Suresh <asur...@apache.org> wrote:
> Hello Jumping
>
> Thanks for chiming in.. Appreciate the offer to help. I did run jdiff on
> branch-2 today.. did not find any red flags. Will post the report here
> shortly for review.
>
> Cheers
> -Arun
>
> On O
Hello folks
We just cut branch-2.9 since all the critical/blocker issues are now
resolved.
We plan to perform some sanity checks for the rest of the week and cut
branch-2.9.0 and push out an RC0 by the end of the week.
Kindly refrain from committing to branch-2.9 without giving us a heads up.
now if you need any help
> here.
>
> Thanks,
>
> Junping
>
> From: Subru Krishnan <su...@apache.org>
> Sent: Monday, October 30, 2017 12:39 PM
> To: common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org;
> hdfs-...@h
I actually like this idea:
> One approach: do a dependency:list of each module and for those that show
a
change with the patch we run tests there.
Can 'jdeps' be used to prune the list of sub modules on which we do
pre-commit ? Essentially, we figure out which classes actually use the
modified
I've added Subru and Chris.
@Vrushali, I could not find your username. I think you need to sign up to
confluence first with your apache id.
Cheers
-Arun
On Mon, Sep 11, 2017 at 3:36 PM, Vrushali C wrote:
> I would like to get access to the Hadoop Wiki as well. My
Hello Folks
Thanks to everyone who chimed in in the discussion thread:
https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27104.html
We have updated the roadmap wiki based on the feedback:
https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Version2.9
Given the explosion
Thanks Akira
Cheers
-Arun
On Fri, Sep 8, 2017 at 6:53 PM, Akira Ajisaka <aajis...@apache.org> wrote:
> You can get the privilege by sending an e-mail to the dev ML.
> I added it for you.
>
> Thanks,
> Akira
>
>
> On 2017/09/09 4:50, Arun Suresh wrote:
>
>&
Hi folks
How do we get access to edit the confluence wiki;
https://cwiki.apache.org/confluence/display/HADOOP ?
We were hoping to update it with hadoop 2.9 release details.
Cheers
-Arun
Thanks for starting this Wangda.
I would also like to add:
- YARN-5972: Support Pausing/Freezing of opportunistic containers
Cheers
-Arun
On Wed, Sep 6, 2017 at 1:49 PM, Steve Loughran
wrote:
>
> > On 6 Sep 2017, at 19:13, Wangda Tan wrote:
> >
>
Thanks for all the work on this folks.
I know the VOTE thread has started for this.
But I did have a couple of questions:
* Is there a branch-2 merge planned for this ?
* I understand YARN-7126 has some introductory documentation, But I think
we need to flesh it up a bit more before release, I
+1 (binding).
Cheers
-Arun
On Fri, Sep 1, 2017 at 5:24 PM, Wangda Tan wrote:
> +1 (Binding), I tried to use YARN service assembly before to run different
> kinds of jobs (for example, distributed Tensorflow), it is really easy for
> end user to run jobs on YARN.
>
> Thanks
Thanks for starting this Daniel.
I think we should also add a section for store compatibility (all state
stores including RM, NM, Federation etc.). Essentially an explicit policy
detailing when is it ok to change the major and minor versions and how it
should relate to the hadoop release version.
application/server),
> > we have setup a cluster with this feature turned on runs for several
> weeks,
> > we didn't see any issues by far.
> >
> > Merge JIRA: YARN-7013 (Jenkins gave +1 already).
> > Documentation: YARN-7056
> >
> > Special thanks to a team of folks wh
Hi
I had the Apr-14 eve version of trunk on my local machine. I've pushed that.
Don't know if anything was committed over the weekend though.
Cheers
-Arun
On Mon, Apr 17, 2017 at 7:17 AM, Anu Engineer
wrote:
> Hi Allen,
>
>
Thanks Allen..
On Apr 14, 2017 3:29 PM, "Allen Wittenauer" <a...@effectivemachines.com>
wrote:
>
> > On Apr 13, 2017, at 11:13 PM, Arun Suresh <asur...@apache.org> wrote:
> >
> > Yup,
> >
> > YARN Pre-Commit tests are having the same problem
Yup,
YARN Pre-Commit tests are having the same problem as well.
Is there anything that can be done to fix this ? Ping Yetus folks (Allen /
Sean)
All seem to be failing with:
<--
Compiling
/tmp/pip_build_root/pylint/pylint/test/functional/abstract_class_instantiated_py3.py
...
It might be problem with maven repository, What's your maven repository
> path..?
>
> May be you can change to shorter path and check.
>
>
>
> Regards
> Brahma Reddy Battula
>
> -Original Message-
> From: Arun Suresh [mailto:asur...@apache.org]
> Sent: 29
Hi
I was wondering if folks who compile Hadoop trunk on windows regularly have
hit this issue:
[INFO] --- native-maven-plugin:1.0-alpha-8:javah (default) @ hadoop-common
---
[INFO] cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_121\bin\javah" -d
Thanks for all the work on this Andrew. I agree with Chris and Karthik.
+1 (binding) from me as well
- Ran a 9 node cluster HDFS and YARN cluster
- Played with some hdfs commands
- Ran some sleep jobs using YARN ReservationSytstem + Opportunistic
Containers
- Compiled REEF and Spark
Just filed https://issues.apache.org/jira/browse/HADOOP-13999
To add a mvn flag and profile that can disable shading
On Wed, Jan 18, 2017 at 9:46 AM, Daniel Templeton
wrote:
> For most purposes, you can get by with "mvn clean process-resources
> process-test-resources
Arun Suresh created HADOOP-13999:
Summary: Add maven profile to dissable jar shading to reduce
compile time
Key: HADOOP-13999
URL: https://issues.apache.org/jira/browse/HADOOP-13999
Project: Hadoop
It looks like *org.apache.hadoop.hdfs.StripeReader* is using a Java 8
lambda expression which commons bcel is still not comfortable with.
As per https://issues.apache.org/jira/browse/BCEL-173 It should be fixed in
commons release 6.0 of bcel.
or maybe replace with bcel-findbugs ? as suggested by :
+1 (binding).
Thanks for driving this Andrew..
* Download and built from source.
* Setup a 5 mode cluster.
* Verified that MR works with opportunistic containers
* Verified that the AMRMClient supports 'allocationRequestId'
Cheers
-Arun
On Thu, Sep 1, 2016 at 4:31 PM, Aaron Fabbri
Arun Suresh created HADOOP-13217:
Summary: Add ensureExecutionType boolean flag in ResourceRequest
Key: HADOOP-13217
URL: https://issues.apache.org/jira/browse/HADOOP-13217
Project: Hadoop Common
Thanks for taking care of this Akira !!
-Arun
On Tue, May 17, 2016 at 10:13 PM, Akira AJISAKA <ajisa...@oss.nttdata.co.jp>
wrote:
> Added Arun Suresh into committer role in MAPREDUCE.
> Added Yufei Gu into contributor role in HDFS.
>
> Thanks,
> Akira
>
> On 5/
>>>>> > >>
>>>>> > >> Please grant me also to the group.
>>>>> > >>
>>>>> > >> Thanks in advance,
>>>>> > >> -Vinay
>>>>> > >> On 17 May 2016
Not sure if this is related.. but It also looks like I am now no longer
allowed to modify description and headline of JIRAs anymore..
Would appreciate greatly if someone can help revert this !
Cheers
-Arun
On Mon, May 16, 2016 at 1:21 AM, Zhihai Xu wrote:
> Currently I also
Thanks for taking a look at this Allen.. Will check out the patch..
-Arun
On Apr 27, 2016 9:16 PM, "Allen Wittenauer"
<allenwittena...@yahoo.com.invalid> wrote:
>
> > On Apr 27, 2016, at 3:53 PM, Arun Suresh <asur...@apache.org> wrote:
> >
> > which
Hi
Looks like the last couple of pre-commit jenkins builds are failing with
the following error :
http://hackage.haskell.org/packages[0m[91m/archive/[0m[91m00-index.tar.gz
: Err[0m[91morMisc
[0m[91m"Unsu[0m[91mcessful [0m[91mHTTP[0m[91m co[0m[91mde:
[0m[91m502"[0m[91m
Hello Sangjin
I had similar problems recently.. Got around it by modifying my /etc/hosts.
I just update the 127.0.0.1 entry to add the short hostname of the machine
like so :
...
127.0.0.1 localhost
...
On Wed, Jan 6, 2016 at 8:52 AM, Sangjin Lee wrote:
> Has anyone set up
.
Where is the FileInfo object persisted? Is it in the NameNode?
How is the KeyMaterial derived from the KeyAlias and where is the mapping
between the two stored? Is it in the KMS?
Thanks much for your help in this.
Sitaraman
On Sun, Jun 14, 2015 at 12:14 PM, Arun Suresh asur...@cloudera.com
Hello Sitaraman,
It is the EZ key version that is used to generate the EDEK (and which is
ultimately stored in the encrypted file's extended attributes
'*raw.hdfs.crypto.encryption.info
http://raw.hdfs.crypto.encryption.info*'), not really the the EZ key
itself (which is stored in the directory's
My 2 cents :
I actually like the lower char limit. Since most of the time, I end up
having to switch between classes, I end up splitting my screen vertically
and having multiple files open. for some reason, I find splitting
vertically is better for readability than horizontally.
In anycase, my IDE
Arun Suresh created HADOOP-11891:
Summary: OsSecureRandom should lazily fill its reservoir
Key: HADOOP-11891
URL: https://issues.apache.org/jira/browse/HADOOP-11891
Project: Hadoop Common
Arun Suresh created HADOOP-11892:
Summary: CryptoCodec#getInstance always returns a new instance of
CryptoCodec. This could be expensive
Key: HADOOP-11892
URL: https://issues.apache.org/jira/browse/HADOOP-11892
, we can get it into 2.7.1
- Or 2.8.
Thanks
+Vinod
On Apr 9, 2015, at 11:41 AM, Arun Suresh asur...@cloudera.com wrote:
Hey Vinod,
I was wondering if you can include
https://issues.apache.org/jira/browse/HDFS-8091 too.. Without it, i
kinda
makes https://issues.apache.org/jira
Arun Suresh created HADOOP-11722:
Summary: Some Instances of Services using
ZKDelegationTokenSecretManager go when old token cannot be deleted
Key: HADOOP-11722
URL: https://issues.apache.org/jira/browse/HADOOP
Arun Suresh created HADOOP-11702:
Summary: Allow StringSignerSecretProvider to read secret from a
file
Key: HADOOP-11702
URL: https://issues.apache.org/jira/browse/HADOOP-11702
Project: Hadoop Common
Arun Suresh created HADOOP-11622:
Summary: Add support for specifying only groups in kms-acls.xml
Key: HADOOP-11622
URL: https://issues.apache.org/jira/browse/HADOOP-11622
Project: Hadoop Common
Arun Suresh created HADOOP-11620:
Summary: Add Support for Load Balancing across a group of KMS
server for HA
Key: HADOOP-11620
URL: https://issues.apache.org/jira/browse/HADOOP-11620
Project: Hadoop
Arun Suresh created HADOOP-11482:
Summary: Clients that cache instances of KMSClientProvider fail
Authentication when addDelegationTokens() method is called after
authentication token validity expires
Key: HADOOP-11482
Arun Suresh created HADOOP-11368:
Summary: Fix OutOfMemory caused due to leaked trustStore reloader
thread in KMSClientProvider
Key: HADOOP-11368
URL: https://issues.apache.org/jira/browse/HADOOP-11368
Arun Suresh created HADOOP-11355:
Summary: When accessing data in HDFS and the key has been deleted,
a Null Pointer Exception is shown.
Key: HADOOP-11355
URL: https://issues.apache.org/jira/browse/HADOOP-11355
Arun Suresh created HADOOP-11344:
Summary: KMS kms-config.sh sets a default value for the keystore
password even in non-ssl setup
Key: HADOOP-11344
URL: https://issues.apache.org/jira/browse/HADOOP-11344
Arun Suresh created HADOOP-11341:
Summary: Check if user is present in default key ACL if user does
not have explicit key ACLS
Key: HADOOP-11341
URL: https://issues.apache.org/jira/browse/HADOOP-11341
Arun Suresh created HADOOP-11300:
Summary: KMS startup scripts must not display the keystore /
truststore passwords
Key: HADOOP-11300
URL: https://issues.apache.org/jira/browse/HADOOP-11300
Project
Arun Suresh created HADOOP-11272:
Summary: Allow ZKSignerSecretProvider and
ZKDelegationTokenSecretManager to use the same curator client
Key: HADOOP-11272
URL: https://issues.apache.org/jira/browse/HADOOP-11272
Arun Suresh created HADOOP-11222:
Summary: Fix findBugs error in SpanReceiverHost
Key: HADOOP-11222
URL: https://issues.apache.org/jira/browse/HADOOP-11222
Project: Hadoop Common
Issue Type
Arun Suresh created HADOOP-11187:
Summary: NameNode - KMS communication fails after a long period of
inactivity
Key: HADOOP-11187
URL: https://issues.apache.org/jira/browse/HADOOP-11187
Project
Arun Suresh created HADOOP-11178:
Summary: Fix findbugs exclude file
Key: HADOOP-11178
URL: https://issues.apache.org/jira/browse/HADOOP-11178
Project: Hadoop Common
Issue Type: Bug
Arun Suresh created HADOOP-11169:
Summary: Fix DelegationTokenAuthenticatedURL to pass the
connection Configurator to the authenticator
Key: HADOOP-11169
URL: https://issues.apache.org/jira/browse/HADOOP-11169
Arun Suresh created HADOOP-11170:
Summary: ZKDelegationTokenSecretManager throws Exception when
trying to renewToken created by a peer
Key: HADOOP-11170
URL: https://issues.apache.org/jira/browse/HADOOP-11170
Arun Suresh created HADOOP-11161:
Summary: Expose close method in KeyProvider to give clients of
Provider implementations a hook to release resources
Key: HADOOP-11161
URL: https://issues.apache.org/jira/browse
Arun Suresh created HADOOP-3:
Summary: Namenode not able to reconnect to KMS after KMS restart
Key: HADOOP-3
URL: https://issues.apache.org/jira/browse/HADOOP-3
Project: Hadoop Common
Arun Suresh created HADOOP-4:
Summary: Add testcase to TestKMSwithZK integrating with
ZKDelegationTokenSecretManager
Key: HADOOP-4
URL: https://issues.apache.org/jira/browse/HADOOP-4
Arun Suresh created HADOOP-5:
Summary: ZKDelegationTokenSecretManager creates two path trees to
store state.. these can be merged
Key: HADOOP-5
URL: https://issues.apache.org/jira/browse/HADOOP-5
Arun Suresh created HADOOP-11046:
Summary: Normalize ACL configuration property across hadoop
components
Key: HADOOP-11046
URL: https://issues.apache.org/jira/browse/HADOOP-11046
Project: Hadoop
Arun Suresh created HADOOP-10939:
Summary: Fix TestKeyProviderFactory testcases to use default 128
bit length keys
Key: HADOOP-10939
URL: https://issues.apache.org/jira/browse/HADOOP-10939
Project
Arun Suresh created HADOOP-10937:
Summary: Fix NullPointerException when touchz-ing a file
Key: HADOOP-10937
URL: https://issues.apache.org/jira/browse/HADOOP-10937
Project: Hadoop Common
Arun Suresh created HADOOP-10412:
Summary: First call from Client fails after Server restart
Key: HADOOP-10412
URL: https://issues.apache.org/jira/browse/HADOOP-10412
Project: Hadoop Common
90 matches
Mail list logo