Re: Large number of application attempts

2015-07-08 Thread Akmal Abbasov
But I’ve noticed that there are ~4000 znodes in /hadoop-ha/testhbase1/rmstore/ZKRMStateRoot/RMAppRoot in Zookeeper. Can I manually remove this znodes, or do I need to do this through yarn command line? Thank you. On 07 Jul 2015, at 10:23, Akmal Abbasov akmal.abba...@icloud.com wrote: Hi

Re: Large number of application attempts

2015-07-08 Thread Akmal Abbasov
Hi, Are you running a lots of apps ? I have 5 tables in HBase, and I run a cron job each hour which will create a snapshot of each table, and tries to export it to a remote HBase cluster. Or could you, maybe, share some of file names of those znodes, please ? they are all have the name

Re: Large number of application attempts

2015-07-08 Thread Xuan Gong
bq. Can I manually remove this znodes, or do I need to do this through yarn command line? Which version of hadoop you are using ? If the version is 2.6 or after 2.6, you could do yarn resourcemanager -format-state-store which will format all the rmstore (basically delete path

Different outputformats in avro map reduce job

2015-07-08 Thread Nishanth S
Hi All, I have a map reduce job which reads a binary file and needs to output multiple avro files and a textformat file.I was able to output multiplle avro files using (Avromultipleouts).How would I modify the job to output textformat as well along with these avro files.Is it possible.

Re: tools.DistCp: Invalid arguments

2015-07-08 Thread Giri P
will distcp do checksum after it copied the data to target ? On Tue, Feb 3, 2015 at 4:15 AM, Artem Ervits artemerv...@gmail.com wrote: Another good option is hftp. Artem Ervits On Feb 3, 2015 6:42 AM, xeonmailinglist xeonmailingl...@gmail.com wrote: I have found the problem. I started to

Re: Different outputformats in avro map reduce job

2015-07-08 Thread Harsh J
You can write out the text file by using a direct HDFS file writer. Your only concern in doing this approach would be the use of proper target directories, which is documented at http://wiki.apache.org/hadoop/FAQ#Can_I_write_create.2Fwrite-to_hdfs_files_directly_from_map.2Freduce_tasks.3F. On

Re: share the same namespace in 2 YARN instances?

2015-07-08 Thread Anoop Dawar
Xeon I believe there is a YARN federation project going on and another project Apache Myriad that you may want to look at. YARN Federation https://issues.apache.org/jira/browse/YARN-2915 Apache Myriad http://incubator.apache.org/projects/myriad.html -- Thanks, Anoop Dawar Senior Director,