be appreciated,
Thanks.
--
Harsh J
back so quickly.
The full source code is attached as there's nothing sensitive in it.
Coding wouldn't be my strong point so apologies in advance if it looks a
mess.
Thanks
On Sat, Feb 9, 2013 at 6:09 PM, Harsh J ha...@cloudera.com wrote:
Whatever csatAnalysis.MapClass the compiler picked
bound jobs uselessly taking up all reduce slots.
--
Harsh J
...@gmail.com wrote:
Is there a good reason why the OldCombinerRunner passes Reporter.NULL to the
combiner instead of the actual TaskReporter? The NewCombinerRunner does use
the TaskReporter when creating the context. If this is a bug I will submit a
JIRA with a patch
--
Harsh J
file:///data/1/jenkins/workspace/generic-package-centos32-6/topdir/BUILD/hadoop-2.0.0-cdh4.1.1/src/hadoop-common-project/hadoop-common
-r 581959ba23e4af85afd8db98b7687662fe9c5f20
On Fri, Feb 8, 2013 at 12:04 AM, Harsh J ha...@cloudera.com wrote:
Hey Ravi,
What version of Hadoop is this exactly
missing in the HDFS, We want validate
source to destination.
We have already have source files count
How to find HDFS folder file count please guide me.
hadoop.apache.org/docs/r0.20.2/hdfs_shell.html#count
hadoop fs -count -q
Did I learn something today? If not, I wasted it.
--
Harsh J
for one-way nn to tt
operations?
--
Jay Vyas
http://jayunit100.blogspot.com
--
Harsh J
-setrep -R -w 1 /user
Thanks,
Austin
--
Harsh J
in 1 reducer and then do the cat of
top-N.
Wondering if there is any better approach to do this ?
Regards
Praveenesh
--
Harsh J
setttings .I have setup that acordding
to the maven guide,it does not work.Maybe the proxy site was change.
could anybody give me available proxy settings.
regards.
YouPeng Yang
--
Harsh J
Nodes and hdfs Blacklisted Nodes.
and also how to clear blacklisted nodes to clear start. The only option
to restart the service. some other way clear the Blacklisted Nodes.
please guide me.
-Dhanasekaran.
Did I learn something today? If not, I wasted it.
--
Harsh J
Zhu
School of Computer Science,
McGill University
--
Harsh J
.
There were 2 blocks lost.
Chen
On Wed, Jan 30, 2013 at 10:27 AM, Harsh J ha...@cloudera.com wrote:
NN does recalculate new replication work to do due to unavailable
replicas (under-replication) when it starts and receives all block
reports, but executes this only after out of safemode. When
, and devotion to duty is the highest form of worship of
God.”
Maybe other people will try to limit me but I don't limit myself
--
Harsh J
data
sets.
Is this pretty normal to yall for hadoop 1.0.3?
--
Benjamin Kim
benkimkimben at gmail
--
Benjamin Kim
benkimkimben at gmail
--
Harsh J
the model object with a version number, or change the header
of each sequence file.
Just wondering if there are any nice tricks to this.
--
Harsh J
--
Nitin Pawar
--
Thanks Regards
Xibin Liu
--
Nitin Pawar
--
Thanks Regards
Xibin Liu
--
Harsh J
it in the
yarn-site.xml,but no one tells the description about it。
please,helps will be appreciated .
.
--
Harsh J
?
--
Harsh J
deadlock)?
This was a problem in earlier releases, where an imbalanced or
improper config for topology levels could lead to a jarring halt in
the JobTracker's schedulers. This has been fixed in 1.1.0 already via
https://issues.apache.org/jira/browse/MAPREDUCE-1740.
--
Harsh J
Not allowed to write to a
closed appender.
attempt_201301161411_0244_m_54_0: log4j:WARN Not allowed to write to a
closed appender.
attempt_201301161411_0244_m_54_0: log4j:WARN Not allowed to write to a
closed appender.
--
Harsh J
one can help me ...
Thanks
Regards
Abdul Wajeed
--
Harsh J
?
or do i have to change it in mapred-site.xml following a restart for the
changes to apply?
Thanks in advance.
Cheers!
Manoj.
--
Harsh J
and will they
be partitioned for running MapReduce if i use Localfile system?
Can someone please explain.
Regards
Sundeep
--
Harsh J
which is on my local disk and I want to use
the output of that script for further processing in Map Function to produce
key/Value Pairs.
Can some give me some idea how to do it.
Regards
Sundeep
--
Harsh J
fuse_dfs. Any
advice?
--
Thanks Best Regards
Xibin Liu
--
Harsh J
) determine that the first record is
incomplete and should process starting from the second record in the block
(b2)?
Thanks,
Praveen
--
Harsh J
:
Thanks, http://search-hadoop.com/m/LLBgUiH0Bg2 is my issue , but I still
dont't know how to solve this problem, 3 minutes not respond once an hour
is a big problem for me, any clue for this?
2013/1/24 Harsh J ha...@cloudera.com
Hi,
HDFS does this to estimate space reports. Perhaps
.
Thanks.
--
Thanx and Regards
Vikas Jadhav
--
Harsh J
vikascjadha...@gmail.com wrote:
Hi I am working join operation using MapReduce
So if anyone has useful information plz share it.
Example Code or New Technique along with existing one.
Thank You.
--
Thanx and Regards
Vikas Jadhav
--
Harsh J
it before ?
I'm just looking for a way to submit MapReduce jobs from Java code and be
able to monitor them.
Thanks,
Amit.
--
Harsh J
permission mens
it's can't access other right.
Please guide me.
-Dhanasekaran.
Did I learn something today? If not, I wasted it.
--
Harsh J
, the java action could very
well interact with a file system. I know this is probably better addressed
in Oozie context, but wanted to get the map reduce view of things.
Thanks,
Hemanth
--
Harsh J
weekly and monthly backups upto 12 months.
Any ideas how to do this ?
-- Steve
--
Harsh J
to possibly different patterns each time.
--
Harsh J
org.apache.hadoop.mapred.MapTask: Finished
spill 0
...
Please help me to understand the reason of task fails.
--
Harsh J
, which
is below the configured reserved amount 104857600
On Wed, Jan 23, 2013 at 11:13 AM, Harsh J ha...@cloudera.com wrote:
Hi again,
Yes, you need to add it to hdfs-site.xml and restart the NN.
Thanks Harsh, Do I need to add parameters in hdfs-site.xml and restart
service namenode
into the safemode then I run command to leave
safemode manually. I never got alerts for low disk space on machine level
and i didn't see the space fluctuates GBs into MBs .
On Wed, Jan 23, 2013 at 9:10 PM, Harsh J ha...@cloudera.com wrote:
Mohit,
When do you specifically get the error
:490)
** at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
--
Harsh J
. The link I followed to install is given below
https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode
Thanks,
On Wed, Jan 23, 2013 at 10:12 PM, Harsh J ha...@cloudera.com wrote:
A random switching behavior can only be explained
picked it up.
--
Harsh J
to this transmission.
--
Harsh J
have to build our own AMI or is there another we can safely
use?
Thanks,
Daniel.
--
Harsh J
config I missed?
Thanks.
--
Harsh J
is 15T, so when hdfs exec du, datanode will not respond for
about 3 minuts because of io loading, this cause a lot of problem, anybody
knows why hdfs doing this and how to disable it?
--
Thanks Regards
Xibin Liu
--
Harsh J
you're upgrading. Other than a recompile,
you may mostly not require to do anything else.
May we also know your reason to not use CM when its aimed to make all this
much easier to do and manage? We appreciate any form of feedback, thanks!
--
Harsh J
:50 AM, Steve Loughran
steve.lough...@gmail.comwrote:
ser@hadoop.apache.orglist
--
Harsh J
DFS_NAMENODE_DU_RESERVED_DEFAULT = 1024 *
1024 * 100; // 100 MB
On Wed, Jan 23, 2013 at 10:12 AM, Harsh J ha...@cloudera.com wrote:
Edit your hdfs-site.xml (or whatever place of config your NN uses) to
lower the value of property dfs.namenode.resource.du.reserved. Create a
new property if one does
Thanks
--
View this message in context:
http://hadoop-common.472056.n3.nabble.com/no-jobtracker-to-stop-no-namenode-to-stop-tp34874p4006830.html
Sent from the Users mailing list archive at Nabble.com.
--
Harsh J
function for
inserting data to Hbase Table ?
Mohandes Zebeleh
--
Harsh J
.
The datanodes can only access LAN in the cluster.
[image: cid:image002.jpg@01CDF7E5.F8D45990]
** **
--
Harsh J
image002.jpg
, 2013 at 4:41 PM, Pedro Sá da Costa psdc1...@gmail.comwrote:
This does not save in the xml file. I think this just keep the
variable in memory.
On 19 January 2013 18:48, Arun C Murthy a...@hortonworks.com wrote:
jobConf.set(String, String)?
--
Best regards,
--
Harsh J
https://mtariq.jux.com/
cloudfront.blogspot.com
--
Harsh J
you so much.
Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Sun, Jan 20, 2013 at 4:29 PM, Harsh J ha...@cloudera.com wrote:
If your DN is starting too slow, then you should investigate why.
In any case, Apache Bigtop's (http://bigtop.apache.org)
pseudo-distributed
find another parameter
to limit the concurrent running map tasks.
Hope someone can help me ,Thanks.
--
Harsh J
that slow down the overall NN
performance?
Thanks,
randy
On 01/15/2013 11:14 PM, Harsh J wrote:
The NFS mount is to be soft-mounted; so if the NFS goes down, the NN
ejects it out and continues with the local disk. If auto-restore is
configured, it will re-add the NFS if its detected good
.
Before opening any email and/or attachments, please check them for viruses
and other defects.
--
Harsh J
/
cloudfront.blogspot.com
On Wed, Jan 16, 2013 at 6:15 PM, Chris Embree cemb...@gmail.com
wrote:
Ha, you joke, but we're planning on running with no local OS. If it
works
as planned I'll post a nice summary of our approach. :)
On Wed, Jan 16, 2013 at 2:53 AM, Harsh J ha...@cloudera.com wrote
the
dfs.namenode.replication.work.multiplier.per.iteration option would do the
trick, but that is in v1.1.0 and higher.
Thanks.
--Brennon
--
Harsh J
11:04 AM, Harsh J wrote:
You can limit the bandwidth in bytes/second values applied
via dfs.balance.bandwidthPerSec in each DN's hdfs-site.xml. Default is 1
MB/s (1048576).
Also, unsure if your version already has it, but it can be applied at
runtime too via the dfsadmin
, Jan 18, 2013 at 3:14 AM, Brennon Church bren...@getjar.com wrote:
Pretty spiky. I'll throttle it back to 1MB/s and see if it reduces
things as expected.
Thanks!
--Brennon
On 1/17/13 1:41 PM, Harsh J wrote:
Not true per the sources, it controls all DN-DN copy/move rates, although
message. localhost is bound with 127.0.01 . core-site.xml and
mapreduce-site.xml are empty as they are not modified.
Anybody can give me a hint if I need to do some specific configuration to
run hadoop in standalone mode?
thanks and regards,
Yiyu
--
Harsh J
nameio.sort.mb/name
value300/value
/property
property
nameio.file.buffer.size/name
value65536/value
/property
property
namedfs.datanode.handler.count/name
value8/value
/property
/configuration
--
Harsh J
We don't sort values (only keys) nor apply any manual limits in MR. Can
your post a reproduceable test case to support your suspicion?
On Jan 16, 2013 4:34 PM, Utkarsh Gupta utkarsh_gu...@infosys.com wrote:
Hi,
Thanks for the response. There was some issues with my code. I have
checked
The patch has not been contributed yet. Upstream at open-mpi there does
seem to be a branch that makes some reference to Hadoop, but I think the
features are yet to be made available there too. Apparently waiting on some
form of a product release first? That's all I could gather from some
items
-rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58
/user/test2/subdir/anotherone/CHANGES.txt
2013/1/16, John Meza j_meza...@hotmail.com:
Does hadoop fs -put mmdd.tsv t1/2012/01/01/mmdd.tsv create the
necassary subdirectories in hdfs?thanksJohn
--
Harsh J
: org.apache.hadoop.fs.LocalFileSystem@4ce2cb55
--
Jay Vyas
http://jayunit100.blogspot.com
--
Harsh J
(FileSystem.java:254)
snip
--
Harsh J
understand you I can configure my NN HA cluster
this
way:
- Active NameNode + 1 ZKFC daemon + Journal Node
- Standby NameNode + 1 ZKFC daemon + Journal Node
- JobTracker node + 1 ZKFC daemon + Journal Node,
Is this right?
Thanks in advance,
ESGLinux,
2012/12/27 Harsh J ha
and isolation.
Here, ZooKeeper daemons = ZKFC?
Thanks
ESGLinux,
2013/1/15 Harsh J ha...@cloudera.com
Hi,
I fail to see your confusion.
ZKFC != ZK
ZK is a quorum software, like QJM is. The ZK peers are to be run odd in
numbers, such as JNs are to be.
ZKFC is something the NN needs for its
?
It seems like these would essentially refer to the same thing?
--
Jay Vyas
http://jayunit100.blogspot.com
--
Jay Vyas
http://jayunit100.blogspot.com
--
Harsh J
to the NN and/or performance if there's a problem with the
NFS server? Or the network?
Thanks,
randy
On 01/14/2013 11:36 PM, Harsh J wrote:
Its very rare to observe an NN crash due to a software bug in
production. Most of the times its a hardware fault you should worry about.
On 1.x, or any non
-scheduler.xml
masters - for listing master nodes
Please help me setup this version.
Thanking You,
--
Regards,
Ouch Whisper
010101010101
--
Harsh J
kidding Wipe your OS out. /kidding
Please read: http://search-hadoop.com/m/9Qwi9UgMOe
On Wed, Jan 16, 2013 at 1:16 PM, Vikas Jadhav vikascjadha...@gmail.comwrote:
how to remove non dfs space from hadoop cluster
--
*
*
*
Thanx and Regards*
* Vikas Jadhav*
--
Harsh J
.
--
Harsh J
in the
HadoopUserGroups wiki page. Either giving me the rights or making the
update will do.
My id: PauloMagalhaes
What I want to add:
=== South America ===
* [http://www.meetup.com/SaoPauloHUG/| Sao Paulo HUG]: Hadoop Users
Group in Sao Paulo, Brazil
--
Harsh J
.
My id: PauloMagalhaes
What I want to add:
=== South America ===
* [http://www.meetup.com/SaoPauloHUG/| Sao Paulo HUG]: Hadoop Users Group
in Sao Paulo, Brazil
Thanks,
Paulo Magalhaes
--
Harsh J
,
--
Regards,
Ouch Whisper
010101010101
--
Harsh J
for this. Libraries such as
Kitten [2] help along in this task.
[1] -
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/examples/org/apache/hadoop/examples/SleepJob.java
[2] - https://github.com/cloudera/kitten/
--
Harsh J
if we program our application to schedule its
tasks directly with YARN we should be able to do what I am describing? Is
there any non-native-Java interop for YARN or should we focus on JNI for that?
John
-Original Message-
From: Harsh J [mailto:ha...@cloudera.com]
Sent: Saturday
, they ramp up to 40-60% in just a few minutes, then take hours
to slowly inch their way up the rest of the way to 100%.
What does the complete percentage really mean?
--
Roy Smith
r...@panix.com
--
Harsh J
RabbitMQ and receive data from them using Spring Integration
Data pipe lines.
I cannot afford to loose any of the JSON files received.
Thanking You,
--
Regards,
Ouch Whisper
010101010101
--
Harsh J
=task_201301090834_0041_r_05
0.00%
10-Jan-2013 06:11:07
10-Jan-2013 06:46:38 (35mins, 31sec)
Task attempt_201301090834_0041_r_05_0 failed to report status for 600
seconds. Killing!
0http://23.20.27.135:9003/taskstats.jsp?tipid=task_201301090834_0041_r_05
--
Harsh J
for. I really wonder if it is possible (and how) to make it work in
cdh3u4.
-P
--
Harsh J
--
Harsh J
?
or
- we need to bump up setrep to kind of trigger the number of replication
block ?
or
- ??
Thanks
-P
--
Harsh J
?
--
Harsh J
entire tasks?
-Håvard
--
Håvard Wahl Kongsgård
Faculty of Medicine
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
--
Håvard Wahl Kongsgård
Faculty of Medicine
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
--
Harsh J
distribution,
but for completeness we are running CDH3u3.
Thanks!
stan
--
Harsh J
such that
the classpath is automatically setup for you.
--
Harsh J
184376668
163872819
$
where /tmp/wcc contains
#!/bin/bash
wc -c
Thanks for any answer,
Pavel Hančar
--
Harsh J
? If that
location does not have enough space does reducer fail? What is the solution
for MapReduce jobs if intermediat results for some keys is more than local
disk of reducer?
--
Harsh J
, Pedro Sá da Costa psdc1...@gmail.com wrote:
MapReduce framework has map and reduce slots, that are used to track which
tasks are running. When map tasks are just running, the reduce slots that
the job have will be filled by map tasks?
--
Best regards,
--
Harsh J
--
Harsh J
)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1408)
at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner
--
Harsh J
throw them off and it
was very fast to debug as you code.
Is there any similar way .. where i dont have to run hadoop jobs to debg and
wait and go thru hadoop logs to see that maybe i miss a semi-colon..
Thanks
Jamal
--
Harsh J
can I implement such a counter?
Thank you and happy Christmas time,
Eduard
--
Harsh J
of files in the distributed cache are cleaned up.
Kai
--
Kai Voigt
k...@123.org
--
Kai Voigt
k...@123.org
--
Harsh J
mappers or
reducers which requires the DistributedCache).
regards,
Lin
On Wed, Dec 26, 2012 at 4:51 PM, Harsh J ha...@cloudera.com wrote:
Hi Lin,
DistributedCache files are stored onto the HDFS by the client first.
The TaskTrackers download and localize it. Therefore, as with any
other
:
Thanks Harsh, multiple concurrent read is generally faster or?
regards,
Lin
On Wed, Dec 26, 2012 at 6:21 PM, Harsh J ha...@cloudera.com wrote:
There is no limitation in HDFS that limits reads of a block to a
single client at a time (no reason to do so) - so downloads can be as
concurrent
601 - 700 of 2355 matches
Mail list logo
