Hi all.. I have a question.. I have a mapreduce program that get input from
cassandra. my input is a little big, about 1 data. my problem is
that my program takes too long to process, but I think mapreduce is good
and fast for large volume of data. so I think maybe I have problems in
number
OK, that's great. I just saw the r2.2.0 docs go online. Unfortunately they
are the same as the r2.1.0-beta docs at the moment, which aren't quite as
beginner-friendly as the r1.2.1 (stable) docs (e.g.
http://hadoop.apache.org/docs/stable/single_node_setup.html).
For instance, the 2.2.0 doc for "Se
Hi,
One point in addition to Arun's comment: the docs Arun pointed is
being updated now. Please check this JIRA.
https://issues.apache.org/jira/browse/HADOOP-10050
Thanks, Tsuyoshi
On Fri, Oct 18, 2013 at 2:00 PM, Arun C Murthy wrote:
> Try this?
>
> http://hadoop.apache.org/docs/r2.2.0/hadoop-
Apologies for the late response.
In hadoop-2 TeraSort uses the new org.apache.hadoop.mapreduce apis (not
org.apache.hadoop.mapred).
Did you fiddle with the right TotalOrderPartitioner i.e.
org.apache.hadoop.mapreduce.TotalOrderPartitioner?
Arun
On Oct 17, 2013, at 8:12 PM, sam liu wrote:
>
Try this?
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/SingleCluster.html
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/ClusterSetup.html
Arun
On Oct 18, 2013, at 1:51 PM, just...@cat.utexas.edu wrote:
> I'm just getting started with hadoop map
I'm just getting started with hadoop map reduce, and installed 1.2.1 a
couple of days ago and got some of the examples running.
Moving to 2.2.0 seems like a good idea, so I've downloaded it and began
looking at the examples.
However, it appears that the environment and setup for 2.2.0 is differen
Hi there,
I am trying to run a Hadoop source code on an ARM processor, but
getting the below error. Can anyone suggest anything as why this is
shooting up ?
rmr: cannot remove output: No such file or directory.
13/10/18 11:46:21 WARN mapred.JobClient: No job jar file set. User
classes may
I would recommend you to stop the cluster and then start the daemons one by
one.
1. stop-dfs.sh
2. hadoop-daemon.sh start namenode
3. hadoop-daemon.sh start datanode
This will show start up errors if any, also verify if the datanode is able
to communicate with the namenode.
Thanks
Divye Sheth
O
There isn't anything in the API as such. You could register your own JVM
shut-down hook which does it.
OTOH, if you are running this on Linux and a setsid binary is available, Hadoop
itself will take care of killing these additional processes - it kills the
whole session in this case.
Thanks,
Hi Sam,
Have you tried changing the map or reduce classes and seeing if that has
any effect?
-Sandy
On Fri, Oct 18, 2013 at 8:05 AM, Ravi Prakash wrote:
> Sam, I would guess that the jar file you think is running, is not actually
> the one. I am guessing that in the task classpath, there is a
Hi!
You can go to the JMX page: http://:50070/jmx to find out what the
Heap Memory and usage is. Yes we know that there is a problem in the scripts. I
believe its being handled as part of
https://issues.apache.org/jira/browse/HADOOP-9902
On Friday, October 18, 2013 2:07 AM, ch huang wrote
Sam, I would guess that the jar file you think is running, is not actually the
one. I am guessing that in the task classpath, there is a normal jar file
(without your changes) which is being picked up before your modified jar file.
On Thursday, October 17, 2013 10:13 PM, sam liu wrote:
It
For offline you need to set up local mirror, there's a guide in the manual. As
far as bits, its an Apache release not Hortonworks, same as if you were to
install from BigTop.
Artem Ervits
Data Analyst
New York Presbyterian Hospital
- Original Message -
From: Siddharth Tiwari [mailto:s
I know the task tracker frequently kill mappers. During the mapper setup we
are kicking off a runtime executable and passing data to and from it within the
mapper using unix named pipes.
This works OK until a mappers are killed. Currently when this happens the
standalone executable is left as
Compression is irrelevant with yarn.
If you want to store files with compression, you should compress the file
when they were load to HDFS.
The files on HDFS were compressed according to the parameter
"io.compression.codecs" which was set in core-site.xml.
If you want to specific a novel compressio
Some more exceptions in data node log -:
2013-10-18 10:37:53,693 WARN
org.apache.hadoop.hdfs.server.datanode.DataNode:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: Got blockRec
eived message from unregistered or dead node blk_-2949905629769882833_52274
at
org.apache.hadoop.hdfs.
Hi,
I am running Hbase in pseudo distributed mode.( Hbase 0.94.7 and Hadoop
1.1.2).
I am getting certain exceptions in Hadoop's namenode and data node files
which are :-
Namenode :-
2013-10-18 10:33:37,218 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.heartbeatCheck: lost heartbeat f
Now get a copy of the code, fix the mistake and attach the patch to the JIRA.
- André
On Fri, Oct 18, 2013 at 11:49 AM, Siddharth Tiwari
wrote:
> Opened a Jira https://issues.apache.org/jira/browse/YARN-1319
>
>
>
> **
> Cheers !!!
> Siddharth Tiwari
> Have a refreshing d
Opened a Jira https://issues.apache.org/jira/browse/YARN-1319
**
Cheers !!!
Siddharth Tiwari
Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of
God.”
"Maybe other people will try to limit me but I don't limit myself"
Can I get access to update the same ?
**
Cheers !!!
Siddharth Tiwari
Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of
God.”
"Maybe other people will try to limit me but I don't limit myself"
> Date: Fri, 18 Oct 20
The best thing to do is to open a JIRA here:
https://issues.apache.org/jira/secure/Dashboard.jspa You might also
want to submit a patch, which is very easy.
- André
On Fri, Oct 18, 2013 at 11:28 AM, Siddharth Tiwari
wrote:
> The installation documentation for Hadoop yarn at this link
> http://ha
The installation documentation for Hadoop yarn at this link
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html
has error in the yarn-site for property yarn.nodemanager.aux-services. it
should be mapreduce_shuffle rather than mapreduce.shuffle.
*--
Hi
I get following warning when I start the services in hadoop-2.2.0. What doe sit
signify and how to get rid of it ?
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
/opt/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have
disabled stack guard. The VM will try t
Hi team,
Can Hbase be used with namenode HA in latest hadoop-2.2.0 ?
If yes is there something else required to be done other than following ?
1. Set hbase root dir to logical name of namenode service
2. Keep core site and hdfs site jn hbase conf
I did above two but logical name is not recognized.
Hi team,
Is it possible to deploy hadoop from Apache via Ambari ? Also is there a link
for full offline installation ? We do not have access to outside world and we
want to use Ambari for reploying Hadoop ( not hortonworks release though )
Sent from my iPhone
In CDH3u3 there is
ClientDatanodeProtocol.recoverBlock
method, the method is used to recover block when data streaming is failed.
But in CDH4.3.1 there is not the recoverBlock method in
ClientDatanodeProtocol,
and when data streaming is failed, the block is not recovered, that whether
will lead
hi,all:
what i want is allocate different heap size to namenode and datanode
,so i add the following info in hadoop-env.sh file
export HADOOP_NAMENODE_OPTS="-Xmx2048m"
but, i am very confuse about the output of ps , here is two -Xmx option
value ,i do not know which is the real heap size tha
27 matches
Mail list logo
