Hey
Did u find any class or way out for storing results of Job1 map/reduce in
memory and using that as an input to job2 map/Reduce?I am facing a situation
where I need to do similar thing.If anyone can help me out..
Pankil
On Wed, Apr 8, 2009 at 12:51 AM, Sharad Agarwal
if you need any more details.
Pankil
On Mon, Apr 13, 2009 at 9:51 PM, sharad agarwal shara...@yahoo-inc.comwrote:
Pankil Doshi wrote:
Hey
Did u find any class or way out for storing results of Job1 map/reduce in
memory and using that as an input to job2 map/Reduce?I am facing a
situation
Hello Everyone,
At time I get following error,when i restart my cluster desktops.(Before
that I shutdown mapred and dfs properly though).
Temp folder contains of the directory its looking for.Still I get this
error.
Only solution I found to get rid with this error is I have to format my dfs
On Wed, Apr 15, 2009 at 1:26 AM, Sharad Agarwal shara...@yahoo-inc.comwrote:
I am trying complex queries on hadoop and in which i require more than
one
job to run to get final result..results of job one captures few joins of
the
query and I want to pass those results as input to 2nd job
Hey ,
You can do that.That system should have same usrname like those of cluster
and ofcourse it should be able to ssh name node.Also it should have hadoop
and its hadoop-site.xml should be similar .Then u can access namenode,hdfs
etc.
if you are willing to see the web interface that can be
, 2009 at 6:06 PM, Pankil Doshi forpan...@gmail.com wrote:
Hello Everyone,
At time I get following error,when i restart my cluster desktops.(Before
that I shutdown mapred and dfs properly though).
Temp folder contains of the directory its looking for.Still I get this
error.
Only
Hey
what's your input size?
from the info you gave it seems you have used 4.2GB and so probably if thats
your input size your intermediate results mostly is less then your input.but
that too depends on your map function. Make sure about the size of
intermediate results.
Pankil
On Thu, Apr 16,
hey
you can surely do that using MulipleFileOutputFormat. We have already
implemented that
Pankil
On Fri, Apr 24, 2009 at 8:58 PM, Aaron Kimball aa...@cloudera.com wrote:
Alternatively, just use FileSystem.rename() on the normal output files
after
reducing is complete?
On Sat, Apr 25,
Hello Everyone,
Actually I had a cluster which was up.
But i stopped the cluster as i wanted to format it.But cant start it back.
1)when i give start-dfs.sh I get following on screen
starting namenode, logging to
/Hadoop/hadoop-0.18.3/bin/../logs/hadoop-hadoop-namenode-hadoopmaster.out
the namenode does not start the datanodes, the start script does.
The name node passively waits for the datanodes to connect to it.
On Thu, May 14, 2009 at 6:43 PM, Pankil Doshi forpan...@gmail.com wrote:
Hello Everyone,
Actually I had a cluster which was up.
But i stopped the cluster as i
, Pankil Doshi forpan...@gmail.com wrote:
Can u guide me where can I find datanode log files? As I cannot find it
in
$hadoop/logs and so.
I can only find following files in logs folder :-
hadoop-hadoop-namenode-hadoopmaster.log
hadoop-hadoop-namenode-hadoopmaster.out
hadoop
blocks
got processed in 27 msecs
2009-03-17 03:08:13,392 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic block
scanner.
On Thu, May 14, 2009 at 9:51 PM, Pankil Doshi forpan...@gmail.com wrote:
This is log from datanode.
2009-05-14 00:36:14,559 INFO
Hello everyone,
I got hint how to solve the problem where clusters have different
usernames.but now other problem I face is that i can ssh a machine by using
-i path/to key/ ..I cant ssh them directly but I will have to always pass
the key.
Now i face problem in ssh-ing my machines.Does anyone
is only used as a convenient method to launch
daemons. If you have a preferred way to start things up on your cluster,
you can use that (I think most large clusters don't use ssh... could be
wrong).
Brian
On May 21, 2009, at 2:07 PM, Pankil Doshi wrote:
Hello everyone,
I got hint how
?
- Aaron
On Thu, May 21, 2009 at 3:49 PM, Pankil Doshi forpan...@gmail.com wrote:
The problem is that it also prompts for the pass phrase.
On Thu, May 21, 2009 at 2:14 PM, Brian Bockelman bbock...@cse.unl.edu
wrote:
Hey Pankil,
Use ~/.ssh/config to set the default key location
make sure u can ping that data node and ssh it.
On Thu, May 28, 2009 at 12:02 PM, ashish pareek pareek...@gmail.com wrote:
HI ,
I am trying to step up a hadoop cluster on 512 MB machine and using
hadoop 0.18 and have followed procedure given in apache hadoop site for
hadoop cluster.
Hello Kristi,
I am Research Assistant at University of Texas at Dallas. We are working of
RDF data and we come across many joins in our queries. But We are not able
to carry out all joins in a single job..we also tried our hadoop code using
Pig scripts and found that for each join in PIG script
Yes, If your job gets completed successfully .possibly it removes after
completion of both map and reduce tasks.
Pankil
On Mon, Jun 22, 2009 at 3:15 PM, Qin Gao q...@cs.cmu.edu wrote:
Hi All,
Do you know if the tmp directory on every map/reduce task will be deleted
automatically after the
I am not sure but sometimes you might see that datanodes are working from
cmd prompt..
But actually when you look at the logs you find sme kind of error in
that..Check the logs of datanode..
Pankil
On Wed, Jun 17, 2009 at 1:42 AM, ashish pareek pareek...@gmail.com wrote:
Hi,
When I run
hi Stu,
which block conversion are you talking about? If you are talking abt block
size of data then it remains same in upgrade unless and until you change it.
Pankil
On Tue, Jun 16, 2009 at 5:16 PM, Stu Hood stuart.h...@rackspace.com wrote:
Hey gang,
We're preparing to upgrade our cluster
Hey Alex,
Will Hadoop balancer utility work in this case?
Pankil
On Mon, Jun 22, 2009 at 4:30 PM, Alex Loddengaard a...@cloudera.com wrote:
Are you seeing any exceptions because of the disk being at 99% capacity?
Hadoop should do something sane here and write new data to the disk with
more
will generate new temporary files.
--Q
On Mon, Jun 22, 2009 at 4:19 PM, Pankil Doshi forpan...@gmail.com wrote:
Yes, If your job gets completed successfully .possibly it removes after
completion of both map and reduce tasks.
Pankil
On Mon, Jun 22, 2009 at 3:15 PM, Qin Gao q...@cs.cmu.edu
the output...
% df -h /mnt /mnt2
Thanks.
-Matt
On Jun 22, 2009, at 1:32 PM, Pankil Doshi wrote:
Hey Alex,
Will Hadoop balancer utility work in this case?
Pankil
On Mon, Jun 22, 2009 at 4:30 PM, Alex Loddengaard a...@cloudera.com
wrote:
Are you seeing any exceptions because
23 matches
Mail list logo
