to write MR-Jobs.
Regards,
Thomas
-Original Message-
From: Doug Meil [mailto:doug.m...@explorysmedical.com]
Sent: Freitag, 16. September 2011 21:42
To: user@hbase.apache.org
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
Chris, agreed... There are sometimes
distribution available as VMWare.
Thanks!
Thomas
-Original Message-
From: Doug Meil [mailto:doug.m...@explorysmedical.com]
Sent: Montag, 19. September 2011 15:35
To: user@hbase.apache.org
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
Those were all from
!
Thomas
-Original Message-
From: Sonal Goyal [mailto:sonalgoy...@gmail.com]
Sent: Freitag, 16. September 2011 18:07
To: user@hbase.apache.org
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
Hi Thomas,
I just assumed that you are already using reducers. From what I
is AFAIK the old way to write MR-Jobs.
Regards,
Thomas
-Original Message-
From: Doug Meil [mailto:doug.m...@explorysmedical.com]
Sent: Freitag, 16. September 2011 21:42
To: user@hbase.apache.org
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
Chris, agreed
haven't
benchmarked either against a reducer.
Does that make sense?
-Mike
From: doug.m...@explorysmedical.com
To: user@hbase.apache.org
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer ...
Chris, agreed
There is a DBOutputFormat class in the org.apache,hadoop.mapreduce.lib.db
package, you could use that. Or you could write to the hdfs and then use
something like HIHO[1] to export to the db. I have been working extensively
in this area, you can write to me directly if you need any help.
1.
I think you need to get a little bit more information.
Reducers are expensive.
When Thomas says that he is aggregating data, what exactly does he mean?
When dealing w HBase, you really don't want to use a reducer.
You may want to run two map jobs and it could be that just dumping the output
via
Hi Thomas,
I just assumed that you are already using reducers. From what I understood,
please correct me if I am mistaken,
You have data in HBase and you are running a MR job to aggregate the data.
You have the map as well as reduce phase and as part of the final output,
you want to send the
Michel,
Sorry can you please help me understand what you mean when you say that when
dealing with HBase, you really dont want to use a reducer? Here, Hbase is
being used as the input to the MR job.
Thanks
Sonal
On Fri, Sep 16, 2011 at 2:35 PM, Michel Segel michael_se...@hotmail.comwrote:
I
?
Date: Fri, 16 Sep 2011 21:41:01 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
From: sonalgoy...@gmail.com
To: user@hbase.apache.org
Michel,
Sorry can you please help me understand what you mean when you say that when
dealing with HBase, you really dont
Sep 2011 21:41:01 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
From: sonalgoy...@gmail.com
To: user@hbase.apache.org
Michel,
Sorry can you please help me understand what you mean when you say that
when
dealing with HBase, you really dont want
the
occurrence of each column name via dynamic counters.
There is no need for a reducer.
Does that help?
Date: Fri, 16 Sep 2011 21:41:01 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
From: sonalgoy...@gmail.com
To: user@hbase.apache.org
Michel
done at my client's site, we try very hard to avoid
reducers.
[Note, I'm sure I'm going to get hammered on this when I head to NY in
Nov. :-) ]
Date: Fri, 16 Sep 2011 23:00:49 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer
...
From: sonalgoy...@gmail.com
Chris,
I don't know what sort of aggregation you are doing, but again, why not write
to a temp table instead of using a reducer?
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
From: c...@email.com
Date: Fri, 16 Sep 2011 11:58:05 -0700
To: user
...@explorysmedical.com
To: user@hbase.apache.org
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
Chris, agreed... There are sometimes that reducers aren't required, and
then situations where they are useful. We have both kinds of jobs
instead of using a reducer?
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
From: c...@email.com
Date: Fri, 16 Sep 2011 11:58:05 -0700
To: user@hbase.apache.org
If only I could make NY in Nov :)
We extract out large numbers of DNA sequence reads from HBase
are doing, but again, why not
write to a temp table instead of using a reducer?
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
From: c...@email.com
Date: Fri, 16 Sep 2011 11:58:05 -0700
To: user@hbase.apache.org
If only I could make NY in Nov :)
We
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...
Chris, agreed... There are sometimes that reducers aren't required, and
then situations where they are useful. We have both kinds of jobs.
For others following the thread, I
a reducer.
Does that make sense?
-Mike
From: doug.m...@explorysmedical.com
To: user@hbase.apache.org
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer ...
Chris, agreed... There are sometimes that reducers aren't required
in
Nov. :-) ]
Date: Fri, 16 Sep 2011 23:00:49 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer
...
From: sonalgoy...@gmail.com
To: user@hbase.apache.org
Hi Michael,
Yes, thanks, I understand the fact that reducers can be expensive
with
all
: doug.m...@explorysmedical.com
To: user@hbase.apache.org
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer ...
Chris, agreed... There are sometimes that reducers aren't required,
and
then situations where they are useful. We
head to
NY in
Nov. :-) ]
Date: Fri, 16 Sep 2011 23:00:49 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer
...
From: sonalgoy...@gmail.com
To: user@hbase.apache.org
Hi Michael,
Yes, thanks, I understand the fact that reducers can
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer ...
Chris, agreed... There are sometimes that reducers aren't required,
and
then situations where they are useful. We have both kinds of jobs.
For others following the thread, I
. :-) ]
Date: Fri, 16 Sep 2011 23:00:49 +0530
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer
...
From: sonalgoy...@gmail.com
To: user@hbase.apache.org
Hi Michael,
Yes, thanks, I understand the fact that reducers can be
expensive
with
all
the shuffling
Date: Fri, 16 Sep 2011 15:41:44 -0400
Subject: Re: Writing MR-Job: Something like OracleReducer,
JDBCReducer ...
Chris, agreed... There are sometimes that reducers aren't
required,
and
then situations where they are useful. We have both kinds of
jobs.
For others following the thread
25 matches
Mail list logo
