Hadoop is a great way to offload ETL jobs (especially aggregation) out of the DB. More than likely you would want to use Hadoop as a mechanism to create a file you can load into the database as a batch job (data pump or sql loader with Oracle for example) outside of Hadoop entirely. I would imagine establishing connections inside map/reduce jobs would not be ideal.
Regards, Ryan -----Original Message----- From: Chris K Wensel [mailto:[EMAIL PROTECTED] Sent: Wednesday, July 02, 2008 11:31 AM To: core-user@hadoop.apache.org Subject: Re: hadoop in the ETL process If your referring to loading an RDBMS with data on Hadoop, this is doable. but you will need to write your own JDBC adapters to your tables. But you might review what you are using the RDBMS for and see if those jobs would be better off running on Hadoop entirely, if not for most of the processing. ckw On Jul 2, 2008, at 10:51 AM, David J. O'Dell wrote: > Is anyone using hadoop for any part of the ETL process? > > Given its ability to process large amounts of log files this seems > like > a good fit. > > -- > David O'Dell > Director, Operations > e: [EMAIL PROTECTED] > t: (415) 738-5152 > 180 Townsend St., Third Floor > San Francisco, CA 94107 > -- Chris K Wensel [EMAIL PROTECTED] http://chris.wensel.net/ http://www.cascading.org/
