No, unless a user explicitly puts it into a UDF, none of the built-in UDFs and SPJs use MR.
Hans On Tue, Feb 2, 2016 at 1:21 PM, Gunnar Tapper <[email protected]> wrote: > Hi, > > I think it's a configuration choice. For example: > http://www.cloudera.com/documentation/manager/5-0-x/Cloudera-Manager-Managing-Clusters/cm5mc_mapreduce_service.html > > Are there UDFs, SPJs, and similar functions that use MapReduce, too? > > Thanks, > > Gunnar > > On Tue, Feb 2, 2016 at 2:19 PM, Qifan Chen <[email protected]> wrote: > >> >> Hi Hans, >> >> I think Hive uses MapReduce to sort the data during table population, >> after disabling YARN. This is observed on a workstation. >> >> Thanks -Qifan >> >> On Tue, Feb 2, 2016 at 3:12 PM, Hans Zeller <[email protected]> >> wrote: >> >>> Hi, >>> >>> That decision would be made by Hive, not Trafodion. For people who use >>> install_local_hadoop, we recently changed that setup to use local >>> MapReduce, not YARN, see >>> https://issues.apache.org/jira/browse/TRAFODION-1781. >>> >>> Hans >>> >>> On Tue, Feb 2, 2016 at 12:58 PM, Gunnar Tapper <[email protected]> >>> wrote: >>> >>>> Hi Suresh: >>>> >>>> Thanks for the information. >>>> >>>> Given from what you write, it seems that YARN with MRv2 is required for >>>> full functionality. >>>> >>>> MRv1 is a separate install in current distributions, which is why I am >>>> asking about it. How does Trafodion decide to run the MapReduce job as MRv1 >>>> vs. MRv2 if both are installed? >>>> >>>> Thanks, >>>> >>>> Gunnar >>>> >>>> >>>> >>>> On Tue, Feb 2, 2016 at 1:50 PM, Suresh Subbiah < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> I don't think Trafodion requires YARN for most activity. >>>>> >>>>> For Hive table access, Trafodion uses Hive metadata access Java API >>>>> and libhdfs to actually scan the data file. Therefore YARN is not needed >>>>> for Hive access. >>>>> YARN is not needed for native Hbase or Trafodion table access too. >>>>> YARN is needed for backup/restore, since the HBase exportSnapshot >>>>> class Trafodion calls, used MapReduce to copy large snapshot files to/from >>>>> the backup location. >>>>> YARN is also needed for developer regressions as some vanilla Hive >>>>> commands are executed during the regression run. >>>>> For the last 2 lines I think both MRv1 and MRv2 is supported. >>>>> >>>>> Thanks >>>>> Suresh >>>>> >>>>> >>>>> On Tue, Feb 2, 2016 at 2:36 PM, Gunnar Tapper <[email protected] >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Does Trafodion require YARN with MRv2 or is MRv1 supported, too? >>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> >>>>>> Gunnar >>>>>> *If you think you can you can, if you think you can't you're right.* >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> >>>> Gunnar >>>> *If you think you can you can, if you think you can't you're right.* >>>> >>> >>> >> >> >> -- >> Regards, --Qifan >> >> > > > -- > Thanks, > > Gunnar > *If you think you can you can, if you think you can't you're right.* >
