Awesome! Thanks for your hard work guys! Jarcec, can we use https://github.com/apache/sqoop/commit/81624ddf3c8ca5834ab015ebafc8b8649ac36ab7? In terms of when... let's shoot for 2PM PST?
-Abe On Wed, Jul 23, 2014 at 12:32 PM, Jarek Jarcec Cecho <jar...@apache.org> wrote: > I think that both are in. Do let me know when and from which commit you > want me to branch Abe :-) > > Jarcec > > On Jul 23, 2014, at 10:38 AM, Abraham Elmahrek <a...@cloudera.com> wrote: > > > Hey guys, > > > > As soon as the Ivy and Avro version changes are in, let's start the > > branching process? > > > > -Abe > > > > > > On Wed, Jul 23, 2014 at 1:02 AM, Venkat Ranganathan < > > vranganat...@hortonworks.com> wrote: > > > >> Generally the number of tasks is a hint to InputFormat.getSplits() > >> based on number of splits wanted and it can potentially return > >> different number of splits. > >> > >> If you see ExportInputFormat, the split size is determined by > >> combined file sizes divided by requested num mappers. This is a > >> integer division and there is a potential for getting additional > >> splits. > >> > >> Thanks > >> > >> Venkat > >> > >> On Tue, Jul 22, 2014 at 11:41 PM, David Robson > >> <david.rob...@software.dell.com> wrote: > >>> Yes OraOop is not enabled by default - and also this particular issue > is > >> only when using partitioning - so they would have turned on some extra > >> flags. Then they also have to be using 1 more mapper than requested as > you > >> said - so it's a pretty specific combination of events - so I don't > think > >> we need to hold up the release for it. > >>> > >>> Do you recall what the behaviour should be in regards to this - the > >> documentation leads me to believe if we request 4 mappers we should get > 4 > >> mappers? If this is only a request and we may get more - perhaps we > should > >> update the documentation about this. Otherwise we could change the code > to > >> guarantee no more than 4 mappers? > >>> > >>> -----Original Message----- > >>> From: Abraham Elmahrek [mailto:a...@cloudera.com] > >>> Sent: Wednesday, 23 July 2014 3:16 PM > >>> To: dev@sqoop.apache.org > >>> Subject: Re: Final Jiras and Release Date > >>> > >>> Hey guys, > >>> > >>> AFAIK we can fall back onto the original connector? With that being > >> said, OraOop is an awesome connector... So i'm completely open to > waiting > >> for a fix if it isn't too large. > >>> > >>> David, I'm assuming you're referring to the "-m" option? I do recall > >> some cases where there may be one more split than desired if splitting > is > >> not clean. Hopefully Venkat has more insight! > >>> > >>> -Abe > >>> > >>> > >>> On Tue, Jul 22, 2014 at 9:42 PM, David Robson < > >> david.rob...@software.dell.com> wrote: > >>> > >>>> Hey Venkat, > >>>> > >>>> I had a look into that issue - it seems the problem is even though you > >>>> request 4 mappers - Sqoop runs 5 mappers. Is the number of mappers > >>>> meant to be guaranteed? For example if I say 4 mappers, should I get 4 > >>>> mappers? I guess there could be potential to get less mappers if the > >>>> data was for example 1 row - then that would only be processed by 1 > >>>> mapper. But should it be possible to get more mappers than requested? > >>>> > >>>> OraOop is assuming there will be no more mappers than requested - so > >>>> if this is a valid scenario we would need to modify OraOop. On the > >>>> other hand if this is unexpected behaviour then we should look at why > >>>> the num mappers is not working? > >>>> > >>>> Either way I don't think we need to fix it for this release - as it > >>>> seems customers would be unlikely to hit this issue. > >>>> > >>>> David > >>>> > >>>> -----Original Message----- > >>>> From: Venkat Ranganathan [mailto:vranganat...@hortonworks.com] > >>>> Sent: Wednesday, 23 July 2014 1:04 PM > >>>> To: dev@sqoop.apache.org > >>>> Subject: Re: Final Jiras and Release Date > >>>> > >>>> Abe > >>>> > >>>> Do we want to target SQOOP-1388 also for 1.4.5 - the one that Vidya > >>>> has raised for one of the Oracle connector export failure? > >>>> > >>>> My avro patch is in RB. > >>>> > >>>> Thanks > >>>> > >>>> Venkat > >>>> > >>>> On Tue, Jul 22, 2014 at 6:12 PM, Abraham Elmahrek <a...@cloudera.com> > >>>> wrote: > >>>>> Hey guys, > >>>>> > >>>>> It looks like we still have a couple of Jiras still open. Let's see > >>>>> how things look tomorrow, but let's have these be the last two Jiras > >>>>> slotted for this release. > >>>>> > >>>>> -Abe > >>>>> > >>>>> > >>>>> On Fri, Jul 18, 2014 at 1:20 PM, Venkat Ranganathan < > >>>>> vranganat...@hortonworks.com> wrote: > >>>>> > >>>>>> Sounds good. I have already uploaded the patch for SQOOP-1358. > It > >>>>>> needs to be reviewed by a committer and then committed if no > >>>>>> further changes are needed (or more work needed otherwise). > >>>>>> > >>>>>> Thanks for driving the release! > >>>>>> > >>>>>> Venkat > >>>>>> > >>>>>> On Fri, Jul 18, 2014 at 10:58 AM, Jarek Jarcec Cecho > >>>>>> <jar...@apache.org> > >>>>>> wrote: > >>>>>>> Seems reasonable to me, thank you for driving the release Abe! > >>>>>>> > >>>>>>> Jarcec > >>>>>>> > >>>>>>> On Jul 18, 2014, at 10:40 AM, Abraham Elmahrek <a...@cloudera.com> > >>>> wrote: > >>>>>>> > >>>>>>>> Hey guys, > >>>>>>>> > >>>>>>>> I haven't seen a -1 on this suggestion. So let's create a > >>>>>>>> release branch come Wednesday July 23rd. > >>>>>>>> > >>>>>>>> -Abe > >>>>>>>> > >>>>>>>> > >>>>>>>> On Wed, Jul 16, 2014 at 11:38 AM, Abraham Elmahrek > >>>>>>>> <a...@cloudera.com> > >>>>>> wrote: > >>>>>>>> > >>>>>>>>> Are there any objections to this branch date? > >>>>>>>>> > >>>>>>>>> -Abe > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon, Jul 14, 2014 at 1:22 PM, Abraham Elmahrek > >>>>>>>>> <a...@apache.org> > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hey guys, > >>>>>>>>>> > >>>>>>>>>> It looks like there are a couple of Jiras that are in progress > >>>>>>>>>> and slotted for the 1.4.5 release. If we aim for July 23rd to > >>>>>>>>>> create a > >>>>>> release > >>>>>>>>>> branch, does that give every one enough time to finish up what > >>>>>>>>>> they're working on? > >>>>>>>>>> > >>>>>>>>>> See https://issues.apache.org/jira/browse/SQOOP-1353 for more > >>>>>>>>>> information. > >>>>>>>>>> > >>>>>>>>>> -Abe > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>>>> -- > >>>>>> CONFIDENTIALITY NOTICE > >>>>>> NOTICE: This message is intended for the use of the individual or > >>>>>> entity to which it is addressed and may contain information that is > >>>>>> confidential, privileged and exempt from disclosure under > >>>>>> applicable law. If the reader of this message is not the intended > >>>>>> recipient, you are hereby notified that any printing, copying, > >>>>>> dissemination, distribution, disclosure or forwarding of this > >>>>>> communication is strictly prohibited. If you have received this > >>>>>> communication in error, please contact the sender immediately and > >>>>>> delete it from your > >>>> system. Thank You. > >>>>>> > >>>> > >>>> -- > >>>> CONFIDENTIALITY NOTICE > >>>> NOTICE: This message is intended for the use of the individual or > >>>> entity to which it is addressed and may contain information that is > >>>> confidential, privileged and exempt from disclosure under applicable > >>>> law. If the reader of this message is not the intended recipient, you > >>>> are hereby notified that any printing, copying, dissemination, > >>>> distribution, disclosure or forwarding of this communication is > >>>> strictly prohibited. If you have received this communication in error, > >>>> please contact the sender immediately and delete it from your system. > >> Thank You. > >>>> > >> > >> -- > >> CONFIDENTIALITY NOTICE > >> NOTICE: This message is intended for the use of the individual or > entity to > >> which it is addressed and may contain information that is confidential, > >> privileged and exempt from disclosure under applicable law. If the > reader > >> of this message is not the intended recipient, you are hereby notified > that > >> any printing, copying, dissemination, distribution, disclosure or > >> forwarding of this communication is strictly prohibited. If you have > >> received this communication in error, please contact the sender > immediately > >> and delete it from your system. Thank You. > >> > >
