> On March 5, 2014, 6:03 p.m., Purshotam Shah wrote: > > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 962 > > <https://reviews.apache.org/r/18762/diff/1/?file=510568#file510568line962> > > > > I see some issue with this. Earlier we start from end and break the for > > loop if nominalInstanceCal.compareTo(initInstance). > > There are still some instance being return, but not in this case. > > > > Eg > > <input-events> > > <data-in name="input" dataset="raw-logs"> > > <start-instance>${coord:current(-200)}</start-instance> > > <end-instance>${coord:current(0)}</end-instance> > > </data-in> > > </input-events> > > > > Where coord:current(-10) < initInstance. > > > > Earlier it will return -9 to 0, but now none. > > > > > >
Actually, instead of returning partial instances and user wondering what went wrong with this coord action, we should add this check at coord submit. Coord submit should fail if the instances span before/after dataset time interval > On March 5, 2014, 6:03 p.m., Purshotam Shah wrote: > > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java, line 961 > > <https://reviews.apache.org/r/18762/diff/1/?file=510568#file510568line961> > > > > This can be moved outside from for loop. since you are starting from > > start and it's increasing. I will add check in coord submit. So, this check won't be required anymore - shwethags ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18762/#review36237 ----------------------------------------------------------- On March 5, 2014, 5:28 a.m., shwethags wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18762/ > ----------------------------------------------------------- > > (Updated March 5, 2014, 5:28 a.m.) > > > Review request for oozie. > > > Bugs: OOZIE-1709 > https://issues.apache.org/jira/browse/OOZIE-1709 > > > Repository: oozie-git > > > Description > ------- > > CoordELFunctions.getCurrentInstance() has this code: > while (current.compareTo(calEffectiveTime) <= 0) { > current = (Calendar) origCurrent.clone(); > instanceCount[0]++; > current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * > dsFreq); > } > > For coords with smaller frequency and start time in very past, this is very > expensive. On prod, we have seen materialisation of each instance taking few > mins sometimes for coords with 1 min frequency > > > Diffs > ----- > > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java d73bc7d > core/src/test/java/org/apache/oozie/coord/TestCoordELFunctions.java 34a428f > > Diff: https://reviews.apache.org/r/18762/diff/ > > > Testing > ------- > > UT > > > Thanks, > > shwethags > >
