> On Jan. 4, 2016, 11:11 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala, > > line 204 > > <https://reviews.apache.org/r/41663/diff/2-3/?file=1181044#file1181044line204> > > > > I am a bit confused here. If the refreshJobModel() function actually > > returns a different jobModel than what's already set in jobModelRef, isn't > > the whole point is to replace it w/ the new one? > > Yi Pan (Data Infrastructure) wrote: > Ah, I saw it now. Actually, I think that we should keep this line in > initializaJobModel() method, but remove the following lines in > getCoordinator(): > {code} > val jobModel: JobModel = jobModelGenerator() > jobModelRef.set(jobModel) > {code}
The thing I noticed is that refreshJobModel and initializeJobModel are both private. The only time when the job model is getting refreshed is when getJobCoordinator is called. Now that I think about it, after the change to the JobServlet, we don't really make a call to refresh the job model. The only other place where we do a jobCoordinator.jobmodel() is in SamzaTaskManager. AT this point, we use the jobModel simply to invoke the readContainerLocality method on it. Technically, nothing refreshes the jobModel after initialization. Better discussed in person :) - Navina ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41663/#review112677 ----------------------------------------------------------- On Jan. 4, 2016, 10:40 p.m., Navina Ramesh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/41663/ > ----------------------------------------------------------- > > (Updated Jan. 4, 2016, 10:40 p.m.) > > > Review request for samza, Boris Shkolnik, Yan Fang, Chris Riccomini, Jake > Maes, Jagadish Venkatraman, Xinyu Liu, and Yi Pan (Data Infrastructure). > > > Bugs: SAMZA-843 > https://issues.apache.org/jira/browse/SAMZA-843 > > > Repository: samza > > > Description > ------- > > SAMZA-843 : Slow start of Samza jobs with large number of containers > > > Diffs > ----- > > > samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java > 87346bc9f3d92e3ae86ed264c82d12a4d27f3188 > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala > ddce1481cee41dfb997103dbe3d5df44a123564e > samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala > 112ec1c2935fb6c2ebe9b5d9090263cea94c0cf9 > > samza-core/src/main/scala/org/apache/samza/coordinator/server/JobServlet.scala > a3baddbe81c96f58e718e062e964485eaa2dc701 > samza-core/src/main/scala/org/apache/samza/util/Util.scala > 58fbb8f8177a109d35659a29ef6660e239334de2 > > samza-core/src/test/scala/org/apache/samza/container/TestSamzaContainer.scala > 365ff0a8ece808a8143ee3580f1f42238dd292d1 > > samza-core/src/test/scala/org/apache/samza/coordinator/TestJobCoordinator.scala > 80cccf3104e6e0ce7457303f40baa2f4fa807782 > > Diff: https://reviews.apache.org/r/41663/diff/ > > > Testing > ------- > > Added some additional unit tests > ./gradlew clean build > > > Thanks, > > Navina Ramesh > >
