Thanks David. Anyone else wants to contribute something to the release notes? It is time to wrap it up.
Thanks, Thomas On Tue, Dec 6, 2016 at 3:14 PM, David Yan <da...@datatorrent.com> wrote: > For the work in the Windowed Operator that is included in the 3.6 release: > > There is now an option to use spillable data structures for the state > storage of the Windowed Operator. This enables the Windowed Operator to > store large states and perform efficient checkpointing. > > We also did benchmarking on WindowedOperator with the spillable data > structures. From the result of our findings, we improved greatly how > objects are serialized and reduced garbage collection considerably in the > Managed State layer. > > Work is still in progress for purging state that is not needed any more and > further improving the performance of Managed State that the spillable data > structures depend on. > > Siyuan and Bright, please add to this if I'm missing anything. > > David > > On Fri, Dec 2, 2016 at 8:55 PM, Thomas Weise <t...@apache.org> wrote: > > > The release is available already. For the announcement, would anyone want > > to provide some information that can go on the blog page, similar to what > > we had last time: > > > > https://blogs.apache.org/apex/entry/apache_apex_malhar_3_5 > > > > Thanks, > > Thomas > > > > > > On Tue, Nov 22, 2016 at 2:14 PM, Thomas Weise <t...@apache.org> wrote: > > > > > IMO we should not delay the release any longer. I will cut the RC > > tomorrow > > > unless there is an objection. > > > > > > Thomas > > > > > > > > > On Tue, Nov 22, 2016 at 2:05 PM, David Yan <da...@datatorrent.com> > > wrote: > > > > > >> Thomas, > > >> > > >> We need more time for the timebucket change and the purging feature, > and > > >> we > > >> need to do benchmarking after those features have been implemented. > > >> Is it possible to postpone the release? If not, we can shoot for > > >> completing > > >> the features for the 3.7.0 release. > > >> > > >> David > > >> > > >> On Tue, Nov 22, 2016 at 12:08 PM, Thomas Weise <t...@apache.org> > wrote: > > >> > > >> > David, Siyuan, > > >> > > > >> > Any update, I would like to cut the RC. > > >> > > > >> > Thanks > > >> > > > >> > On Sun, Nov 20, 2016 at 10:23 PM, Siyuan Hua < > siy...@datatorrent.com> > > >> > wrote: > > >> > > > >> > > Yes yes, > > >> > > We need to put the clear plan upfront > > >> > > > > >> > > This ticket was thought and scheduled to be done after the > > >> > APEXMALHAR-2271 > > >> > > SpillableSetMultimap which bucketize elements on the time from > > values. > > >> > > But per discussion with David, we realized this ticket is more > > >> important > > >> > > and affect the general purging functionality so I just jumped to > > this > > >> > issue > > >> > > late last week. > > >> > > Now the status is the pull request is there. I feel it can be > merged > > >> > soon. > > >> > > https://github.com/apache/apex-malhar/pull/503 > > >> > > > > >> > > Regards, > > >> > > Siyuan > > >> > > > > >> > > On Sat, Nov 19, 2016 at 12:56 AM, Thomas Weise <t...@apache.org> > > >> wrote: > > >> > > > > >> > > > David, > > >> > > > > > >> > > > APEXMALHAR-2301 is still open. Are we on track to complete the > > >> JIRAs by > > >> > > > Monday or should we move to next release? > > >> > > > > > >> > > > Siyuan, > > >> > > > > > >> > > > There is a question on the ticket from a month ago that wasn't > > >> > > addressed. I > > >> > > > think there needs to be more visibility on the thought process > and > > >> > > > decisions made, otherwise it is very difficult for others to > > follow. > > >> > Open > > >> > > > collaboration is key. > > >> > > > > > >> > > > Thanks > > >> > > > > > >> > > > > > >> > > > On Thu, Nov 17, 2016 at 8:29 PM, David Yan < > da...@datatorrent.com > > > > > >> > > wrote: > > >> > > > > > >> > > > > Hi Thomas, > > >> > > > > > > >> > > > > We would like to finish the following tickets if possible for > > the > > >> > 3.6.0 > > >> > > > > release: > > >> > > > > > > >> > > > > https://issues.apache.org/jira/browse/APEXMALHAR-2301 (ETA: > > >> 11/18) > > >> > > > > https://issues.apache.org/jira/browse/APEXMALHAR-2339 (ETA: > > >> 11/21) > > >> > > > > https://issues.apache.org/jira/browse/APEXMALHAR-2345 (ETA: > > >> 11/21) > > >> > > > > > > >> > > > > For https://issues.apache.org/jira/browse/APEXMALHAR-2271, > > since > > >> > this > > >> > > > only > > >> > > > > affects Session Windows and still quite some work needs to be > > done > > >> > > there, > > >> > > > > we would like to defer that to the next release. > > >> > > > > > > >> > > > > Siyuan and Bright, please also chime in if I'm missing > anything. > > >> > > > > > > >> > > > > David > > >> > > > > > > >> > > > > On Thu, Nov 17, 2016 at 2:49 AM, Thomas Weise <t...@apache.org > > > > >> > wrote: > > >> > > > > > > >> > > > > > David, > > >> > > > > > > > >> > > > > > Any update WRT APEXMALHAR-2130 > > >> > > > > > <https://issues.apache.org/jira/browse/APEXMALHAR-2130> / > > >> > > > > APEXMALHAR-2301 > > >> > > > > > <https://issues.apache.org/jira/browse/APEXMALHAR-2301> ? > > >> > > > > > > > >> > > > > > I would like to cut first RC by end of the week. > > >> > > > > > > > >> > > > > > Thanks > > >> > > > > > > > >> > > > > > On Wed, Nov 16, 2016 at 12:44 AM, Thomas Weise < > > t...@apache.org> > > >> > > wrote: > > >> > > > > > > > >> > > > > > > It has been a while and the issues that we were waiting > for > > >> are > > >> > > still > > >> > > > > > > unresolved. > > >> > > > > > > > > >> > > > > > > https://issues.apache.org/jira/issues/?jql=fixVersion% > > >> > > > > > > 20%3D%203.6.0%20AND%20project%20%3D%20APEXMALHAR%20and% > > >> > > > > > 20resolution%20%3D% > > >> > > > > > > 20unresolved%20ORDER%20BY%20status%20ASC > > >> > > > > > > > > >> > > > > > > I would suggest: > > >> > > > > > > > > >> > > > > > > APEXMALHAR-2300 <https://issues.apache.org/ > > >> > > > jira/browse/APEXMALHAR-2300 > > >> > > > > > > > >> > > > > > - > > >> > > > > > > defer > > >> > > > > > > APEXMALHAR-2130 <https://issues.apache.org/ > > >> > > > jira/browse/APEXMALHAR-2130 > > >> > > > > > > > >> > > > > > / > > >> > > > > > > APEXMALHAR-2301 <https://issues.apache.org/ > > >> > > > jira/browse/APEXMALHAR-2301 > > >> > > > > > > > >> > > > > > - > > >> > > > > > > David/Siyuan can you please give an update and > > recommendation. > > >> > > > > > > APEXMALHAR-2284 <https://issues.apache.org/ > > >> > > > jira/browse/APEXMALHAR-2284 > > >> > > > > > > > >> > > > > > - > > >> > > > > > > see discussion on the PR and JIRA. Also discussed it with > > >> Bhupesh > > >> > > and > > >> > > > > > > Chinmay today and we think that this join operator should > > >> > probably > > >> > > be > > >> > > > > > > replaced with the new operator that is based on > > >> WindowOperator. > > >> > > > > > > APEXMALHAR-2203 <https://issues.apache.org/ > > >> > > > jira/browse/APEXMALHAR-2203 > > >> > > > > > > > >> > > > > > - > > >> > > > > > > defer > > >> > > > > > > > > >> > > > > > > Please provide any feedback you may have within a day so > > that > > >> we > > >> > > can > > >> > > > > > > unblock the release. > > >> > > > > > > > > >> > > > > > > Thanks, > > >> > > > > > > Thomas > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > On Mon, Nov 7, 2016 at 6:36 AM, Chaitanya Chebolu < > > >> > > > > > > chaita...@datatorrent.com> wrote: > > >> > > > > > > > > >> > > > > > >> Hi Thomas, > > >> > > > > > >> > > >> > > > > > >> I am working on APEXMALHAR-2284 and will open a PR in > > >> couple > > >> > of > > >> > > > > days. > > >> > > > > > >> > > >> > > > > > >> Regards, > > >> > > > > > >> Chaitanya > > >> > > > > > >> > > >> > > > > > >> On Sun, Nov 6, 2016 at 10:51 PM, Thomas Weise < > > >> t...@apache.org> > > >> > > > wrote: > > >> > > > > > >> > > >> > > > > > >> > Is anyone working on APEXMALHAR-2284 ? > > >> > > > > > >> > > > >> > > > > > >> > On Fri, Oct 28, 2016 at 11:00 AM, Bhupesh Chawda < > > >> > > > > > >> bhup...@datatorrent.com> > > >> > > > > > >> > wrote: > > >> > > > > > >> > > > >> > > > > > >> > > +1 > > >> > > > > > >> > > > > >> > > > > > >> > > ~ Bhupesh > > >> > > > > > >> > > > > >> > > > > > >> > > On Fri, Oct 28, 2016 at 2:29 PM, Tushar Gosavi < > > >> > > > > > >> tus...@datatorrent.com> > > >> > > > > > >> > > wrote: > > >> > > > > > >> > > > > >> > > > > > >> > > > +1 > > >> > > > > > >> > > > > > >> > > > > > >> > > > - Tushar. > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Fri, Oct 28, 2016 at 12:52 PM, Aniruddha > Thombare > > >> > > > > > >> > > > <anirud...@datatorrent.com> wrote: > > >> > > > > > >> > > > > +1 > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > Thanks, > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > A > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > _____________________________________ > > >> > > > > > >> > > > > Sent with difficulty, I mean handheld ;) > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > On 28 Oct 2016 12:30 pm, "Siyuan Hua" < > > >> > > > siy...@datatorrent.com > > >> > > > > > > > >> > > > > > >> > wrote: > > >> > > > > > >> > > > > > > >> > > > > > >> > > > >> +1 > > >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> Thanks！ > > >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> Sent from my iPhone > > >> > > > > > >> > > > >> > > >> > > > > > >> > > > >> > On Oct 26, 2016, at 13:11, Thomas Weise < > > >> > > t...@apache.org> > > >> > > > > > >> wrote: > > >> > > > > > >> > > > >> > > > >> > > > > > >> > > > >> > Hi, > > >> > > > > > >> > > > >> > > > >> > > > > > >> > > > >> > I'm proposing another release of Malhar in > > >> November. > > >> > > > There > > >> > > > > > are > > >> > > > > > >> 49 > > >> > > > > > >> > > > issues > > >> > > > > > >> > > > >> > marked for the release, including important > bug > > >> > fixes, > > >> > > > new > > >> > > > > > >> > > > documentation, > > >> > > > > > >> > > > >> > SQL support and the work for windowed operator > > >> state > > >> > > > > > >> management: > > >> > > > > > >> > > > >> > > > >> > > > > > >> > > > >> > https://issues.apache.org/ > > >> > jira/issues/?jql=fixVersion% > > >> > > > > > >> > > > >> 20%3D%203.6.0%20AND%20project% > > >> > > 20%3D%20APEXMALHAR%20ORDER% > > >> > > > > > >> > > > >> 20BY%20status%20ASC > > >> > > > > > >> > > > >> > > > >> > > > > > >> > > > >> > Currently there is at least one blocker, the > > join > > >> > > > operator > > >> > > > > is > > >> > > > > > >> > broken > > >> > > > > > >> > > > >> after > > >> > > > > > >> > > > >> > change in managed state. It also affects the > SQL > > >> > > feature. > > >> > > > > > >> > > > >> > > > >> > > > > > >> > > > >> > Thanks, > > >> > > > > > >> > > > >> > Thomas > > >> > > > > > >> > > > >> > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
