Hello team The release process is complete, as per our acceptance criteria.
https://s.apache.org/systemds-release-validation Few tasks TODO: 1. Mark SystemDS 3.1.0 as released in the Jira application: - Go to https://issues.apache.org/jira/plugins/servlet/project-config/SYSTEMDS/versions - Hover over the released version and click Release 2. Add a tweet for 3.1.0 release. Please comment on any missing task. Apologies for typo in the first announcement email subject. Best regards Janardhan On Tue, Mar 7, 2023 at 11:07 PM Janardhan <janard...@apache.org> wrote: > > Here is the release acceptance criteria we are using: > > https://s.apache.org/systemds-release-validation (Google sheets link) > > Best regards > Janardhan > > On Tue, Mar 7, 2023 at 10:54 PM Janardhan <janard...@apache.org> wrote: > > > > Hi team, > > > > I guess verifying the release artifacts seems a bit daunting. > > Actually, it is pretty simple. > > > > We simply need to go to the artifact location, download it and test it > > in the way you would normally run SystemDS. > > > > Few things: > > - Your testing does not have to cover each feature area of the > > project, the features that you are comfortable with will do. > > - You do not have to test each and every artifact, please choose the > > ones that you feel are relevant to you. > > - Try your best about which features you are testing and you can vote > > based on that. > > - How much testing is necessary? - Try your best effort. > > > > Before sending the voting email there has been rigorous testing. Also, > > there are some automated checks at each process. Because these tests > > were run on machines which have residual configuration or there might > > be some overlook on things like a missing file, environmental variable > > or configuration files etc. > > > > Here are a basic steps to download the files and test them > > > > https://cwiki.apache.org/confluence/display/SYSTEMDS/Verifying+SystemDS+Release+Artifacts > > > > Best regards > > Janardhan > > > > > > On Thu, Mar 2, 2023 at 5:47 PM arnab phani <phaniar...@gmail.com> wrote: > > > > > > Hi Janardhan, let me know if you want to have a call during the rc cut. > > > I am back in Berlin and can have a call today. > > > > > > Regards, > > > Arnab.. > > > > > > On Thu, Mar 2, 2023 at 1:07 PM Matthias Boehm <mboe...@gmail.com> wrote: > > >> > > >> thanks Janardhan - sounds good to me. > > >> > > >> The perftest ran fine, except for the single-threaded read issue we send > > >> to user@hadoop but did not receive any feedback yet (and an exploration > > >> of different configurations did not yield improvements yet). > > >> > > >> Regards, > > >> Matthias > > >> > > >> On 3/2/2023 6:32 AM, Janardhan wrote: > > >> > Hello team > > >> > > > >> > I will start an rc tonight. > > >> > > > >> > Best regards > > >> > Janardhan > > >> > > > >> > On Sat, Feb 25, 2023 at 12:47 AM Janardhan <janard...@apache.org> > > >> > wrote: > > >> >> Hi Matthias > > >> >> > > >> >> Please do take time. I am also doing some preparations on my side. > > >> >> We can start the release process once the perf test is done. > > >> >> > > >> >> Thank you > > >> >> Janardhan > > >> >> > > >> >> On Sat, Feb 25, 2023 at 12:37 AM Matthias Boehm <mboe...@gmail.com> > > >> >> wrote: > > >> >>> unfortunately, I was a bit to eager - the clean run of the perftest > > >> >>> suite now showed additional issues, which we should fix. I would fix > > >> >>> what I'm aware of tonight and then kickoff the perftest suite for a > > >> >>> clean run over the weekend. > > >> >>> > > >> >>> Regards, > > >> >>> Matthias > > >> >>> > > >> >>> On 2/23/2023 11:01 PM, Matthias Boehm wrote: > > >> >>>> thanks for asking - I was just about to write it up. The perftest > > >> >>>> revealed a couple of configuration problems of our new hardware > > >> >>>> cluster, and one regression that I just fixed (but missed two > > >> >>>> test-related changes). Other than that the numbers look quite good: > > >> >>>> consistent moderate performance improvements across all algorithms > > >> >>>> and > > >> >>>> use cases. So once SYSTEMDS-3500 is closed, we can start cutting the > > >> >>>> first RC. Concurrently we will run another full pass of the perftest > > >> >>>> to keep the numbers for future comparisons. > > >> >>>> > > >> >>>> Regards, > > >> >>>> Matthias > > >> >>>> > > >> >>>> On 2/23/2023 10:53 PM, Janardhan wrote: > > >> >>>>> Hello team > > >> >>>>> > > >> >>>>> Did we face any issues during the performance testing? Shall we be > > >> >>>>> needing some time or can we go ahead with the release plan as > > >> >>>>> discussed? > > >> >>>>> > > >> >>>>> Best regards > > >> >>>>> > > >> >>>>> Janardhan > > >> >>>>> > > >> >>>>> On Fri, Feb 10, 2023 at 11:44 PM Matthias Boehm <mboe...@gmail.com> > > >> >>>>> wrote: > > >> >>>>>> thanks double checking - I'm almost done, but there is still a > > >> >>>>>> bug with > > >> >>>>>> unique() which I encountered while added the auc() builtin and I > > >> >>>>>> will > > >> >>>>>> resolve this issue in the next hours. > > >> >>>>>> > > >> >>>>>> Regards, > > >> >>>>>> Matthias > > >> >>>>>> > > >> >>>>>> On 2/10/2023 5:25 PM, Janardhan wrote: > > >> >>>>>>> Hello everyone, > > >> >>>>>>> > > >> >>>>>>> As planned before, we shall freeze code on the development branch > > >> >>>>>>> tonight (GMT), for the team to run performance testing suites. > > >> >>>>>>> > > >> >>>>>>> Are you in the middle of implementing some feature/bug fix? - If > > >> >>>>>>> yes, > > >> >>>>>>> would you be needing some time?. > > >> >>>>>>> > > >> >>>>>>> Kindly reply if you have any concerns. > > >> >>>>>>> > > >> >>>>>>> Best regards, > > >> >>>>>>> Janardhan > > >> >>>>>>> > > >> >>>>>>> On Sat, Jan 28, 2023 at 10:58 PM Janardhan <janard...@apache.org> > > >> >>>>>>> wrote: > > >> >>>>>>>> Thank you for the response. > > >> >>>>>>>> > > >> >>>>>>>> The feature freeze will be on Feb 10th, 2023 > > >> >>>>>>>> Tentative release date is Feb 24th, 2023 > > >> >>>>>>>> > > >> >>>>>>>> If you have pending PRs, now is the best time to get it running > > >> >>>>>>>> and > > >> >>>>>>>> merge it once the code looks good. After the feature freeze, > > >> >>>>>>>> any new > > >> >>>>>>>> features will go to the next release cycle. > > >> >>>>>>>> > > >> >>>>>>>> Best regards, > > >> >>>>>>>> Janardhan > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>>> On Fri, Jan 27, 2023 at 4:21 AM Matthias Boehm > > >> >>>>>>>> <mboe...@gmail.com> > > >> >>>>>>>> wrote: > > >> >>>>>>>>> indeed, thanks everybody for starting the discussion. From my > > >> >>>>>>>>> end, I'm > > >> >>>>>>>>> pressured under few deadlines until beginning of February right > > >> >>>>>>>>> now, but > > >> >>>>>>>>> would like to bring in a few built-ins to add missing > > >> >>>>>>>>> functionality for > > >> >>>>>>>>> model evaluation early Feb (until Feb 5 if that fits into the > > >> >>>>>>>>> release > > >> >>>>>>>>> date). How about a feature freeze Feb 10 and then a week or > > >> >>>>>>>>> two for > > >> >>>>>>>>> running the perftest suite? > > >> >>>>>>>>> > > >> >>>>>>>>> Regards, > > >> >>>>>>>>> Matthias > > >> >>>>>>>>> > > >> >>>>>>>>> On 1/26/2023 5:25 PM, arnab phani wrote: > > >> >>>>>>>>>> Thanks Janardhan for initiating the discussion. > > >> >>>>>>>>>> > > >> >>>>>>>>>> Regards, > > >> >>>>>>>>>> Arnab.. > > >> >>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> On Thu, Jan 26, 2023 at 6:21 AM Janardhan > > >> >>>>>>>>>> <janard...@apache.org> > > >> >>>>>>>>>> wrote: > > >> >>>>>>>>>> > > >> >>>>>>>>>>> Hi all, > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Let us initiate the release preparations. I'd like to > > >> >>>>>>>>>>> mention the > > >> >>>>>>>>>>> release date shortly after our discussion in this mail > > >> >>>>>>>>>>> thread. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Let us take this opportunity to wrap up any features recently > > >> >>>>>>>>>>> introduced. Hopefully, the features are functional and > > >> >>>>>>>>>>> reasonably > > >> >>>>>>>>>>> tested for the release. If not, you can mention here if you > > >> >>>>>>>>>>> would need > > >> >>>>>>>>>>> some assistance, and reviews. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> You can start with Updating version details in Jira as > > >> >>>>>>>>>>> follows: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> 1. Apply version > > >> >>>>>>>>>>> Affects Version/s: SystemDS 3.1 > > >> >>>>>>>>>>> Fix Version/s: SystemDS 3.1 > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> 2. Fill in Components the Jira is related to > > >> >>>>>>>>>>> Component/s: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> some of component names: > > >> >>>>>>>>>>> Documentation, Runtime, Compiler, Build, APIs, Algorithms, > > >> >>>>>>>>>>> Federated, Compression, Builtins, Frame, Matrix, GPU, CLA > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> -- > > >> >>>>>>>>>>> Helpful links > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> [1] all Jira components list: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> https://issues.apache.org/jira/projects/SYSTEMDS?selectedItem=com.atlassian.jira.jira-projects-plugin:components-page > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> [2] SystemDS Kanban Board: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=SYSTEMDS&rapidView=466 > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> [3] Release Process: > > >> >>>>>>>>>>> https://apache.github.io/systemds/site/release-process.html > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Best Regards, > > >> >>>>>>>>>>> Janardhan > > >> >>>>>>>>>>> > > >>
