Re: [ANNOUNCE] Apache Hadoop 3.3.1 release
Thank you Wei-Chiu! Best regards Xun Liu On Wed, Jun 16, 2021 at 11:03 AM Brahma Reddy Battula wrote: > Hi Wei-Chiu, > > Thanks for driving this.. > > > Looks index.html is not updated and Changelog is broken. > > > On Wed, Jun 16, 2021 at 7:59 AM Wei-Chiu Chuang > wrote: > > > Hi All, > > > > It gives me great pleasure to announce that the Apache Hadoop community > has > > voted to release Apache Hadoop 3.3.1. > > > > This is the first stable release of Apache Hadoop 3.3.x line. It contains > > 697 bug fixes, improvements and enhancements since 3.3.0. > > > > Users are encouraged to read the overview of major changes > > <https://hadoop.apache.org/docs/r3.3.1/index.html> since 3.3.0. For > > details > > of 697 bug fixes, improvements, and other enhancements since the previous > > 3.3.0 release, please check release notes > > < > > > http://hadoop.apache.org/docs/r3.3.1/hadoop-project-dist/hadoop-common/release/3.3.1/RELEASENOTES.3.3.1.html > > > > > and changelog > > < > > > http://hadoop.apache.org/docs/r3.3.1/hadoop-project-dist/hadoop-common/release/3.3.1/CHANGES.3.3.1.html > > > > > detail > > the changes since 3.3.0. > > > > Many thanks to everyone who contributed to the release, and everyone in > the > > Apache Hadoop community! This release is a direct result of your great > > contributions. > > > > Many thanks to everyone who helped in this release process! > > > > Many thanks to Sean Busbey, Chao Sun, Steve Loughran, Masatake Iwasaki, > > Michael Stack, Viraj Jasani, Eric Payne, Ayush Saxena, Vinayakumar B, > > Takanobu Asanuma, Xiaoqiao He and other folks who continued helps for > this > > release process. > > > > Best Regards, > > Wei-Chiu Chuang > > > > > -- > > > > --Brahma Reddy Battula >
Re: [ANNOUNCE] Hui Fei is a new Apache Hadoop Committer
Hui Fei, Congratulations! On Thu, Sep 24, 2020 at 2:07 AM Wei-Chiu Chuang wrote: > I am pleased to announce that Hui Fei has accepted the invitation to become > a Hadoop committer. > > He started contributing to the project in October 2016. Over the past 4 > years he has contributed a lot in HDFS, especially in Erasure Coding, > Hadoop 3 upgrade, RBF and Standby Serving reads. > > One of the biggest contributions is Hadoop 2->3 rolling upgrade support. > This was a major blocker for any existing Hadoop users to adopt Hadoop 3. > The adoption of Hadoop 3 has gone up after this. In the past the community > discussed a lot about Hadoop 3 rolling upgrade being a must-have, but no > one took the initiative to make it happen. I am personally very grateful > for this. > > The work on EC is impressive as well. He managed to onboard EC in > production at scale, fixing tricky problems. Again, I am impressed and > grateful for the contribution in EC. > > In addition to code contributions, he invested a lot in the community: > > > > >- Apache Hadoop Community 2019 Beijing Meetup > > > https://blogs.apache.org/hadoop/entry/hadoop-community-meetup-beijing-aug > where > >he discussed the operational experience of RBF in production > > > > > >- Apache Hadoop Storage Community Sync Online > > > https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit#heading=h.irqxw1iy16zo > where > >he discussed the Hadoop 3 rolling upgrade support > > > > > Let's congratulate Hui for this new role! > > Cheers, > Wei-Chiu Chuang (on behalf of the Apache Hadoop PMC) >
Re: Sep. Y2020 Hadoop Meetup in China
hi This is good news. The hadoop meetup held in Beijing last year was very successful and attracted many participants. Looking forward to it very much !!! On Wed, Sep 16, 2020 at 1:36 PM 俊平堵 wrote: > Hi, > Last year Aug, we had a very successful meetup in Beijing ( > https://blogs.apache.org/hadoop) which help to promote hadoop user > adoption > and development locally. This year, we would like to do it again in > Shanghai on 9/26. > So if you have topic to share on hadoop development or some relevant > interesting case. Please nominate it by replying this letter or contact me > directly. Slot is limited, first come first serve. Thanks! > BTW, this meetup is plan to be offline. If many people want to share > but cannot come to Shanghai by then, we can plan something online later. > > Thanks, > > Junping >
Re: [VOTE] Moving Submarine to a separate Apache project proposal
+1 Hello everyone, I am a member of the submarine development team. I have been contributing to submarine for more than a year. I have seen the progress of submarine development very fast. In more than a year, there are 9 long-term developers of different companies. Contributing, submarine cumulative code has more than 200,000 lines of code, is growing very fast, and is used in the production environment of multiple companies. In the submarine development group, there are 5 PMCs and 7committer members from Hadoop, spark, zeppelin projects. They are very familiar with the development process and specifications of the apache community, and can well grasp the project development progress and project quality. So I recommend submarine to be a TLP project directly. We will continue to contribute to the submarine project. :-) Xun Liu Regards On Tue, 3 Sep 2019 at 12:01, Devaraj K wrote: > +1 > > Thanks Wangda for the proposal. > I would like to participate in this project, Please add me also to the > project. > > Regards > Devaraj K > > On Mon, Sep 2, 2019 at 8:50 PM zac yuan wrote: > > > +1 > > > > Submarine will be a complete solution for AI service development. It can > > take advantage of two best cluster systems: yarn and k8s, which will help > > more and more people get AI ability. To be a separate Apache project, > will > > accelerate the procedure of development apparently. > > > > Look forward to a big success in submarine project~ > > > > 朱林浩 于2019年9月3日周二 上午10:38写道： > > > > > +1, > > > Hopefully, that will become the top project, > > > > > > I also hope to make more contributions to this project. > > > > > > > > > > > > > > > > > > > > > > > > > > > At 2019-09-03 09:26:53, "Naganarasimha Garla" < > > naganarasimha...@apache.org> > > > wrote: > > > >+ 1, > > > >I would also like start participate in this project, hope to get > > > myself > > > >added to the project. > > > > > > > >Thanks and Regards, > > > >+ Naga > > > > > > > >On Tue, Sep 3, 2019 at 8:35 AM Wangda Tan > wrote: > > > > > > > >> Hi Sree, > > > >> > > > >> I put it to the proposal, please let me know what you think: > > > >> > > > >> The traditional path at Apache would have been to create an > incubator > > > >> > project, but the code is already being released by Apache and most > > of > > > the > > > >> > developers are familiar with Apache rules and guidelines. In > > > particular, > > > >> > the proposed PMC has 2 Apache TLP PMCs and proposed initial > > committers > > > >> > have 4 Apache TLP PMCs (from 3 different companies). They will > > provide > > > >> > oversight and guidance for the developers that are less > experienced > > in > > > >> the > > > >> > Apache Way. Therefore, the Submarine project would like to propose > > > >> becoming > > > >> > a Top Level Project at Apache. > > > >> > > > > >> > > > >> To me, putting to TLP has mostly pros, it is an easier process (same > > as > > > ORC > > > >> spin-off from Hive), much less overhead to both dev community and > > Apache > > > >> side. > > > >> > > > >> Thanks, > > > >> Wangda > > > >> > > > >> On Sun, Sep 1, 2019 at 2:04 PM Sree Vaddi > > > wrote: > > > >> > > > >> > +1 to move submarine to separate apache project. > > > >> > > > > >> > It is not clear in the proposal, if submarine majority voted to > move > > > to a > > > >> > seperate apache project, > > > >> > will it go through the incubation and TLP (top level project) > later > > ? > > > >> > 1. Incubation > > > >> > pros and cons > > > >> > efforts towards making it a TLP > > > >> > > > > >> > 2. direct TLP > > > >> > > > > >> > > > > >> > Thank you. > > > >> > /Sree > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > On Saturday, August 31, 2019, 10:19:06 PM PDT, Wangda Tan < > > > >
Re: Any thoughts making Submarine a separate Apache project?
@adam.antal The submarine development team has completed the following preparations: 1. Established a temporary test repository on Github. 2. Change the package name of hadoop submarine from org.hadoop.submarine to org.submarine 3. Combine the Linkedin/TonY code into the Hadoop submarine module; 4. On the Github docked travis-ci system, all test cases have been tested; 5. Several Hadoop submarine users completed the system test using the code in this repository. 赵欣 于2019年7月22日周一 上午9:38写道： > Hi > > I am a teacher at Southeast University (https://www.seu.edu.cn/). We are > a major in electrical engineering. Our teaching teams and students use > bigoop submarine for big data analysis and automation control of electrical > equipment. > > Many thanks to the hadoop community for providing us with machine learning > tools like submarine. > > I wish hadoop submarine is getting better and better. > > > == > 赵欣 > 东南大学电气工程学院 > > - > > Zhao XIN > > School of Electrical Engineering > > == > 2019-07-18 > > > *From:* Xun Liu > *Date:* 2019-07-18 09:46 > *To:* xinzhao > *Subject:* Fwd: Re: Any thoughts making Submarine a separate Apache > project? > > > -- Forwarded message - > 发件人： dashuiguailu...@gmail.com > Date: 2019年7月17日周三 下午3:17 > Subject: Re: Re: Any thoughts making Submarine a separate Apache project? > To: Szilard Nemeth , runlin zhang < > runlin...@gmail.com> > Cc: Xun Liu , common-dev , > yarn-dev , hdfs-dev < > hdfs-...@hadoop.apache.org>, mapreduce-dev < > mapreduce-dev@hadoop.apache.org>, submarine-dev < > submarine-...@hadoop.apache.org> > > > +1 ，Good idea, we are very much looking forward to it. > > -- > dashuiguailu...@gmail.com > > > *From:* Szilard Nemeth > *Date:* 2019-07-17 14:55 > *To:* runlin zhang > *CC:* Xun Liu ; Hadoop Common > ; yarn-dev ; > Hdfs-dev ; mapreduce-dev > ; submarine-dev > > *Subject:* Re: Any thoughts making Submarine a separate Apache project? > +1, this is a very great idea. > As Hadoop repository has already grown huge and contains many projects, I > think in general it's a good idea to separate projects in the early phase. > > > On Wed, Jul 17, 2019, 08:50 runlin zhang wrote: > > > +1 ，That will be great ！ > > > > > 在 2019年7月10日，下午3:34，Xun Liu 写道： > > > > > > Hi all, > > > > > > This is Xun Liu contributing to the Submarine project for deep learning > > > workloads running with big data workloads together on Hadoop clusters. > > > > > > There are a bunch of integrations of Submarine to other projects are > > > finished or going on, such as Apache Zeppelin, TonY, Azkaban. The next > > step > > > of Submarine is going to integrate with more projects like Apache > Arrow, > > > Redis, MLflow, etc. & be able to handle end-to-end machine learning use > > > cases like model serving, notebook management, advanced training > > > optimizations (like auto parameter tuning, memory cache optimizations > for > > > large datasets for training, etc.), and make it run on other platforms > > like > > > Kubernetes or natively on Cloud. LinkedIn also wants to donate TonY > > project > > > to Apache so we can put Submarine and TonY together to the same > codebase > > > (Page #30. > > > > > > https://www.slideshare.net/xkrogen/hadoop-meetup-jan-2019-tony-tensorflow-on-yarn-and-beyond#30 > > > ). > > > > > > This expands the scope of the original Submarine project in exciting > new > > > ways. Toward that end, would it make sense to create a separate > Submarine > > > project at Apache? This can make faster adoption of Submarine, and > allow > > > Submarine to grow to a full-blown machine learning platform. > > > > > > There will be lots of technical details to work out, but any initial > > > thoughts on this? > > > > > > Best Regards, > > > Xun Liu > > > > > > - > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > > > > >
Any thoughts making Submarine a separate Apache project?
Hi all, This is Xun Liu contributing to the Submarine project for deep learning workloads running with big data workloads together on Hadoop clusters. There are a bunch of integrations of Submarine to other projects are finished or going on, such as Apache Zeppelin, TonY, Azkaban. The next step of Submarine is going to integrate with more projects like Apache Arrow, Redis, MLflow, etc. & be able to handle end-to-end machine learning use cases like model serving, notebook management, advanced training optimizations (like auto parameter tuning, memory cache optimizations for large datasets for training, etc.), and make it run on other platforms like Kubernetes or natively on Cloud. LinkedIn also wants to donate TonY project to Apache so we can put Submarine and TonY together to the same codebase (Page #30. https://www.slideshare.net/xkrogen/hadoop-meetup-jan-2019-tony-tensorflow-on-yarn-and-beyond#30 ). This expands the scope of the original Submarine project in exciting new ways. Toward that end, would it make sense to create a separate Submarine project at Apache? This can make faster adoption of Submarine, and allow Submarine to grow to a full-blown machine learning platform. There will be lots of technical details to work out, but any initial thoughts on this? Best Regards, Xun Liu
