+1 for Beta given major performance improvement work went in Ozone Manager and Datanode Pipeline.
I have been testing Teragen runs and now we have consistent runs and performance is almost near to HDFS with disaggregated Storage and compute cluster. Thanks, Bharat On Sun, Feb 23, 2020 at 6:35 PM Sammi Chen <sammic...@apache.org> wrote: > +1, Impressive performance achievement on OzoneManager, let's move to > Beta. > > Bests, > Sammi Chen > > On Thu, Feb 20, 2020 at 4:17 AM Anu Engineer <aengin...@apache.org> wrote: > > > Hi All, > > > > > > I would like to propose moving Ozone from 'Alpha' tags to 'Beta' tags > when > > we do future releases. Here are a couple of reasons why I think we should > > make this move. > > > > > > > > 1. Ozone Manager or the Namenode for Ozone scales to more than 1 > billion > > keys. We tested this in our labs in an organic fashion; that is, we > were > > able to create more than 1 billion keys from external clients with no > > loss > > in performance. > > 2. The ozone Manager meets the performance and resource constraints > that > > we set out to achieve. We were able to sustain the same throughput at > > Ozone > > manager for over three days that took us to get this 1 billion keys. > > That > > is, we did not have to shut down or resize memory for the namenode as > we > > went through this exercise. > > 3. The most critical, we did this experiment with 64GB of memory > > allocation in JVM and 64 GB of RAM off-heap allocation. That is, the > > Ozone > > Manager was able to achieve this scale with far less memory footprint > > than > > HDFS. > > 4. Ozone's performance is at par with HDFS when running workloads like > > Hive ( > > > > > https://blog.cloudera.com/benchmarking-ozone-clouderas-next-generation-storage-for-cdp/ > > ) > > 5. We have been able to run long-running clusters with Ozone. > > > > > > Having achieved these goals, I propose that we move from the planned > > 0.4.2-Alpha release to 0.5.0-Beta as our next release. If we hear no > > concerns about this, we would like to move Ozone from Alpha to Beta > > releases. > > > > > > Thanks > > > > Anu > > > > > > P.S. I am CC-ing HDFS dev since many people who are interested in Ozone > > still have not subscribed to Ozone dev lists. My apologies if it feels > like > > spam, I promise that over time we will become less noisy in the HDFS > > channel. > > > > > > PPS. I know lots of you will want to know more specifics; Our blog > presses > > are working overtime and I promise you that you will get to see all the > > details pretty soon. > > >
