Thank you RJ, Cos. it's really worth to think about the datagen and deployment. I will talk with team and get back soon.
-----Original Message----- From: RJ Nowling [mailto:[email protected]] Sent: Wednesday, January 27, 2016 12:09 PM To: [email protected] Subject: Re: HiBench as part of Bigtop If HiBench itself were contributed to Bigtop, maybe we could work on joining forces with our efforts on the Bigtop Data Generators? Currently, the data generators are used in the BigPetStore examples but also for some Kubernetes examples. We could share in the maintenance (and vice versa), and I would be love to talk about ways to reuse code. I'm open to adopting any code you might have or helping incorporating our code and data generators into HiBench. My strength tends to be building data generators (the math modeling and implementation) but my weakness is providing them in an easy-to-use form to end users. HiBench has done a great job of the latter. On Tue, Jan 26, 2016 at 10:00 PM, Konstantin Boudnik <[email protected]> wrote: > it depends in part of how you position HiBench. Ie, does it make sense > to make it part of the deployable stack and be able to install it on a > node to benchmark the whole cluster? Or there is a different > deployment scenario you have in mind? > > Cos > > On Wed, Jan 27, 2016 at 03:45AM, Cheng, Hao wrote: > > Which way does the community prefer? Just for packaging or the put > entire project into bigtop? > > > > -----Original Message----- > > From: RJ Nowling [mailto:[email protected]] > > Sent: Wednesday, January 27, 2016 11:24 AM > > To: [email protected] > > Subject: Re: HiBench as part of Bigtop > > > > A few of us have been working on BigPetStore, a family of example > > applications (blueprints). We've also been working on data > > generators to create relatively complex fake data for those demo apps. > > > > On Tue, Jan 26, 2016 at 9:10 PM, Cheng, Hao <[email protected]> wrote: > > > > > Hi RJ, > > > > > > Currently, we are targeting packaging the HiBench in Bigtop; but > > > is there any other subproject entirely be part of BigTop? > > > > > > Thanks, > > > Hao > > > > > > -----Original Message----- > > > From: RJ Nowling [mailto:[email protected]] > > > Sent: Wednesday, January 27, 2016 11:04 AM > > > To: [email protected] > > > Subject: Re: HiBench as part of Bigtop > > > > > > Hi Hao, > > > > > > Are you interested in just packaging HiBench in Bigtop or > > > contributing the entire project to Bigtop? > > > > > > Is there someone who would be willing to take responsibility for > > > maintenance? > > > > > > We generally like to find a maintainer for each package and add > > > them to our maintainers list. This way we know who to contact if > > > a build fails and blocks a release. If there is no maintainer > > > (and the build has problems), we use that as grounds for removing > > > the package from > Bigtop. > > > > > > Thanks! > > > RJ > > > > > > > > > On Tue, Jan 26, 2016 at 1:15 AM, Cheng, Hao <[email protected]> > wrote: > > > > > > > Thank you Cos for the reply, we will take a look at YCSB. :) > > > > > > > > Hao > > > > > > > > -----Original Message----- > > > > From: Konstantin Boudnik [mailto:[email protected]] > > > > Sent: Tuesday, January 26, 2016 2:33 PM > > > > To: [email protected] > > > > Subject: Re: HiBench as part of Bigtop > > > > > > > > Hi Hao. > > > > > > > > Thanks for your interest in the project and for considering us > > > > as the new home for this benchmarking tool! We already have YCSB > > > > as the part of stack, but clearly the more the merrier ;) > > > > > > > > A quick look at the source code shows that HiBench is already > > > > licensed under ASL2, so the rest of it should be just a SMOP ;) > > > > As you know, we do package all our components for both deb and > > > > rpm package managers, deployment code, as well as do have > > > > requirements for package and integration testing. I think the > > > > good starting point would be to take a look at YCSB > > > > > > > > bigtop-packages/src/*/ycsb/ > > > > bigtop-deploy/puppet/modules/ycsb/ > > > > > > > > and/or other components in the stack. This link > > > > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contri > > > > bute > > > > > > > > should also be helpful to understand the process of contributing > > > > into the project. And you already found dev@ list, so you'll set > > > > ;) > > > > > > > > I wonder how others in the community see this? > > > > Cos > > > > > > > > On Tue, Jan 26, 2016 at 05:37AM, Cheng, Hao wrote: > > > > > Dear BigTop Devs, > > > > > > > > > > I am from Intel Big Data Technology team, and we are the owner > > > > > of HiBench, an open source benchmark suite for Hadoop / Spark > > > > > ecosystem; as widely used, more and more trivial requirements > > > > > from HiBench users, due to the limited resources, particularly > > > > > the ease of deployment, we are exploring the possibility of > > > > > getting the code > > > into BigTop. > > > > > > > > > > HiBench code can be found at: > > > > > https://github.com/intel-hadoop/HiBench > > > > > > > > > > Looking forward to your reply. > > > > > > > > > > Regards, > > > > > Hao > > > > > > > > > > > > >
