Hi Sun, Phoenix-Spark would be a nice addon if you can open source it. I am planning/thinking to using Spark on HBase for one of my project.
~Anil On Wed, Jan 7, 2015 at 12:17 AM, [email protected] < [email protected]> wrote: > Hi, > spark-phoenix integration would be great as Spark community is greately > active now and more > and more developers are using Apache Spark. > > Thanks, > Sun. > > ------------------------------ > ------------------------------ > > > *From:* James Taylor <[email protected]> > *Date:* 2015-01-07 16:10 > *To:* [email protected] > *Subject:* Re: Fwd: Phoenix in production > This is great, Sun! Thank you so much. Would you mind posting this on our > user list in response to Siddharth's email? I think other Phoenix users > would find it interesting as well. > > On a side note, not sure how general what you developed is, but it would > be interesting to pursue a general Spark integration in Phoenix as an open > source contribution. > > Thanks, > James > > On Tue, Jan 6, 2015 at 5:41 PM, [email protected] < > [email protected]> wrote: > >> Hi, James & Siddharth >> >> Glad to share our experience of using Phoenix in Production. I believe >> that Siddharth had done >> sufficient tests and practices about Phoenix performance. Here are some >> tips about how we are using >> Phoenix for our projects: >> 1. We facilitate Phoenix to give convinience for both RD and QA >> engineers, as they are glad to use >> standard sql to operate hbase with no much loss of query performance. >> 2. In Production environment, we mainly integrate Apache Spark with >> Phoenix to optimize data loading to >> Phoenix tables with or withour secondary indexes. Glad that current >> performance of writing had worked >> smoothly with both compared to previously used MySQL InfoBright and other >> sql schema. We both had tested >> a lot for secondary indexes and query optimization for Phoenix before >> moving Phoenix to Production >> environment. Now we can get most of the features worked for Phoenix in >> our job. >> 3. Challenges had a lot too, such as bulkload performance with wal >> enabled, query optimization, statistical data collection with Phoenix >> full table scan, and so on. However, we believe Phoenix be a sufficient >> schema for sql query over HBase and we are glad >> that even more our projects are considering using Phoenix. >> >> Thanks, >> Sun. >> >> ------------------------------ >> ------------------------------ >> >> CertusNet >> >> >> *From:* James Taylor <[email protected]> >> *Date:* 2015-01-07 09:10 >> *To:* [email protected] >> *Subject:* Fwd: Phoenix in production >> Hi Sun, >> Any experiences you can share with Siddharth? >> Thanks, >> James >> >> ---------- Forwarded message ---------- >> From: Siddharth Ubale <[email protected]> >> Date: Thu, Jan 1, 2015 at 11:21 PM >> Subject: Phoenix in production >> To: "[email protected]" <[email protected]> >> >> >> Hi Guys, >> >> >> >> We are seriously thinking of phoenix in Production environment , however, >> we have no much data of how Phoenix is behaving in production. >> >> Can anyone let us know if anyone is using Phoenix in Production and any >> challenges which they have experienced. >> >> >> >> Thanks, >> >> Siddharth Ubale, >> >> *Synchronized Communications * >> >> *#43, Velankani Tech Park, Block No. II, * >> >> *3rd Floor, Electronic City Phase I,* >> >> *Bangalore – 560 100* >> >> *Tel : +91 80 3202 4060 <%2B91%2080%203202%204060>* >> >> *Web:* *www.syncoms.com* <http://www.syncoms.com/> >> >> *[image: LogoNEWmohLARGE]* >> >> *London*|*Bangalore*|*Orlando* >> >> >> >> *we innovate, plan, execute, and transform the business* >> >> >> >> > -- Thanks & Regards, Anil Gupta
