Hi, We are in process of migrating Pig on spark. What is your currrent Spark setup? Version & cluster management that you use? Also what is the datasize you are working with right now. Regards Mayur
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Thu, May 22, 2014 at 8:19 PM, William Kang <weliam.cl...@gmail.com>wrote: > Hi, > We are moving into adopting the full stack of Spark. So far, we have used > Shark to do some ETL work, which is not bad but is not prefect either. We > ended writing UDF and UDGF, UDAF that can be avoided if we could use Pig. > > Do you have any suggestions with the ETL solution in Spark stack? > > And did any one have a working work flow management solution with Spark? > > Many thanks. > > > Cao >
