RE: [Spark Namespace]: Expanding Spark ML under Different Namespace?

2017-03-03 Thread Shouheng Yi
/browse/SPARK-19498 From: Tim Hunter [mailto:timhun...@databricks.com] Sent: Friday, February 24, 2017 9:08 AM To: Joseph Bradley mailto:jos...@databricks.com>> Cc: Steve Loughran mailto:ste...@hortonworks.com>>; Shouheng Yi mailto:sho...@microsoft.com.invalid>>; Apache Sp

RE: [Spark Namespace]: Expanding Spark ML under Different Namespace?

2017-03-03 Thread Shouheng Yi
/browse/SPARK-19498 From: Tim Hunter [mailto:timhun...@databricks.com] Sent: Friday, February 24, 2017 9:08 AM To: Joseph Bradley Cc: Steve Loughran ; Shouheng Yi ; Apache Spark Dev ; Markus Weimer ; Rogan Carr ; Pei Jiang ; Miruna Oprescu Subject: Re: [Spark Namespace]: Expanding Spark ML under

[Spark Namespace]: Expanding Spark ML under Different Namespace?

2017-02-22 Thread Shouheng Yi
Hi Spark developers, Currently my team at Microsoft is extending Spark's machine learning functionalities to include new learners and transformers. We would like users to use these within spark pipelines so that they can mix and match with existing Spark learners/transformers, and overall have

A New Global Numerical Optimization Algo

2015-11-05 Thread Shouheng Yi
Hi all, I hacked a new global optimizer that is based on Differential Evolution and a local search algorithm. I can make it to solve 1000+ dimensional problems on a cluster within a very reasonable error and time. It's using Akka's actor system, so it scales very well and very easily just by tweak