GitHub user YanjieGao opened a pull request:
https://github.com/apache/spark/pull/1306
[SPARK-2373]RDD add span function (split an RDD to two RDD based on user's
function)]
def span(p: T => Boolean): (RDD[T], RDD[T])
Splits this RDD into a prefix/suffix pair according to a predicate .
returns
a pair consisting of the longest prefix of this RDD whose elements all
satisfy p, and the rest of this list.
JIRA:https://issues.apache.org/jira/browse/SPARK-2373
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/YanjieGao/spark rdd_span
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1306.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1306
----
commit e5bff618f09b0b33968e4c12b360e3d30f2878f9
Author: Yanjie Gao <[email protected]>
Date: 2014-06-20T07:20:12Z
Spark SQL basicOperator add Intersect operator
Hi all,
I want to submit a basic operator Intersect
For example , in sql case
select * from table1
intersect
select * from table2
So ,i want use this operator support this function in Spark SQL
This operator will return the the intersection of SparkPlan child table
RDD .
commit 469f099c510b20d0871c2a22927e65b48c968964
Author: Yanjie Gao <[email protected]>
Date: 2014-06-23T08:04:33Z
Update basicOperators.scala
commit 61e88e7db2c118023fe501a7381c51c3da7f3940
Author: Yanjie Gao <[email protected]>
Date: 2014-06-23T08:08:03Z
Update SqlParser.scala
commit d4ac5e559485e6f948100a7e6875831b7a7b46a4
Author: Yanjie Gao <[email protected]>
Date: 2014-06-23T08:10:11Z
Update HiveQl.scala
commit ac73e60ef80ca78b2bc63d0ecc45f4b2a963d13c
Author: Yanjie Gao <[email protected]>
Date: 2014-06-23T08:11:45Z
Update basicOperators.scala
commit 790765d915e7325a7dfdb46780c37a5e7b0bdf31
Author: Yanjie Gao <[email protected]>
Date: 2014-06-23T08:14:05Z
Update SparkStrategies.scala
commit 4dd453e2bf0d85b6cbfdfe703be403b83858818c
Author: Yanjie Gao <[email protected]>
Date: 2014-06-23T08:17:20Z
Update SQLQuerySuite.scala
commit e2b64be1a643d43748c90cfb341177f1157db15d
Author: Yanjie Gao <[email protected]>
Date: 2014-06-24T03:14:45Z
Update basicOperators.scala
commit f1288b46bb031fd34ed8d0217bcb4144d720d880
Author: Yanjie Gao <[email protected]>
Date: 2014-06-27T08:54:03Z
delete annotation
commit 0b4983723d39488b8a2ce7f3e13f5bdb1d25ac83
Author: Yanjie Gao <[email protected]>
Date: 2014-06-27T08:56:02Z
delete the annotation
commit bdc4a05f46f8dfdee7442be0230901cb7d1ef864
Author: Yanjie Gao <[email protected]>
Date: 2014-06-27T08:56:28Z
Update basicOperators.scala
commit f7961f6b9f839d58f5c5b1caf9702cd1e688fee7
Author: Yanjie Gao <[email protected]>
Date: 2014-06-27T10:29:33Z
update the line less than
commit 5e374c754c471e2a3485a4fcb68ba26f3af5dfbd
Author: YanjieGao <[email protected]>
Date: 2014-07-03T06:07:45Z
resolve conflict in SparkStrategies and basicOperator
commit a802ca88e1dda66e116af01520cadb965036c455
Author: YanjieGao <[email protected]>
Date: 2014-07-04T03:25:51Z
Merge remote branch 'upstream/master' into patch-5
commit 0c7cca5ea9c3e68758675c493570be87b38d346a
Author: YanjieGao <[email protected]>
Date: 2014-07-04T03:58:28Z
modify format problem
commit ea78f3397ce645f0680a59b814f8472db91c6adb
Author: YanjieGao <[email protected]>
Date: 2014-07-04T10:27:34Z
resolve conflict and add annotation on basicOperator and remove HiveQl
commit 1cfbfe6593ef939182d99481384cb1adb5990ad2
Author: YanjieGao <[email protected]>
Date: 2014-07-04T10:43:54Z
refomat some files
commit b1a641cc1697da147ef47a8d1fb7a61b5f6c5990
Author: YanjieGao <[email protected]>
Date: 2014-07-05T07:31:07Z
Merge remote branch 'upstream/master' into rdd_span
commit 8c4eafe3dd91810eb63287bc7d59e9cf9569956b
Author: YanjieGao <[email protected]>
Date: 2014-07-05T07:40:28Z
RDD add span function
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
