GitHub user kaka1992 opened a pull request:
https://github.com/apache/spark/pull/5870
[SPARK-7324][SQL] Add DataFrame.dropDuplicates
Similar to
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop_duplicates.html
def dropDuplicates(): DataFrame
def dropDuplicates(subset: Seq[String]): DataFrame
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kaka1992/spark dropDuplicates
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5870.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5870
----
commit d11d5b95ef82a208c579daa0073bdc072a682be5
Author: äºå³¤ <[email protected]>
Date: 2015-05-01T15:50:12Z
[SPARK-7294] ADD BETWEEN
commit baf839b4a4aa8d7d4ab8cdb1a5b82affd3ce376e
Author: äºå³¤ <[email protected]>
Date: 2015-05-02T01:39:17Z
[SPARK-7294] ADD BETWEEN
commit 7d623680d2c726a53b9e36c78f654e34c40f3dba
Author: äºå³¤ <[email protected]>
Date: 2015-05-02T06:17:10Z
[SPARK-7294] ADD BETWEEN
commit 76f0c511d0879af017f6fc10606a224b19d7d423
Author: äºå³¤ <[email protected]>
Date: 2015-05-02T13:57:53Z
Merge remote-tracking branch 'remotes/upstream/master'
commit f080f8d118f00e4f27936d55e74d391bac690c33
Author: äºå³¤ <[email protected]>
Date: 2015-05-02T14:00:12Z
update pep8
commit c6e49bcdb37c7e783849464858552293bcbea4fe
Author: äºå³¤ <[email protected]>
Date: 2015-05-03T03:09:20Z
undo
commit d6cc28d08eea876c96866b4c1626b49bf3ba4b73
Author: äºå³¤ <[email protected]>
Date: 2015-05-03T05:53:33Z
undo
commit aab51ef2e4aa0a97eba5324bb37dc2d08f55896a
Author: äºå³¤ <[email protected]>
Date: 2015-05-03T06:12:02Z
update
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
