Github user JoshRosen closed the pull request at:
https://github.com/apache/spark/pull/2482
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is e
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-57572018
I'm going to close this for now. My approach has some confusing semantics
and may be more general than what most users need.
---
If your project is set up for it, you
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56316673
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20635/consoleFull)
for PR 2482 at commit
[`4882082`](https://github.com/a
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56315058
I've taken another pass at this. This time, I kept AsyncRDDActions but
re-implemented it using `runAsync`, but I'm actually on the fence about that
change. The one di
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56314921
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20635/consoleFull)
for PR 2482 at commit
[`4882082`](https://github.com/ap
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56312285
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20633/consoleFull)
for PR 2482 at commit
[`3171939`](https://github.com/a
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2482#discussion_r17827924
--- Diff: core/src/main/scala/org/apache/spark/RunAsyncResult.scala ---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under on
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2482#discussion_r17827908
--- Diff: core/src/main/scala/org/apache/spark/RunAsyncResult.scala ---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under on
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56310596
@rxin Do you have any examples of why a user might prefer the old model,
besides backwards-compatibility? I'd like to understand if the old model (in
its current form)
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56310259
Yes, I know that they are now Experimental, but they weren't always so,
since we didn't have the Experimental designation/policy when AsyncRDDActions
was introduced.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56310133
Fair enough, although the `AsyncRDDActions` class was marked as
`@Experimental` and the documentation for that annotation explicitly warns that
experimental APIs might
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56310100
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20633/consoleFull)
for PR 2482 at commit
[`3171939`](https://github.com/ap
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56310103
+1 @rxin
Just scanned through the code quickly, and I didn't immediately see
anything that would preclude retaining and deprecating the old code while
intro
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56309919
I don't think we can just wipe the old one out. At the very least, we need
to "deprecate" it. Even that is debatable because some applications might
prefer this async model.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56309826
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20632/consoleFull)
for PR 2482 at commit
[`c715511`](https://github.com/a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2482#issuecomment-56309790
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20632/consoleFull)
for PR 2482 at commit
[`c715511`](https://github.com/ap
GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/2482
[SPARK-3626] [WIP] Replace AsyncRDDActions with a more general runAsync()
mechanism
### Background
The `AsyncRDDActions` methods were introduced in
e33b1839e27249e232a2126cec67a38109e032
17 matches
Mail list logo
