Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22104
Just realized that the PR title and description is not updated. @icexelloss
can you update them? thanks!
---
-
To
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
Thanks all for the review!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95312/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95312 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95312/testReport)**
for PR 22104 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22104
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22104
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95317/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95317 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95317/testReport)**
for PR 22104 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95317 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95317/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95312 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95312/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95309/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95309 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95309/testReport)**
for PR 22104 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95309 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95309/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22104
can we make `ExtractPythonUDFs` a logical plan instead of physical? then
all the problems go away since it happens before the data source strategy.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22104
I mean, the current code will still break partitioned tables:
```
== Physical Plan ==
*(3) Project [_c0#223, pythonUDF0#231 AS v1#226]
+- BatchEvalPython [(0)], [_c0#223,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22104
@icexelloss, why
https://github.com/apache/spark/pull/22104/commits/ccb27bb1ab75e33913f37a4dbe84793e6b9ddeec
was reverted in this PR? Looks this is the correct approach.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95246/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95246 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95246/testReport)**
for PR 22104 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95246 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95246/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22104
Let me take another look today or tomorrow.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22104
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95171/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95171 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95171/testReport)**
for PR 22104 at commit
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
@HyukjinKwon I addressed the comments. Do you mind taking a another look?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95169/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95169 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95169/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95171 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95171/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #95169 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95169/testReport)**
for PR 22104 at commit
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
Tests pass now. This comment
https://github.com/apache/spark/pull/22104/files#r210414941 requires some
attention. @cloud-fan Do you think this is the right way to handle
GenericInternalRow
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94848/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94848 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94848/testReport)**
for PR 22104 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94848 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94848/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94822/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94822 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94822/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94826/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94826 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94826/testReport)**
for PR 22104 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94826 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94826/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94822 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94822/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94820 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94820/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94820/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94820 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94820/testReport)**
for PR 22104 at commit
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
Thanks @HyukjinKwon and @cloud-fan ! I will take a look
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22104
> we can implement a dummy data source v1/v2 at scala side
There's an example https://github.com/apache/spark/pull/21007 that
implement something in Scala and use it in Python side
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22104
@icexelloss we can implement a dummy data source v1/v2 at scala side and
scan them in PySpark test.
---
-
To unsubscribe,
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
I think another way to fix this is to move the logic to `ExtractPythonUDF`
to ignore `FileScanExec` `DataSourceScanExec` and `DataSourceV2ScanExec`
instead of changing all three rules. The
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
@gatorsmile Can you advise how to create a df with data source? All my
attempts end up triggering FileSourceStrategy not DataSourceStrategy
---
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
@gatorsmile Possibly, let me see if I can create a test case
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22104
@icexelloss Do we face the same issue for DataSourceStrategy?
---
-
To unsubscribe, e-mail:
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
retest please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94747/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94747 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94747/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/22104
cc @cloud-fan . Followed your suggestion here:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94747 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94747/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94746 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94746/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94746/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22104
**[Test build #94746 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94746/testReport)**
for PR 22104 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22104
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
92 matches
Mail list logo
