Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
Folks, thanks for merging this PR.
@gatorsmile, re: a request to backport to 2.1. Is this something that
happens automatically? If not, could you point me to some instructions on how
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/18309
oh, Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
@maropu yes, I've submitted a fix
[here](https://github.com/apache/spark/pull/18415).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/18309
@gatorsmile This failure is related to this change?
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78575/console
---
If your project is set up for it, you can reply to this
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
Could you submit a backport PR to 2.1? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
Thanks! Merging to master/2.2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78387/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78387 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78387/testReport)**
for PR 18309 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78386/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78386 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78386/testReport)**
for PR 18309 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78387 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78387/testReport)**
for PR 18309 at commit
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@gatorsmile, thank you for review. All the comments should be addressed
now. Please, check it out.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78386 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78386/testReport)**
for PR 18309 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78359/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78359 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78359/testReport)**
for PR 18309 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
Except the comments about test cases, the other parts look good to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78359 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78359/testReport)**
for PR 18309 at commit
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78343/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78343 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78343/testReport)**
for PR 18309 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78343 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78343/testReport)**
for PR 18309 at commit
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78328/
Test FAILed.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78328 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78328/testReport)**
for PR 18309 at commit
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
crap, this clobbered the pypy install. fixing this now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@shaneknapp, I can't think of why PySpark unit tests failed other than
flakiness of the tests or infrastructure. Any chance you could trigger a re-run?
---
If your project is set up for it, you
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78298/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78298 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78298/testReport)**
for PR 18309 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78298 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78298/testReport)**
for PR 18309 at commit
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
Thank you, @shaneknapp .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/18309
i'll re-trigger this test once jenkins is back up.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78292/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78292 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78292/testReport)**
for PR 18309 at commit
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@gatorsmile, thanks for clarifying. I added a test to analyze a table with
only some of partitions being visible. Would you take another look?
---
If your project is set up for it, you can
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
@mbasmanova
```
create table ${fullTableName} using parquet options (path "${fullPath}")
```
If the path points to the directory of a partitioned table, the partitions
are
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78250/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78250 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78250/testReport)**
for PR 18309 at commit
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@gatorsmile, re: additional test, how can I create a table with a mix of
visible and invisible partitions?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78250 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78250/testReport)**
for PR 18309 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78231/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78231 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78231/testReport)**
for PR 18309 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
Could you add another test case? First, creates an external partitioned
data source table (with an external location). Second, add only a subset of
partitions. Third, analyze table but the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78231 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78231/testReport)**
for PR 18309 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
LGTM except some minor comments.
ping @cloud-fan @gatorsmile Could you start Jenkins for this pr?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@wzhfy, thank you careful review and helpful suggestions. I think I
incorporated all of your comments. Would you take another look?
---
If your project is set up for it, you can reply to this
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@wzhfy , @tejasapatil , I updated description of this PR, enhanced the test
and renamed a function I missed before in Tejas's comment. How does it look now?
---
If your project is set up for
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
@mbasmanova Let's first fix the above comments and ping admins to unblock
this pr :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
Please also update description of PR, the key point of the problem is URI
for table is different from that of its partitions, otherwise, it works fine
for partitioned tables.
---
If your project is
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@rxin, could you help unblock Jenkins for this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mbasmanova commented on the issue:
https://github.com/apache/spark/pull/18309
@wzhfy, @tejasapatil, thanks for your feedback. I think I addressed all of
your comments and added a test. Please, take another look.
---
If your project is set up for it, you can reply to this
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
Can you add a test case? In the test, we can add partitions with different
paths by ALTER TABLE SET LOCATION command. I think that can reproduce your
scenario, right?
---
If your project is set up
Github user wzhfy commented on the issue:
https://github.com/apache/spark/pull/18309
Please explain the failed scenario in PR description. That's why our test
cases cannot catch it and we need to test it manually.
---
If your project is set up for it, you can reply to this email and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18309
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
70 matches
Mail list logo
