[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14680 Thanks @HyukjinKwon You're helping me a lot! I'll work on the unit test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14680 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14680 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64105/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14680 **[Test build #64105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64105/consoleFull)** for PR 14680 at commit [`e780208`](https://github.com/apache/spark/commit/e780208aa6361de34983b244cdb0ac4544c1645c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14680 Thanks for bearing with me. That was just my personal opinion. As you already know, I can't decide what should be added into Spark. BTW, we should fix https://github.com/apache/spark/blob/e50efd53f073890d789a8448f850cc219cca7708/sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala#L715-L724This fix is being failed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14680 How about now @HyukjinKwon ? The more I look at it the more I think it should calculated automatically out of the class name when constructor's called. It's of little to no value to a FileFormat developer. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14680 **[Test build #64105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64105/consoleFull)** for PR 14680 at commit [`e780208`](https://github.com/apache/spark/commit/e780208aa6361de34983b244cdb0ac4544c1645c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14680 @jaceklaskowski It seems the test [here] (https://github.com/apache/spark/blob/e50efd53f073890d789a8448f850cc219cca7708/sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala#L715-L724) is related with this change. It seems it will passes the test if we change `TextFileFormat` to `TEXT`. BTW, how about changing them to `Parquet` and `Text` maybe? I believe this might be about personal taste though.. I feel like `shortName.toUpperCase` is not always the string representation of each data source. I mean.. if my understanding is correct, the proper name might be `Parquet` rather than `PARQUET`, at least. It seems `ORC`, `JSON` and `CSV` are correct names because they are abbreviated names but I feel like it is questionable for `PARQUET` and `TEXT`. If the purpose of this change is only to see the information about plans to human via `explain(...)` regardless of anything, it might be better if it is more close to human readable and correct names as string representation. This is just my personal opinion. I think we need @rxin 's sign off here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user jaceklaskowski commented on the issue: https://github.com/apache/spark/pull/14680 @rxin @HyukjinKwon Mind reviewing it again and letting me know what you think? I know it's minor but would greatly appreciate having it merged at your earliest convenience. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14680 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63935/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14680 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14680 **[Test build #63935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63935/consoleFull)** for PR 14680 at commit [`52f5ba5`](https://github.com/apache/spark/commit/52f5ba57c27959d1abea2f8f456ba006d241d0c2). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14680: [SPARK-17101][SQL] Provide consistent format identifiers...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14680 **[Test build #63935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63935/consoleFull)** for PR 14680 at commit [`52f5ba5`](https://github.com/apache/spark/commit/52f5ba57c27959d1abea2f8f456ba006d241d0c2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
