[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Merging in master/branch-2.1. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16007 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69177/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16007 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16007 **[Test build #69177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69177/consoleFull)** for PR 16007 at commit [`2657d95`](https://github.com/apache/spark/commit/2657d955741299431f708c99584514e999ef90c4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Thanks - LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16007 **[Test build #69177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69177/consoleFull)** for PR 16007 at commit [`2657d95`](https://github.com/apache/spark/commit/2657d955741299431f708c99584514e999ef90c4). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 I see, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Yes! That's what I meant -- change it false and add some documentation and one require to force that contract. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 @rxin Sorry but finally we can change the nullable value to `false`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Alright I looked more into this -- I think your approach might be better actually. Can you add an require in `InputFileNameHolder.setInputFileName` to verify the input is not null, and then document in InputFileNameHolder to say the returned value should never be null, and empty string if it is unknown? Then we can change the nullable value to true for this expression. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 @rxin And also we should modify the generated code to check the value is null or not, shouldn't we? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16007 I see, I'll revert this and add the comment. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 I wouldn't change the default as it might break compatibility. That said, I don't think it is safe to just set this to non-nullable because it is a very implicit assumption, and setting it to be nullable is never "wrong". I'd add some comment explaining why it is nullable (e.g. "It depends on the semantics of the caller for InputFileNameHolder, and there is no guarantee that it won't be null") --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/16007
@rxin The default value is `""` (`UTF8String.fromString("")`) if input file
name is not set for now.
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/InputFileNameHolder.scala#L32
Should we change the default value to `null`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16007 Actually this can be nullable if input file name is not set, isn't it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16007 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16007 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69142/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16007 **[Test build #69142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69142/consoleFull)** for PR 16007 at commit [`4592c09`](https://github.com/apache/spark/commit/4592c09a6e3712474761db28471fb6e1a3a5148f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16007: [SPARK-18583][SQL] Fix nullability of InputFileName.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16007 **[Test build #69142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69142/consoleFull)** for PR 16007 at commit [`4592c09`](https://github.com/apache/spark/commit/4592c09a6e3712474761db28471fb6e1a3a5148f). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
