Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/12337
@kevincox, It seems inactive for few months. Should this be maybe closed
for now if you are currently not able to proceed this further?
---
If your project is set up for it, you can reply to
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/12337
+1 to this feature! I think this might be the first step in a better story
for people trying to use `nullable = false` as an enforcement mechanism (I'd
bring this idea up on
Github user kevincox commented on the issue:
https://github.com/apache/spark/pull/12337
The thing is that if you to `take(1)` you have to evaluate the dataframe
(at least partially) to see if there are any nulls. It has very significant
overhead. This version, which works inline,
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/12337
I mean it seems like you could do a filter with the current functions and
take(1) to get the same effect? Is it a common use case that you want an
exception thrown for a null value rather than
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/12337
**[Test build #3331 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3331/consoleFull)**
for PR 12337 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/12337
**[Test build #3331 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3331/consoleFull)**
for PR 12337 at commit
Github user kevincox commented on the issue:
https://github.com/apache/spark/pull/12337
@holdenk The point is that this is inline. It doesn't require evaluating
the whole dataframe and counting the nulls you find. Instead you use this on a
column and it asserts that every value
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/12337
So I'm curious what is the intention of this versus the existing isnull
function? It seems like this might not be needed in which case we should
consider closing the PR & JR (of course if there is
