zero323 commented on a change in pull request #29947:
URL: https://github.com/apache/spark/pull/29947#discussion_r501502832
##########
File path: python/pyspark/sql/functions.pyi
##########
@@ -137,6 +137,8 @@ def sha1(col: ColumnOrName) -> Column: ...
def sha2(col: ColumnOrName, numBits: int) -> Column: ...
def hash(*cols: ColumnOrName) -> Column: ...
def xxhash64(*cols: ColumnOrName) -> Column: ...
+def assert_true(col: ColumnOrName, errMsg: Union[Column, str] = ...): ...
Review comment:
Two small notes (sorry for being late):
- I think we should annotate return type for `assert_true` - it will type
check because of implicit `Any`, but I think it is better to avoid such cases
```python
def assert_true(col: ColumnOrName, errMsg: Union[Column, str] = ...) ->
Column: ...
```
- For `def raise_error` I'd use
[`NoReturn`](https://docs.python.org/3/library/typing.html#typing.NoReturn):
```python
from typing import NoReturn
def raise_error(errMsg: Union[Column, str]) -> NoReturn: ...
```
##########
File path: python/pyspark/sql/functions.pyi
##########
@@ -137,6 +137,8 @@ def sha1(col: ColumnOrName) -> Column: ...
def sha2(col: ColumnOrName, numBits: int) -> Column: ...
def hash(*cols: ColumnOrName) -> Column: ...
def xxhash64(*cols: ColumnOrName) -> Column: ...
+def assert_true(col: ColumnOrName, errMsg: Union[Column, str] = ...): ...
Review comment:
> * For `def raise_error` I'd use
[`NoReturn`](https://docs.python.org/3/library/typing.html#typing.NoReturn):
This might indicate intention here, though technically speaking it's still a
`Column`, so
```python
def raise_error(errMsg: Union[Column, str]) -> Column: ...
```
is still correct (and literal one). Do you have any thoughts about it
@HyukjinKwon?
##########
File path: R/pkg/R/functions.R
##########
@@ -826,6 +826,55 @@ setMethod("xxhash64",
column(jc)
})
+#' @details
+#' \code{assert_true}: Returns null if the input column is true; throws an
exception
+#' with the provided error message otherwise.
+#'
+#' @param errMsg (optional) The error message to be thrown.
+#'
+#' @rdname column_misc_functions
+#' @aliases assert_true assert_true,Column-method
+#' @examples
+#' \dontrun{
+#' tmp <- mutate(df, v1 = assert_true(df$vs < 2),
+#' v2 = assert_true(df$vs < 2, "custom error message"),
+#' v3 = assert_true(df$vs < 2, df$vs))
+#' head(tmp)}
+#' @note assert_true since 3.1.0
+setMethod("assert_true",
+ signature(x = "Column"),
+ function(x, errMsg = NULL) {
+ jc <- if (is.null(errMsg)) {
+ callJStatic("org.apache.spark.sql.functions", "assert_true",
x@jc)
+ } else {
+ if (is.character(errMsg) && length(errMsg) == 1) {
Review comment:
Shouldn't we throw an exception if `length(errMsg) != 1`? Just in case
user does something like this?
```
> assert_true(column("foo"), c("foo", "bar"))
Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
trying to get slot "jc" from an object of a basic class ("character") with
no slots
```
##########
File path: R/pkg/R/functions.R
##########
@@ -826,6 +826,55 @@ setMethod("xxhash64",
column(jc)
})
+#' @details
+#' \code{assert_true}: Returns null if the input column is true; throws an
exception
+#' with the provided error message otherwise.
+#'
+#' @param errMsg (optional) The error message to be thrown.
+#'
+#' @rdname column_misc_functions
+#' @aliases assert_true assert_true,Column-method
+#' @examples
+#' \dontrun{
+#' tmp <- mutate(df, v1 = assert_true(df$vs < 2),
+#' v2 = assert_true(df$vs < 2, "custom error message"),
+#' v3 = assert_true(df$vs < 2, df$vs))
+#' head(tmp)}
+#' @note assert_true since 3.1.0
+setMethod("assert_true",
+ signature(x = "Column"),
+ function(x, errMsg = NULL) {
+ jc <- if (is.null(errMsg)) {
+ callJStatic("org.apache.spark.sql.functions", "assert_true",
x@jc)
+ } else {
+ if (is.character(errMsg) && length(errMsg) == 1) {
Review comment:
Shouldn't we throw an exception if `length(errMsg) != 1`? Just in case
user does something like this?
```R
> assert_true(column("foo"), c("foo", "bar"))
Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
trying to get slot "jc" from an object of a basic class ("character") with
no slots
```
##########
File path: R/pkg/R/functions.R
##########
@@ -826,6 +826,55 @@ setMethod("xxhash64",
column(jc)
})
+#' @details
+#' \code{assert_true}: Returns null if the input column is true; throws an
exception
+#' with the provided error message otherwise.
+#'
+#' @param errMsg (optional) The error message to be thrown.
+#'
+#' @rdname column_misc_functions
+#' @aliases assert_true assert_true,Column-method
+#' @examples
+#' \dontrun{
+#' tmp <- mutate(df, v1 = assert_true(df$vs < 2),
+#' v2 = assert_true(df$vs < 2, "custom error message"),
+#' v3 = assert_true(df$vs < 2, df$vs))
+#' head(tmp)}
+#' @note assert_true since 3.1.0
+setMethod("assert_true",
+ signature(x = "Column"),
+ function(x, errMsg = NULL) {
+ jc <- if (is.null(errMsg)) {
+ callJStatic("org.apache.spark.sql.functions", "assert_true",
x@jc)
+ } else {
+ if (is.character(errMsg) && length(errMsg) == 1) {
Review comment:
Shouldn't we throw an exception if `length(errMsg) != 1`? Just in case
user does something like this?
```R
> assert_true(column("foo"), c("foo", "bar"))
Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
trying to get slot "jc" from an object of a basic class ("character") with
no slots
```
i.e.
```R
} else {
if (is.character(errMsg) {
stopifnot(length(errMsg) == 1)
```
##########
File path: R/pkg/R/functions.R
##########
@@ -826,6 +826,55 @@ setMethod("xxhash64",
column(jc)
})
+#' @details
+#' \code{assert_true}: Returns null if the input column is true; throws an
exception
+#' with the provided error message otherwise.
+#'
+#' @param errMsg (optional) The error message to be thrown.
+#'
+#' @rdname column_misc_functions
+#' @aliases assert_true assert_true,Column-method
+#' @examples
+#' \dontrun{
+#' tmp <- mutate(df, v1 = assert_true(df$vs < 2),
+#' v2 = assert_true(df$vs < 2, "custom error message"),
+#' v3 = assert_true(df$vs < 2, df$vs))
+#' head(tmp)}
+#' @note assert_true since 3.1.0
+setMethod("assert_true",
+ signature(x = "Column"),
+ function(x, errMsg = NULL) {
+ jc <- if (is.null(errMsg)) {
+ callJStatic("org.apache.spark.sql.functions", "assert_true",
x@jc)
+ } else {
+ if (is.character(errMsg) && length(errMsg) == 1) {
Review comment:
Shouldn't we throw an exception if `length(errMsg) != 1`? Just in case
user does something like this?
```R
> assert_true(column("foo"), c("foo", "bar"))
Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
trying to get slot "jc" from an object of a basic class ("character") with
no slots
```
i.e.
```R
...
} else {
if (is.character(errMsg) {
stopifnot(length(errMsg) == 1)
...
```
##########
File path: R/pkg/R/functions.R
##########
@@ -826,6 +826,55 @@ setMethod("xxhash64",
column(jc)
})
+#' @details
+#' \code{assert_true}: Returns null if the input column is true; throws an
exception
+#' with the provided error message otherwise.
+#'
+#' @param errMsg (optional) The error message to be thrown.
+#'
+#' @rdname column_misc_functions
+#' @aliases assert_true assert_true,Column-method
+#' @examples
+#' \dontrun{
+#' tmp <- mutate(df, v1 = assert_true(df$vs < 2),
+#' v2 = assert_true(df$vs < 2, "custom error message"),
+#' v3 = assert_true(df$vs < 2, df$vs))
+#' head(tmp)}
+#' @note assert_true since 3.1.0
+setMethod("assert_true",
+ signature(x = "Column"),
+ function(x, errMsg = NULL) {
+ jc <- if (is.null(errMsg)) {
+ callJStatic("org.apache.spark.sql.functions", "assert_true",
x@jc)
+ } else {
+ if (is.character(errMsg) && length(errMsg) == 1) {
Review comment:
In practice we make this check anyway, so it is only a question if we do
something about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
