Github user CHOIJAEHONG1 commented on the pull request:
https://github.com/apache/spark/pull/7494#issuecomment-122939278
Unfortunately, not.
I guess the string in the testcase should be changed to be native form.
```
readString <- function(con) {
stringLen <- readInt(con)
raw <- readBin(con, raw(), stringLen, endian = "big")
string <- rawToChar(raw)
Encoding(string) <- "UTF-8"
enc2native(string)
}
```
```
1. Failure (at test_sparkSQL.R#432): collect() support Unicode characters
------
rdf$name[1] not equal to
"\354\225\210\353\205\225\355\225\230\354\204\270\354\232\224"
1 string mismatches:
x[1]: "\354\225\210\353\205\225\355\225\230\354\204\270\354\232\224"
y[1]: "<U+C548><U+B155><U+D558><U+C138><U+C694>"
2. Failure (at test_sparkSQL.R#433): collect() support Unicode characters
------
rdf$name[2] not equal to "\346\202\250\345\245\275"
1 string mismatches:
x[1]: "\346\202\250\345\245\275"
y[1]: "<U+60A8><U+597D>"
3. Failure (at test_sparkSQL.R#434): collect() support Unicode characters
------
rdf$name[3] not equal to
"\343\201\223\343\202\223\343\201\253\343\201\241\343\201\257"
1 string mismatches:
x[1]: "\343\201\223\343\202\223\343\201\253\343\201\241\343\201\257"
y[1]: "<U+3053><U+3093><U+306B><U+3061><U+306F>"
4. Failure (at test_sparkSQL.R#435): collect() support Unicode characters
------
rdf$name[4] not equal to "Xin ch\303\240o"
1 string mismatches:
x[1]: "Xin ch\303\240o"
y[1]: "Xin ch<U+00E0>o"
5. Error: collect() support Unicode characters
---------------------------------
Unsupported type for deserialization
1: withCallingHandlers(eval(code, new_test_environment), error =
capture_calls, message = function(c) invokeRestart("muffleMessage"),
warning = function(c) invokeRestart("muffleWarning"))
2: eval(code, new_test_environment)
3: eval(expr, envir, enclos)
4: expect_equal(collect(where(df2, df2$name ==
"\346\202\250\345\245\275"))$name, "\346\202\250\345\245\275") at
test_sparkSQL.R:438
5: expect_that(object, equals(expected, label = expected.label, ...), info
= info, label = label)
6: condition(object)
7: compare(expected, actual, ...)
8: compare.character(expected, actual, ...)
9: identical(x, y)
10: collect(where(df2, df2$name == "\346\202\250\345\245\275"))
11: collect(where(df2, df2$name == "\346\202\250\345\245\275"))
12: .local(x, ...)
13: lapply(listCols, function(col) {
objRaw <- rawConnection(col)
numRows <- readInt(objRaw)
col <- readCol(objRaw, numRows)
close(objRaw)
col
})
14: lapply(listCols, function(col) {
objRaw <- rawConnection(col)
numRows <- readInt(objRaw)
col <- readCol(objRaw, numRows)
close(objRaw)
col
})
15: FUN(X[[i]], ...)
16: readCol(objRaw, numRows)
17: do.call(c, lapply(1:numRows, function(x) {
value <- readObject(inputCon)
if (is.null(value))
NA
else value
}))
18: lapply(1:numRows, function(x) {
value <- readObject(inputCon)
if (is.null(value))
NA
else value
})
19: lapply(1:numRows, function(x) {
value <- readObject(inputCon)
if (is.null(value))
NA
else value
})
20: FUN(X[[i]], ...)
21: readObject(inputCon)
22: readTypedObject(con, type)
23: stop(paste("Unsupported type for deserialization", type))
24: .handleSimpleError(function (e)
{
e$calls <- head(sys.calls()[-seq_len(frame + 7)], -2)
signalCondition(e)
}, "Unsupported type for deserialization ", quote(readTypedObject(con,
type)))
Error: Test failures
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
