Github user MaxGekk commented on a diff in the pull request:
    --- Diff: 
    @@ -361,6 +361,15 @@ class JacksonParser(
             // For such records, all fields other than the field configured by
             // `columnNameOfCorruptRecord` are set to `null`.
             throw BadRecordException(() => recordLiteral(record), () => None, 
    +      case e: CharConversionException if options.encoding.isEmpty =>
    +        val msg =
    +          """Failed to parse a character. Encoding was detected 
    --- End diff --
    ok, speaking about this concrete exception handling. The exception with the 
message is thrown ONLY when options.encoding.isEmpty is `true`. It means 
`encoding` is not set and actual encoding of a file was autodetected. The `msg` 
says about that actually:  `Encoding was detected automatically`.
    Maybe `encoding` was detected correctly but the file contains a wrong char. 
In that case, the first sentence says this `Failed to parse a character`. The 
same could happen if you set `encoding` explicitly because you cannot guarantee 
that inputs are alway correct.
    > I think automatic detection is true only when multuline is enabled.
    Wrong char in input file can be in a file with UTF-8 read with `multiline = 
false` and in a file in UTF-16LE with `multiline = true`.
    My point is the mention of the `multiline` option in the error message 
doesn't help to user to solve the issue. A possible solution is to set 
`encoding` explicitly - what the message says actually.


To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to