Github user zhichao-li commented on a diff in the pull request:
https://github.com/apache/spark/pull/7113#discussion_r33651510
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math.scala
---
@@ -354,6 +356,35 @@ case class Pow(left: Expression, right: Expression)
}
}
+/**
+ * Performs the inverse operation of HEX.
+ * Resulting characters are returned as a byte array.
+ */
+case class UnHex(child: Expression)
+ extends UnaryExpression with AutoCastInputTypes with Serializable {
+
+ override def expectedChildTypes: Seq[DataType] = Seq(StringType)
+
+ override def dataType: DataType = BinaryType
+
+ override def eval(input: InternalRow): Any = {
+ val num = child.eval(input)
+ if (num == null) {
+ null
+ } else {
+ unhex(num.asInstanceOf[UTF8String])
+ }
+ }
+
+ private def unhex(utf8Str: UTF8String): Array[Byte] = {
+ try {
+ new
org.apache.commons.codec.binary.Hex(StandardCharsets.UTF_8).decode(utf8Str.getBytes)
--- End diff --
How about I reverse it back to the previous version by using
`utf8String.toString()` to convert byte array to char sequence? We need to do
this conversion even in the apache common lib
``` scala
public byte[] decode(final byte[] array) throws DecoderException {
return decodeHex(new String(array, getCharset()).toCharArray());
}
```
and there's limitation in that lib when the size is odd. It would throw
exception instead of padding 0 at the front end which adopted in hive .
``` scala
public static byte[] decodeHex(final char[] data) throws
DecoderException {
final int len = data.length;
if ((len & 0x01) != 0) {
throw new DecoderException("Odd number of characters.");
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
