Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7437#discussion_r34856887
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/UnsafeRowConverter.scala
---
@@ -261,6 +265,13 @@ private class StringUnsafeColumnWriter private()
extends BytesUnsafeColumnWriter
def getBytes(source: InternalRow, column: Int): Array[Byte] = {
source.getAs[UTF8String](column).getBytes
}
+ // TODO(davies): refactor this
+ // specialized for codegen
+ def getSize(value: UTF8String): Int =
+ ByteArrayMethods.roundNumberOfBytesToNearestWord(value.numBytes())
+ def write(target: UnsafeRow, value: UTF8String, column: Int, cursor:
Int): Int = {
+ write(target, value.getBytes, column, cursor)
+ }
}
private class BinaryUnsafeColumnWriter private() extends
BytesUnsafeColumnWriter {
--- End diff --
I've noticed that we have both objects and classes for each of these
ColumnWriters, but forget why I needed to do this. I tried it out in IntelliJ
and I think that we can just convert these classes into `object`s and drop the
`private object` lines near line 154.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
