Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/20850#discussion_r175391993
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala
---
@@ -111,27 +111,27 @@ object GenerateUnsafeProjection extends
CodeGenerator[Seq[Expression], UnsafePro
s"""
// Remember the current cursor so that we can calculate how
many bytes are
// written later.
- final int $tmpCursor = $bufferHolder.cursor;
- ${writeStructToBuffer(ctx, input.value, t.map(_.dataType),
bufferHolder)}
- $rowWriter.setOffsetAndSize($index, $tmpCursor,
$bufferHolder.cursor - $tmpCursor);
+ final int $tmpCursor = $rowWriter.cursor();
--- End diff --
It seems a bit weird that we have to are storing state internal to the
`UnsafeWriter`/`BufferHolder` here. It would be very nice if we can internalize
this code into the `UnsafeWriter`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
