GitHub user gczsjdy opened a pull request:
https://github.com/apache/spark/pull/18632
Reset BufferHolder while initialize an UnsafeRowWriter
## What changes were proposed in this pull request?
`UnsafeRowWriter`'s construtor should contain `BufferHolder.reset` to make
the writer out of the box.
While writing `UnsafeRow` using `UnsafeRowWriter`, developers should
manually call `BufferHolder.reset` to make the `BufferHolder`'s `cursor`(which
indicates where to write variable length portion) right in order to write
variable length fields like UTF8String, byte[], etc.
If a developer doesn't reuse the `BufferHolder` so maybe he never noticed
`reset` and the comments in code, the `UnsafeRow` won't be correct if he also
writes variable length UTF8String. This API design doesn't make sense. We
should reset the `BufferHolder` to make `UnsafeRowWriter` out of the box, but
not let user read the code and manually call it.
## How was this patch tested?
`makeKeyRow(long k1, String k2)` in `RowBasedKeyValueBatchSuite` already
tested it.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gczsjdy/spark unsaferow_buf
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18632.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18632
----
commit a0985404363f2975bf673e37306d0bd1c700a4d0
Author: GuoChenzhao <[email protected]>
Date: 2017-07-14T02:17:23Z
Reset BufferHolder while initialize an UnsafeRowWriter
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
