kiszk commented on a change in pull request #24795: [SPARK-27945][SQL] Minimal
changes to support columnar processing
URL: https://github.com/apache/spark/pull/24795#discussion_r291232137
##########
File path:
sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java
##########
@@ -50,19 +50,44 @@
@Evolving
public abstract class ColumnVector implements AutoCloseable {
+ private int refCount = 1;
+
/**
* Returns the data type of this column vector.
*/
public final DataType dataType() { return type; }
+ /**
+ * Increment the reference count for this vector. This is an implementation
detail and
+ * only BoundReference should call this directly.
+ * @return this for easy chaining.
+ */
+ public final ColumnVector incRefCount() {
+ refCount++;
+ return this;
+ }
+
/**
* Cleans up memory for this column vector. The column vector is not usable
after this.
+ * In reality it decrements the reference count and if it reaches 0 the
resources are released
+ * but this is an implementation detail that most code should just ignore.
*
* This overwrites `AutoCloseable.close` to remove the `throws` clause, as
column vector is
* in-memory and we don't expect any exception to happen during closing.
*/
@Override
- public abstract void close();
+ public final void close() {
+ refCount--;
+ if (refCount == 0) {
+ doClose();
+ }
+ }
+
+ /**
+ * Actually cleans up memory for this column vector. The column vector is
really not usable after
+ * this.
+ */
+ protected abstract void doClose();
Review comment:
Can we use more self-declarative name? This is because `columnVector` is
public API for developers who want to support their storage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
