Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/incubator-phoenix/pull/8#discussion_r9894185
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/expression/ArrayConstructorExpression.java
---
@@ -62,27 +67,62 @@ public void reset() {
position = 0;
Arrays.fill(elements, null);
}
-
+
@Override
public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) {
- for (int i = position >= 0 ? position : 0; i < elements.length;
i++) {
- Expression child = children.get(i);
- if (!child.evaluate(tuple, ptr)) {
- if (tuple != null && !tuple.isImmutable()) {
- if (position >= 0) position = i;
- return false;
+ try {
+ int offset = 0;
+ // track the elementlength for variable array
+ int elementLength = 0;
+ for (int i = position >= 0 ? position : 0; i <
elements.length; i++) {
+ Expression child = children.get(i);
+ if (!child.evaluate(tuple, ptr)) {
+ if (tuple != null && !tuple.isImmutable()) {
+ if (position >= 0) position = i;
+ return false;
+ }
+ } else {
+ // track the offset position here from the size of the
byteStream
+ if (!baseType.isFixedWidth() ||
baseType.isCoercibleTo(PDataType.VARCHAR)) {
+ Bytes.putInt(offsetArr, offset,
(byteStream.size()));
+ offset += Bytes.SIZEOF_INT;
+ elementLength += ptr.getLength();
+ }
+ if (!child.isStateless()) {
+ oStream.write(ptr.get(), ptr.getOffset(),
ptr.getLength());
+ } else {
+ oStream.write(ptr.get());
+ }
}
- } else {
- elements[i] = baseType.toObject(ptr, child.getDataType(),
child.getSortOrder());
+ }
+ if (position >= 0) position = elements.length;
+ if (!baseType.isFixedWidth() ||
baseType.isCoercibleTo(PDataType.VARCHAR)) {
+ oStream.writeByte(0);
--- End diff --
Actually, I've come up with a way of handling this (i.e. being able to
directly compare the byte array without the header throwing anything off).
Encode a null element values as a null byte followed by the number of
consecutive null elements as the next byte. For example, a single null element
would be represented as 00 01, while five consecutive null elements would be
represented as 00 05. Your offsets would all point to the null byte, so they'd
all decode fine. Then, terminate the element data with a double null byte. In
this way, we can't possibly have anything smaller than the double null byte
within our data. This solves the case where you're comparing two arrays of
different lengths, one with trailing null elements:
a, b, null, null would become: 65 00 66 00 00 02 00 00 <followed by the
header info>
a, b would become: 65 00 66 00 00 00 <followed by the header info>
So there's never a case where we "accidentally" start comparing bytes in
the header, but instead we'll always stop before that if the elements run out
or are different.
Make sense?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
[email protected] or file a JIRA ticket with INFRA.
---
