Jackie-Jiang commented on code in PR #18140:
URL: https://github.com/apache/pinot/pull/18140#discussion_r3067554828
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/bloomfilter/BloomFilterHandler.java:
##########
@@ -126,7 +126,7 @@ private void
createAndSealBloomFilterForDictionaryColumn(File indexDir, ColumnMe
Dictionary dictionary = getDictionaryReader(columnMetadata,
segmentWriter)) {
int length = dictionary.length();
for (int i = 0; i < length; i++) {
- bloomFilterCreator.add(dictionary.getStringValue(i));
+ bloomFilterCreator.add(dataType.toString(dictionary.getInternal(i)));
Review Comment:
Do we need this special handling? What value do we send to bloom filter?
Based on current logic it should be the stored format.
If this is a bug, let's fix the bug separately
##########
pinot-spi/src/main/java/org/apache/pinot/spi/data/Schema.java:
##########
@@ -621,6 +622,9 @@ public void validate() {
String fieldName = fieldSpec.getName();
try {
validate(fieldType, dataType);
+ if (dataType == DataType.UUID && !fieldSpec.isSingleValueField()) {
Review Comment:
Where is the type check for BIG_DECIMAL? We should put them at the same
places.
We should update javadoc for this method
##########
pinot-spi/src/main/java/org/apache/pinot/spi/utils/UuidUtils.java:
##########
@@ -0,0 +1,144 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.spi.utils;
+
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+import java.util.UUID;
+
+
+/**
+ * Utilities for Pinot's logical UUID type.
+ *
+ * <p>UUID values are externally represented as canonical lowercase RFC 4122
strings and internally represented as
+ * fixed-width 16-byte values.
+ */
+public final class UuidUtils {
+ public static final int UUID_NUM_BYTES = 16;
+ public static final byte[] NIL_UUID_BYTES = new byte[UUID_NUM_BYTES];
Review Comment:
We usually use `NULL` over `NIL`
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/invertedindex/InvertedIndexHandler.java:
##########
@@ -181,7 +181,7 @@ private void
createInvertedIndexForColumn(SegmentDirectory.Writer segmentWriter,
ForwardIndexReaderContext readerContext =
forwardIndexReader.createContext()) {
if (columnMetadata.isSingleValue()) {
// Single-value column
- switch (columnMetadata.getDataType()) {
+ switch (columnMetadata.getDataType().getStoredType()) {
Review Comment:
This is a bug-fix. Let's make a separate fix for it.
`BIG_DECIMAL` is also not handled
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/BytesDictionary.java:
##########
@@ -47,7 +54,7 @@ public int indexOf(ByteArray bytesValue) {
@Override
public int insertionIndexOf(String stringValue) {
- return binarySearch(BytesUtils.toBytes(stringValue));
+ return binarySearch(parseBytes(stringValue));
Review Comment:
I don't see similar handling for TIMESTAMP in LongDictionary. Do we need
such special handling here?
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/dictionary/MutableDictionaryFactory.java:
##########
@@ -29,10 +29,11 @@ private MutableDictionaryFactory() {
public static MutableDictionary getMutableDictionary(DataType dataType,
boolean isOffHeapAllocation,
Review Comment:
The input is already stored type. Consider renaming it to `valueType` to be
consistent as `Dictionary.getValueType()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
