zhangweilst commented on code in PR #373:
URL: https://github.com/apache/paimon-cpp/pull/373#discussion_r4119625051
##########
src/paimon/core/casting/casting_utils.cpp:
##########
@@ -23,6 +23,19 @@
#include "paimon/common/utils/checked_cast.h"
namespace paimon {
+Result<std::shared_ptr<arrow::Array>> CastingUtils::DecodeDictionary(
+ const std::shared_ptr<arrow::Array>& array, arrow::MemoryPool* pool) {
+ if (array->type_id() != arrow::Type::DICTIONARY) {
+ return array;
+ }
+ const auto& dictionary_type = checked_cast<const
arrow::DictionaryType&>(*array->type());
+ std::shared_ptr<arrow::DataType> value_type = dictionary_type.value_type();
+ if (value_type->id() == arrow::Type::LARGE_STRING) {
+ value_type = arrow::utf8();
+ }
Review Comment:
Yes, the dictionary value type can be large_binary. But as suggested in
`src/paimon/common/utils/field_type_utils.h`, Paimon BLOB is expecting Arrow
LARGE_BINARY.
Converting large_binary to binary here would lose BLOB semantics, thus
LARGE_BINARY is intentionally preserved instead of being cast. And we have a
unit test making sure large binary is not cast.
This is different from casting LARGE_STRING to string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
