rok commented on a change in pull request #8510:
URL: https://github.com/apache/arrow/pull/8510#discussion_r722227879
##########
File path: cpp/src/arrow/extension_type_test.cc
##########
@@ -333,4 +334,144 @@ TEST_F(TestExtensionType, ValidateExtensionArray) {
ASSERT_OK(ext_arr4->ValidateFull());
}
+class TensorArray : public ExtensionArray {
+ public:
+ using ExtensionArray::ExtensionArray;
+};
+
+class TensorArrayType : public ExtensionType {
+ public:
+ explicit TensorArrayType(const std::shared_ptr<DataType>& type,
+ const std::vector<int64_t>& shape,
+ const std::vector<int64_t>& strides)
+ : ExtensionType(type), type_(type), shape_(shape), strides_(strides) {}
+
+ std::shared_ptr<DataType> type() const { return type_; }
+ std::vector<int64_t> shape() const { return shape_; }
+ std::vector<int64_t> strides() const { return strides_; }
+
+ std::string extension_name() const override {
+ std::stringstream s;
+ s << "ext-array-tensor-type<type=" << *storage_type() << ", shape=(";
+ for (uint64_t i = 0; i < shape_.size(); i++) {
+ s << shape_[i];
+ if (i < shape_.size() - 1) {
+ s << ", ";
+ }
+ }
+ s << "), strides=(";
+ for (uint64_t i = 0; i < strides_.size(); i++) {
+ s << strides_[i];
+ if (i < strides_.size() - 1) {
+ s << ", ";
+ }
+ }
+ s << ")>";
+ return s.str();
+ }
+
+ bool ExtensionEquals(const ExtensionType& other) const override {
+ return this->shape() == static_cast<const TensorArrayType&>(other).shape();
Review comment:
I removed the `ndim` check and made equality comparison only check for
type name (`ext-array-tensor-type`).
> > In that case do we even want to keep ndim for equality comparison?
>
> This is a good question. I might lean towards saying not, but I'm not a
maintainer. I guess it depends on how the type is used throughout the rest of
the Arrow ecosystem -- you mentioned the Compute Engine for example.
I'm not aware of active work on tensor computation in Arrow at the moment so
I don't think there were any decisions made on this yet. It is super
interesting to see what were the trade-offs in other places (numba, jax) though.
At the moment this is an `ExtensionArray` in
`cpp/src/arrow/extension_type_test.cc`. My understanding is there are no plans
to make these extension arrays available elsewhere @jorisvandenbossche @wesm ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
