lidavidm commented on code in PR #41646:
URL: https://github.com/apache/arrow/pull/41646#discussion_r1624376446
##########
java/dataset/src/main/cpp/jni_wrapper.cc:
##########
@@ -153,6 +154,19 @@ arrow::Result<std::shared_ptr<arrow::dataset::FileFormat>>
GetFileFormat(
}
}
+arrow::Result<std::shared_ptr<arrow::dataset::FragmentScanOptions>>
+GetFragmentScanOptions(jint file_format_id,
+ const std::unordered_map<std::string, std::string>&
configs) {
+ switch (file_format_id) {
+#ifdef ARROW_CSV
+ case 3:
Review Comment:
Is there some enum/constant that defines the possible values?
##########
cpp/src/arrow/dataset/file_csv.h:
##########
@@ -85,6 +85,10 @@ class ARROW_DS_EXPORT CsvFileFormat : public FileFormat {
struct ARROW_DS_EXPORT CsvFragmentScanOptions : public FragmentScanOptions {
std::string type_name() const override { return kCsvTypeName; }
+ /// \brief Construct FragmentScanOptions from config map
Review Comment:
Ideally we'd document the possible options? But also, this also feels like
something that should be inside the JNI bindings specifically and not the
public API.
##########
java/dataset/pom.xml:
##########
@@ -25,6 +25,8 @@
<arrow.cpp.build.dir>../../../cpp/release-build/</arrow.cpp.build.dir>
<parquet.version>1.13.1</parquet.version>
<avro.version>1.11.3</avro.version>
+ <substrait.version>0.31.0</substrait.version>
+ <protobuf.version>3.25.3</protobuf.version>
Review Comment:
Is this not already defined by dependencyManagement elsewhere?
##########
cpp/src/arrow/engine/substrait/serde.h:
##########
@@ -183,6 +184,13 @@ ARROW_ENGINE_EXPORT Result<BoundExpressions>
DeserializeExpressions(
const ConversionOptions& conversion_options = {},
ExtensionSet* ext_set_out = NULLPTR);
+/// \brief Deserialize a Substrait Literal message to the map
+///
+/// \param[in] buf a buffer containing the protobuf serialization of a
Substrait Literal
+/// \param[out] out deserialize to this map.
+ARROW_ENGINE_EXPORT Status
+DeserializeMap(const Buffer& buf, std::unordered_map<std::string,
std::string>& out);
Review Comment:
This all feels very specific to the internals of the JNI bindings. I don't
think this or the new FromProto should be in the public API or the Arrow
libraries, ideally, but in the JNI bindings instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
