alamb commented on a change in pull request #8714:
URL: https://github.com/apache/arrow/pull/8714#discussion_r528198753
##########
File path: rust/arrow/src/csv/reader.rs
##########
@@ -446,8 +446,57 @@ fn parse(
arrays.and_then(|arr| RecordBatch::try_new(projected_schema, arr))
}
+trait Parser: ArrowPrimitiveType {
+ fn parse(string: &str) -> Option<Self::Native> {
+ string.parse::<Self::Native>().ok()
+ }
+}
+
+impl Parser for BooleanType {
+ fn parse(string: &str) -> Option<bool> {
Review comment:
I wondered how this related to the rust standard boolean parsing:
https://doc.rust-lang.org/src/core/str/traits.rs.html#590
Seems like it anything it would be slightly slower, but also support mixed
case (`true` and `True`). Seems like a good improvement to me, though adding a
test to encode the expected behavior would probably be a good idea.
##########
File path: rust/arrow/src/csv/reader.rs
##########
@@ -446,8 +446,57 @@ fn parse(
arrays.and_then(|arr| RecordBatch::try_new(projected_schema, arr))
}
+trait Parser: ArrowPrimitiveType {
+ fn parse(string: &str) -> Option<Self::Native> {
+ string.parse::<Self::Native>().ok()
+ }
+}
+
+impl Parser for BooleanType {
+ fn parse(string: &str) -> Option<bool> {
+ if string.eq_ignore_ascii_case("false") {
+ return Some(false);
+ }
+ if string.eq_ignore_ascii_case("true") {
+ return Some(true);
+ }
+ None
+ }
+}
+
+impl Parser for Float32Type {
+ fn parse(string: &str) -> Option<f32> {
+ lexical_core::parse(string.as_bytes()).ok()
+ }
+}
+impl Parser for Float64Type {
+ fn parse(string: &str) -> Option<f64> {
+ lexical_core::parse(string.as_bytes()).ok()
+ }
+}
+
+impl Parser for UInt64Type {}
Review comment:
Perhaps @nevi-me was asking if there are plans to improve the parsing
performance of these types as well
##########
File path: rust/arrow/src/csv/reader.rs
##########
@@ -446,8 +446,57 @@ fn parse(
arrays.and_then(|arr| RecordBatch::try_new(projected_schema, arr))
}
+trait Parser: ArrowPrimitiveType {
+ fn parse(string: &str) -> Option<Self::Native> {
+ string.parse::<Self::Native>().ok()
+ }
+}
+
+impl Parser for BooleanType {
+ fn parse(string: &str) -> Option<bool> {
Review comment:
Added https://issues.apache.org/jira/browse/ARROW-10677 to track extra
parsing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
