tustvold commented on code in PR #4795:
URL: https://github.com/apache/arrow-rs/pull/4795#discussion_r1322108046
##########
arrow-csv/src/reader/mod.rs:
##########
@@ -241,6 +242,11 @@ impl Format {
self
}
+ pub fn with_null_regex(mut self, null_regex: Regex) -> Self {
Review Comment:
Perhaps some doc comments?
##########
arrow-csv/src/reader/mod.rs:
##########
@@ -336,6 +343,7 @@ impl Format {
if let Some(t) = self.terminator {
builder.terminator(csv_core::Terminator::Any(t));
}
+ // TODO: Null regex
Review Comment:
?
##########
arrow-csv/src/reader/mod.rs:
##########
@@ -319,6 +325,7 @@ impl Format {
if let Some(t) = self.terminator {
builder.terminator(csv::Terminator::Any(t));
}
+ // TODO: Null regex
Review Comment:
?
##########
arrow-csv/src/reader/mod.rs:
##########
@@ -827,11 +880,12 @@ fn build_decimal_array<T: DecimalType>(
col_idx: usize,
precision: u8,
scale: i8,
+ null_regex: Option<&Regex>,
) -> Result<ArrayRef, ArrowError> {
let mut decimal_builder = PrimitiveBuilder::<T>::with_capacity(rows.len());
for row in rows.iter() {
let s = row.get(col_idx);
- if s.is_empty() {
+ if s.is_empty() || null_regex.is_some_and(|r| r.is_match(s)) {
Review Comment:
If a null regex is provided should we also treat empty strings as nulls?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
