sgilmore10 opened a new pull request, #35827:
URL: https://github.com/apache/arrow/pull/35827
<!--
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how
to contribute here:
* [New Contributor's
Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing
Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
If this is not a [minor
PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes).
Could you open an issue for this pull request on GitHub?
https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the
[Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.)
of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
-->
### Rationale for this change
This change lets users control toggle the automatic null-value detection
behavior. By default, values MATLAB considers to be missing (e.g. `NaN` for
`double`, `<missing>` for `string`, and `NaT` for `datetime`) will be treated
as `null` values. Users can toggle this behavior on and off using the
`InferNulls` name-value pair.
**Example**
```matlab
>> matlabArray = [1 NaN 3]'
matlabArray =
1
NaN
3
% Treat NaN as a null value
>> arrowArray1 = arrow.array.Float64Array(maltabArray, InferNulls=true)
arrowArray1 =
[
1,
null,
3
]
% Don't treat NaN as a null value
>> arrowArray2 = arrow.array.Float64Array(maltabArray, InferNulls=false)
arrowArray2 =
[
1,
nan,
3
]
```
We've only added this nv-pair to `arrow.array.Float64Array` for now. We'll
add this nv-pair to the other types in a followup changelist.
### What changes are included in this PR?
1. Added `InferNulls` name-value pair to `arrow.array.Float64Array`.
2. Added common validation function `arrow.args.validateTypeAndShape` to
remove duplicate validation code among the numeric classes.
3. Added a functioned called `arrow.args.parseValidElements` that the
`arrow.array.<Type>Array` classes will be able to share for generating the
logical mask of valid elements.
### Are these changes tested?
Yes, we added a test pointed called `InferNulls` to the test
class`tFloat64Array.m`.
### Are there any user-facing changes?
Yes, users can know control how `NaN` values are treated when creating an
`arrow.array.Float64Array`.
### Future Directions
1. Add a name-value pair called `ValidIndices` or `NullIndices` to allow
users to supply the list of valid elements themselves.
2. Extend null support to other numeric types.
3. We've been working on adding error-handling support to
`mathworks/libmexclass`. We have a prototype to do this using status-like and
result-like objects already pushed to a
[branch](https://github.com/mathworks/libmexclass/tree/33). Once this branch is
merged with the `main` branch of `mathworks/libmexclass`, we'll port it over.
### Notes
Thank you @kevingurney for all the help with this PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
