[
https://issues.apache.org/jira/browse/ARROW-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617240#comment-17617240
]
Apache Arrow JIRA Bot commented on ARROW-8221:
----------------------------------------------
This issue was last updated over 90 days ago, which may be an indication it is
no longer being actively worked. To better reflect the current state, the issue
is being unassigned per [project
policy|https://arrow.apache.org/docs/dev/developers/bug_reports.html#issue-assignment].
Please feel free to re-take assignment of the issue if it is being actively
worked, or if you plan to start that work soon.
> [Python][Dataset] Expose schema inference / validation options in the factory
> -----------------------------------------------------------------------------
>
> Key: ARROW-8221
> URL: https://issues.apache.org/jira/browse/ARROW-8221
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Joris Van den Bossche
> Assignee: Joris Van den Bossche
> Priority: Major
> Labels: dataset, pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> ARROW-8058 added options related to schema inference / validation for the
> Dataset factory. We should expose this in Python in the {{dataset(..)}}
> factory function:
> - Add ability to pass a user-specified schema with a {{schema}} keyword,
> instead of inferring the schema from (one of) the files (to be passed to the
> factory finish method)
> - Add {{validate_schema}} option to toggle whether the schema is validated
> against the actual files or not.
> - Expose in some way the number of fragments to be inspected when inferring
> or validating the schema. Not sure yet what the best API for this would be.
> Some relevant notes from the original PR:
> https://github.com/apache/arrow/pull/6687#issuecomment-604394407
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
