arjunsr1 commented on issue #13847:
URL: https://github.com/apache/arrow/issues/13847#issuecomment-1217297409
Hello @kou - I had tested the changes locally and everything looked fine,
but in a testing environment for one of our tables, the addition of
column_types did not work in terms of changing the schema. I've attached the
file and the code we're using. Any help in diagnosing the problem would be
greatly appreciated.
```
ARROW_STRING_TYPE = :string
ARROW_UINT16_TYPE = :uint16
TIMESTAMP_MS = lambda {
Arrow::TimestampDataType.new(Arrow::TimeUnit::MILLI) }
COLUMN_TYPES = {
group_name: ARROW_STRING_TYPE,
created_time: TIMESTAMP_MS.call,
completed_time: TIMESTAMP_MS.call,
source: ARROW_STRING_TYPE,
status: ARROW_STRING_TYPE,
runtime: :double,
is_single_query: ARROW_STRING_TYPE,
user_name: ARROW_STRING_TYPE,
user_email: ARROW_STRING_TYPE,
user_is_disabled: ARROW_STRING_TYPE,
is_looker_employee: ARROW_STRING_TYPE,
query_id: ARROW_UINT16_TYPE,
query_filters: ARROW_STRING_TYPE,
query_fields: ARROW_STRING_TYPE,
query_dynamic_fields: ARROW_STRING_TYPE,
instance_number: ARROW_UINT16_TYPE
}
table = Arrow::Table.load(path_to_file, column_types: COLUMN_TYPES)
# table.schema.to_s.split("\n") continues to show date64[ms] as well
as other un-converted column types.
# Note that the code works on multiple other tables we had tried.
```
[instance_id8.csv](https://github.com/apache/arrow/files/9355369/instance_id8.csv)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
