GitHub user manishgupta88 opened a pull request:
https://github.com/apache/carbondata/pull/2774
[CARBONDATA-2979] select count fails when carbondata file is written
through SDK and read through sparkfileformat for complex datatype
map(struct->array->map)
**Problem**
Select query failed issue for map type when data is loaded using avro SDK
and external table using carbon file format is used to query the data
**Analysis**
When data is loaded through Avro SDK which has a schema of type
struct<array<map>>, fieldName was hard coded to "val" because of which during
query the schema written in the file footer and schema inferred for the
external table had a mismatch which lead to failure.
**Solution**
Instead of hard coding the field value as "val" use the given field name in
the schema
- [ ] Any interfaces changed?
No
- [ ] Any backward compatibility impacted?
No
- [ ] Document update required?
No
- [ ] Testing done
UT Added
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manishgupta88/carbondata fix_map_type_issues
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2774.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2774
----
commit 1e1eda1dcfcfdf424c9fa7748555d4400c6c14c3
Author: manishgupta88 <tomanishgupta18@...>
Date: 2018-09-27T12:32:34Z
Fixed query failure issue for map type when data is loaded using avro SDK
and external table using carbon file format is used to query the data
----
---
