GitHub user jeanlyn opened a pull request:
https://github.com/apache/spark/pull/4289
[SPARK-5498][SPARK-SQL]fix bug when query the data when partition schema
does not match table schema
In hive,the schema of partition may be difference from the table
schema.When we use spark-sql to query the data of partition which schema is
difference from the table schema,we will get the exceptions as the description
of the [jira](https://issues.apache.org/jira/browse/SPARK-5498) .For example:
1.We take a look of the schema for the partition and the table
```sql
DESCRIBE partition_test PARTITION (dt='1');
id int None
name string None
dt string None
# Partition Information
# col_name data_type comment
dt string None
```
```
DESCRIBE partition_test;
OK
id bigint None
name string None
dt string None
# Partition Information
# col_name data_type comment
dt string None
```
2. run the sql
```sql
SELECT * FROM partition_test where dt='1';
```
we will get the cast exception `java.lang.ClassCastException:
org.apache.spark.sql.catalyst.expressions.MutableLong cannot be cast to
org.apache.spark.sql.catalyst.expressions.MutableInt`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jeanlyn/spark schema
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4289.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4289
----
commit adfc7defb278667d0c27c6128b00339bb8d52bb1
Author: jeanlyn <[email protected]>
Date: 2015-01-30T13:48:21Z
SPARK-5498:fix bug when query the data when partition schema does not match
table schema
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
