GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/2701
[WIP][SPARK-3569][SQL] Add metadata field to StructField
Add `metadata` to `StructField`, which is a map that can store information
about the column. The metadata is preserved through simple operations like
`SELECT`. This PR doesn't handle Ser/De of schema.
Questions:
1. Should we use a mutable map or immutable map for metadata?
2. I put the tests in `SQLQuerySuite`. Is it the right place?
@marmbrus @liancheng
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark structfield-metadata
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2701.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2701
----
commit c194d5e4827955427b1ffac5bb582b994ae20cac
Author: Xiangrui Meng <[email protected]>
Date: 2014-09-17T21:41:49Z
add metadata field to StructField and Attribute
commit 367d237b3d5e445a67e6a8b9c9ae79abff26a045
Author: Xiangrui Meng <[email protected]>
Date: 2014-10-07T20:17:02Z
add test
commit d65072e483da9fd2dbd4999a4976befe2ce054d1
Author: Xiangrui Meng <[email protected]>
Date: 2014-10-07T21:46:48Z
remove Map.empty
commit 67fdebb7412484c6674052d2ac6c94573e846ce5
Author: Xiangrui Meng <[email protected]>
Date: 2014-10-07T22:06:12Z
add test on join
commit d8af0edc105e767e22d3d2696587a502741b9416
Author: Xiangrui Meng <[email protected]>
Date: 2014-10-08T00:19:33Z
move tests to SQLQuerySuite
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
