Bob Tiernay created SPARK-5302:
----------------------------------
Summary: Add support for SQLContext "partition" columns
Key: SPARK-5302
URL: https://issues.apache.org/jira/browse/SPARK-5302
Project: Spark
Issue Type: New Feature
Components: SQL
Reporter: Bob Tiernay
For {{SQLContext}} (not {{HiveContext}}) it would be very convenient to support
a virtual column that maps to part of the the file path, similar to what is
done in Hive for partitions. The API could allow the user to type the column
using an appropriate {{DataType}} instance. This new field could be addressed
in SQL statements much the same as is done in Hive. As a consequence, this
would provide an nice interop and migration strategy for Hive users who may one
day use {{SQLContext}} directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
