[
https://issues.apache.org/jira/browse/CARBONDATA-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xubo245 updated CARBONDATA-2368:
--------------------------------
Description:
Now Spark DataSource changed. Carbondata fileformat class extend FileFormat
class. But can not obtain files in sub-directory. Any one can give idea to
solve it?
for example :
{code:java}
class SparkCarbonFileFormat extends FileFormat
with DataSourceRegister
with Logging
with Serializable {
override def inferSchema(sparkSession: SparkSession,
options: Map[String, String],
files: Seq[FileStatus]): Option[StructType] = {
...
{code}
was:
Now Spark DataSource changed. Carbondata fileformat class extend FileFormat
class. But can not obtain files in sub-directory. Any one can give idea to
solve it?
for example :
{code:java}
class SparkCarbonFileFormat extends FileFormat
with DataSourceRegister
with Logging
with Serializable {
@transient val LOGGER = LogServiceFactory.getLogService(this.getClass.getName)
override def inferSchema(sparkSession: SparkSession,
options: Map[String, String],
files: Seq[FileStatus]): Option[StructType] = {
{code}
> Support run SQL on carbon files directly, which has schema and metadata
> -----------------------------------------------------------------------
>
> Key: CARBONDATA-2368
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2368
> Project: CarbonData
> Issue Type: Improvement
> Reporter: xubo245
> Assignee: xubo245
> Priority: Major
>
> Now Spark DataSource changed. Carbondata fileformat class extend FileFormat
> class. But can not obtain files in sub-directory. Any one can give idea to
> solve it?
> for example :
> {code:java}
> class SparkCarbonFileFormat extends FileFormat
> with DataSourceRegister
> with Logging
> with Serializable {
> override def inferSchema(sparkSession: SparkSession,
> options: Map[String, String],
> files: Seq[FileStatus]): Option[StructType] = {
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
