[
https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375590#comment-14375590
]
iward edited comment on SPARK-3720 at 3/23/15 9:10 AM:
-------------------------------------------------------
hi,[~zhzhan] , I have the same problem with your issues of spark-2883.And I
just contact orcFile on spark，I can not quite understand your patch ,I would
like to ask you a few questions：
#1，why spark would read the whole files,what's the detail of problem on spark?
#2，could you tell me what should we do to solve the problem?
thanks
was (Author: iward):
hi,[~zhzhan] , I have the same problem.And I just contact orcFile on spark，I
can not quite understand your patch ,I would like to ask you a few questions：
#1，why spark would read the whole files,what's the detail of problem on spark?
#2，could you tell me what should we do to solve the problem?
thanks
> support ORC in spark sql
> ------------------------
>
> Key: SPARK-3720
> URL: https://issues.apache.org/jira/browse/SPARK-3720
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 1.1.0
> Reporter: Fei Wang
> Attachments: orc.diff
>
>
> The Optimized Row Columnar (ORC) file format provides a highly efficient way
> to store data on hdfs.ORC file format has many advantages such as:
> 1 a single file as the output of each task, which reduces the NameNode's load
> 2 Hive type support including datetime, decimal, and the complex types
> (struct, list, map, and union)
> 3 light-weight indexes stored within the file
> skip row groups that don't pass predicate filtering
> seek to a given row
> 4 block-mode compression based on data type
> run-length encoding for integer columns
> dictionary encoding for string columns
> 5 concurrent reads of the same file using separate RecordReaders
> 6 ability to split files without scanning for markers
> 7 bound the amount of memory needed for reading or writing
> 8 metadata stored using Protocol Buffers, which allows addition and removal
> of fields
> Now spark sql support Parquet, support ORC provide people more opts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
