[
https://issues.apache.org/jira/browse/TRAFODION-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16469304#comment-16469304
]
ASF GitHub Bot commented on TRAFODION-3065:
-------------------------------------------
GitHub user selvaganesang opened a pull request:
https://github.com/apache/trafodion/pull/1557
Changes for [TRAFODION-3065] and [TRAFODION-2982]
[TRAFODION-3065] Trafodion to support compressed Hive Text formatted tables
Compressed text files are now supported via the new implementation using
HDFS java APIs. When Hadoop is not configured to support a particular type
of compression, an error is thrown.
[TRAFODION-2982] JNI HDFS interface should support varied sized large
buffers for read/write
A new CQD HDFS_IO_INTERIM_BYTEARRAY_SIZE_IN_KB is introduced to chunk
the read and write when byteArray is involved.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/selvaganesang/trafodion
hdfs_compression_support
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafodion/pull/1557.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1557
----
commit f216cdb31b89cb6a8b80717c113a6dcdfd3a24e1
Author: selvaganesang <selva.govindarajan@...>
Date: 2018-05-09T00:13:42Z
[TRAFODION-2917] Refactor Trafodion implementation of hdfs scan for text
formatted hive tables
Fix for compGeneral/TEST045
When a hive scan was prepared and executed many times, incorrect array
containing runtime ranges
were deallocated leading to memory corruption. Also, fixed a memory leak in
the JNI layer.
commit 96cab4ddd086a59ebc0eab8ac4a93ee3cf315aac
Author: selvaganesang <selva.govindarajan@...>
Date: 2018-05-09T00:36:04Z
[TRAFODION-3065] Trafodion to support compressed Hive Text formatted tables
Compressed text files are now supported via the new implementation using
HDFS java APIs. When the hadoop is not configured to support a particular
type
of compression, an error is thrown.
[TRAFODION-2982] JNI HDFS interface should support varied sized large
buffers for read/write
A new CQD HDFS_IO_INTERIM_BYTEARRAY_SIZE_IN_KB is introduced to chunk
the read and write when byteArray is involved.
----
> Trafodion to support compressed Hive Text formatted tables
> ----------------------------------------------------------
>
> Key: TRAFODION-3065
> URL: https://issues.apache.org/jira/browse/TRAFODION-3065
> Project: Apache Trafodion
> Issue Type: New Feature
> Components: sql-cmp, sql-exe
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Priority: Major
> Fix For: 2.3
>
>
> Currently, Trafodion doesn't support Hive text formatted compressed files.
> The compressed file support will be provided with the refactored Hdfs scan
> implementation only.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
