[
https://issues.apache.org/jira/browse/IMPALA-10771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17455374#comment-17455374
]
ASF subversion and git services commented on IMPALA-10771:
----------------------------------------------------------
Commit 157086cb80cb54c6565da25cc327621f6b2b8513 in impala's branch
refs/heads/master from Fucun Chu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=157086c ]
IMPALA-10771: Add Tencent COS support
This patch adds support for COS(Cloud Object Storage). Using the
hadoop-cos, the implementation is similar to other remote FileSystems.
New flags for COS:
- num_cos_io_threads: Number of COS I/O threads. Defaults to be 16.
Follow-up:
- Support for caching COS file handles will be addressed in
IMPALA-10772.
- test_concurrent_inserts and test_failing_inserts in
test_acid_stress.py are skipped due to slow file listing on
COS (IMPALA-10773).
Tests:
- Upload hdfs test data to a COS bucket. Modify all locations in HMS
DB to point to the COS bucket. Remove some hdfs caching params.
Run CORE tests.
Change-Id: Idce135a7591d1b4c74425e365525be3086a39821
Reviewed-on: http://gerrit.cloudera.org:8080/17503
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Support Tencent COS File System
> -------------------------------
>
> Key: IMPALA-10771
> URL: https://issues.apache.org/jira/browse/IMPALA-10771
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend
> Reporter: Fucun Chu
> Assignee: Fucun Chu
> Priority: Major
> Labels: connector
>
> [Tencent COS|https://intl.cloud.tencent.com/product/cos] is a famous object
> storage system provided by Tencent Corp. Hadoop-COS is a client that makes
> the upper computing systems based on HDFS be able to use the COS as its
> underlying storage system. The big data-processing systems that have been
> identified for support are: Hadoop MR, Spark, Alluxio and etc. In addition,
> Druid also can use COS as its deep storage by configuring HDFS-Load-Plugin
> integerating with HADOOP-COS.
> More information:
> https://hadoop.apache.org/docs/current/hadoop-cos/cloud-storage/index.html
> Hadoop 3.3 officially supports COS storage , see:
> [HADOOP-15616|https://issues.apache.org/jira/browse/HADOOP-15616]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
