[
https://issues.apache.org/jira/browse/IMPALA-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-5442:
-----------------------------------
Labels: ramp-up (was: )
> Allow HDFS replication factor to be set
> ---------------------------------------
>
> Key: IMPALA-5442
> URL: https://issues.apache.org/jira/browse/IMPALA-5442
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
> Affects Versions: Impala 2.8.0
> Reporter: Manish Maheshwari
> Priority: Major
> Labels: ramp-up
>
> Currently for small tables with just 1 HDFS file, the file block are
> potentially available on default 3 DN's. If the file content was static, we
> would run a hdfs dfs -setrep and then issue a refresh table/partition command
> to cache the newly replicated block locations. However since hdfs dfs -setrep
> is not synchronous, we don't really know when to issue a setrep command.
> HDFS caching helps this a bit, but has the same issue that we need to do a
> setrep and issue the caching directive and then refresh the table to get the
> cached and on disk block locations into the catalog.
> A good feature for small tables would be to allow HDFS replication factor be
> specified as part of the Impala INSERT INTO clause, (Cannot have this as part
> of create table options as Hive would not support this)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
