[
https://issues.apache.org/jira/browse/IMPALA-10553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17302637#comment-17302637
]
ASF subversion and git services commented on IMPALA-10553:
----------------------------------------------------------
Commit 122bd9ea06dc9ebd126fe4f99d62651957016176 in impala's branch
refs/heads/master from John Sherman
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=122bd9e ]
IMPALA-10553: External Frontend CTAS support
- Adds the concept of an external staging dir to HdfsTableSink
- This allows an external to specify the destination of the
sink
- When this is set, the external frontend is responsible for
for moving and managing the results
- Any DDL related operations are assumed to be handled by
the external frontend
- External frontends may optionally supply a partition
depth which acts as a hint to skip a certain number of
partitions while creating directories for partitions. This
is for when the external frontend has pre-created a
certain number of the directories in staging (usually the
static portion of a partition specification)/
- Modifies delta/base naming to include 0 prefix padding to
match Hive for dynamic partitioning detection
- External frontends are responsible for doing authorization
checks against the staging directory and it is assumed the
external frontend service is not exposed directly to users.
Co-authored-by: Kurt Deschler <[email protected]>
Change-Id: Iae0ea4a832d8281c563427d0d7da1623bfce437b
Reviewed-by: Kurt Deschler <[email protected]>
Reviewed-on: http://gerrit.cloudera.org:8080/17145
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Support for CTAS/Insert for external frontends
> ----------------------------------------------
>
> Key: IMPALA-10553
> URL: https://issues.apache.org/jira/browse/IMPALA-10553
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: John Sherman
> Assignee: John Sherman
> Priority: Major
>
> The intent is to allow external frontends to INSERT/CTAS and INSERT INTO
> SELECT. The work mostly revolves around support for specifying where the data
> should land (I.E. a staging directory) and leave the responsibility of doing
> the final move to the external frontend.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
