[
https://issues.apache.org/jira/browse/DRILL-8388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8388:
--------------------------------
Description:
When a JDBC client issues a CTAS statement then Drill will return a record for
each completed writer fragment containing the number of records that fragment
wrote. These records are returned in the usual streaming fashion as writer
fragments complete, their order being unknowable in advance. If the client
application immediately closes its clientside JDBC resources after its call to
Statement.executeQuery has returned as follows
{code:java}
Statement ctasStatement = conn.createStatement();
ResultSet ctasResults = ctasStatement.executeQuery(ctasQueryText);
ctasResults.close();
ctasStatement.close();
{code}
it may be that the CTAS statement is still executing, and that is then
unintentionally cancelled depending on good or bad luck with respect to timing.
The cancellation of the CTAS statement is usually benign if it spawned only one
writer fragment, but if it spawned more than one then the chances increase that
at least one writer will be interrupted before it has finished writing,
resulting in incomplete or even corrupted output. Even in the benign case, such
queries conclude in the CANCELLED state rather than the FINISHED state.
To have CTAS queries reliably run to completion, the JDBC client can wait for
all of the writer fragments to complete before it closes its JDBC resources by
scrolling through the ResultSet before closing it. Using try-with-resources
syntax,
{code:java}
try (
Statement ctasStatement = conn.createStatement();
ResultSet ctasResults = ctasStatement.executeQuery(ctasQueryText);
) {
while (ctasResults.next());
}{code}
was:
When a JDBC client issues a CTAS statement then Drill will return a record for
each completed writer fragment containing the number of records that fragment
wrote. These records are returned in the usual streaming fashion as writer
fragments complete, their order being unknowable in advance. If the client
application immediately closes its clientside JDBC resources after its call to
Statement.executeQuery has returned as follows
{code:java}
Statement ctasStatement = conn.createStatement();
ResultSet ctasResults = ctasStatement.executeQuery(ctasQueryText);
ctasResults.close();
ctasStatement.close();
{code}
it may be that the CTAS statement is still executing, and that is then
prematurely cancelled depending on good or bad luck with respect to timing.
The cancellation of the CTAS statement is usually benign if it spawned only one
writer fragment, but if it spawned more than one then it is likely that at
least one writer will be interrupted before it has finished writing, resulting
in incomplete or even corrupted output. Even in the benign case, such queries
conclude in the CANCELLED state rather than the FINISHED state.
To have CTAS queries reliably conclude completely, the JDBC client can wait for
all of the writer fragments to complete before it closes its JDBC resources by
scrolling through the ResultSet before closing it.
{code:java}
while (ctasResults.next());{code}
> CTAS sent over JDBC may be cancelled if query results are not fetched
> ---------------------------------------------------------------------
>
> Key: DRILL-8388
> URL: https://issues.apache.org/jira/browse/DRILL-8388
> Project: Apache Drill
> Issue Type: Task
> Components: Client - JDBC, Storage - Writer
> Affects Versions: 1.20.3
> Reporter: James Turton
> Assignee: James Turton
> Priority: Minor
> Fix For: Future
>
>
> When a JDBC client issues a CTAS statement then Drill will return a record
> for each completed writer fragment containing the number of records that
> fragment wrote. These records are returned in the usual streaming fashion as
> writer fragments complete, their order being unknowable in advance. If the
> client application immediately closes its clientside JDBC resources after its
> call to Statement.executeQuery has returned as follows
> {code:java}
> Statement ctasStatement = conn.createStatement();
> ResultSet ctasResults = ctasStatement.executeQuery(ctasQueryText);
> ctasResults.close();
> ctasStatement.close();
> {code}
> it may be that the CTAS statement is still executing, and that is then
> unintentionally cancelled depending on good or bad luck with respect to
> timing.
> The cancellation of the CTAS statement is usually benign if it spawned only
> one writer fragment, but if it spawned more than one then the chances
> increase that at least one writer will be interrupted before it has finished
> writing, resulting in incomplete or even corrupted output. Even in the benign
> case, such queries conclude in the CANCELLED state rather than the FINISHED
> state.
> To have CTAS queries reliably run to completion, the JDBC client can wait for
> all of the writer fragments to complete before it closes its JDBC resources
> by scrolling through the ResultSet before closing it. Using
> try-with-resources syntax,
> {code:java}
> try (
> Statement ctasStatement = conn.createStatement();
> ResultSet ctasResults = ctasStatement.executeQuery(ctasQueryText);
> ) {
> while (ctasResults.next());
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
