[
https://issues.apache.org/jira/browse/ATLAS-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
VINAYAK MARRAIYA updated ATLAS-5001:
------------------------------------
Description:
Impala SQL queries that don’t include the “WITH” clause can populate lineage in
Atlas, while Impala SQL queries that include the “WITH” clause do not populate
lineage in Atlas.
Reference Impala SQL querie:
{code:java}
(1) CREATE TABLE default.new_target_table111 AS
SELECT *
FROM default.new_source_table;
(2)
【Insert data into the new target table using WITH clause】
====================
--Create a new example source table
CREATE TABLE IF NOT EXISTS new_db.new_source_table (
id INT,
name STRING,
amount DOUBLE
) STORED AS PARQUET;
-- Insert sample data
INSERT INTO new_db.new_source_table (id, name, amount) VALUES
(1, 'Alice', 150.0),
(2, 'Bob', 90.0),
(3, 'Charlie', 200.0);
-- Create a new target table
CREATE TABLE IF NOT EXISTS new_db.new_target_table_with (
id INT,
name STRING,
amount DOUBLE
) STORED AS PARQUET;
-- Insert data into the new target table using WITH clause
WITH filtered_data AS (
SELECT
id,
name,
amount
FROM
new_db.new_source_table
WHERE
amount > 100
)
INSERT INTO new_db.new_target_table_with
SELECT
id,
name,
amount
FROM
filtered_data;
====================
【Insert data into the new target table without using WITH clause】
====================
-- Create a new example source table
CREATE TABLE IF NOT EXISTS new_db.new_source_table1 (
id INT,
name STRING,
amount DOUBLE
) STORED AS PARQUET;
-- Insert sample data
INSERT INTO new_db.new_source_table1 (id, name, amount) VALUES
(1, 'Alice', 150.0),
(2, 'Bob', 90.0),
(3, 'Charlie', 200.0);
-- Create a new target table
CREATE TABLE IF NOT EXISTS new_db.new_target_table1 (
id INT,
name STRING,
amount DOUBLE
) STORED AS PARQUET;
-- Insert data into the new target table without using WITH clause
INSERT INTO new_db.new_target_table1
SELECT
id,
name,
amount
FROM
new_db.new_source_table1
WHERE
amount > 100;{code}
> Impala SQL queries that include the “WITH” clause should populate lineage in
> Atlas
> ----------------------------------------------------------------------------------
>
> Key: ATLAS-5001
> URL: https://issues.apache.org/jira/browse/ATLAS-5001
> Project: Atlas
> Issue Type: Bug
> Reporter: VINAYAK MARRAIYA
> Assignee: VINAYAK MARRAIYA
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Impala SQL queries that don’t include the “WITH” clause can populate lineage
> in Atlas, while Impala SQL queries that include the “WITH” clause do not
> populate lineage in Atlas.
> Reference Impala SQL querie:
> {code:java}
> (1) CREATE TABLE default.new_target_table111 AS
> SELECT *
> FROM default.new_source_table;
> (2)
> 【Insert data into the new target table using WITH clause】
> ====================
> --Create a new example source table
> CREATE TABLE IF NOT EXISTS new_db.new_source_table (
> id INT,
> name STRING,
> amount DOUBLE
> ) STORED AS PARQUET;
> -- Insert sample data
> INSERT INTO new_db.new_source_table (id, name, amount) VALUES
> (1, 'Alice', 150.0),
> (2, 'Bob', 90.0),
> (3, 'Charlie', 200.0);
> -- Create a new target table
> CREATE TABLE IF NOT EXISTS new_db.new_target_table_with (
> id INT,
> name STRING,
> amount DOUBLE
> ) STORED AS PARQUET;
> -- Insert data into the new target table using WITH clause
> WITH filtered_data AS (
> SELECT
> id,
> name,
> amount
> FROM
> new_db.new_source_table
> WHERE
> amount > 100
> )
> INSERT INTO new_db.new_target_table_with
> SELECT
> id,
> name,
> amount
> FROM
> filtered_data;
> ====================
> 【Insert data into the new target table without using WITH clause】
> ====================
> -- Create a new example source table
> CREATE TABLE IF NOT EXISTS new_db.new_source_table1 (
> id INT,
> name STRING,
> amount DOUBLE
> ) STORED AS PARQUET;
> -- Insert sample data
> INSERT INTO new_db.new_source_table1 (id, name, amount) VALUES
> (1, 'Alice', 150.0),
> (2, 'Bob', 90.0),
> (3, 'Charlie', 200.0);
> -- Create a new target table
> CREATE TABLE IF NOT EXISTS new_db.new_target_table1 (
> id INT,
> name STRING,
> amount DOUBLE
> ) STORED AS PARQUET;
> -- Insert data into the new target table without using WITH clause
> INSERT INTO new_db.new_target_table1
> SELECT
> id,
> name,
> amount
> FROM
> new_db.new_source_table1
> WHERE
> amount > 100;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
