[
https://issues.apache.org/jira/browse/IGNITE-25977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Orlov updated IGNITE-25977:
--------------------------------------
Description:
Take a look at test case below:
{code}
// org.apache.ignite.internal.sql.engine.planner.UnionPlannerTest
@Test
void unionAllHugeNumberOfTables() throws Exception {
int tablesCount = 50;
IgniteTable[] tables = IntStream.range(0, tablesCount)
.mapToObj(i -> createTestTable("TABLE_" + i))
.toArray(IgniteTable[]::new);
IgniteSchema schema = createSchema(tables);
String query = IntStream.range(0, tablesCount)
.mapToObj(i -> "SELECT \n"
+ " 'label_" + i + "' As l_name, \n"
+ " CASE WHEN EXISTS (\n"
+ " SELECT 1 FROM table_" + i + "\n"
+ " ) then true ELSE false END AS has_data")
.collect(Collectors.joining(" UNION "));
assertPlan(query, schema, node -> true);
}
{code}
This test fails with {{java.lang.OutOfMemoryError}} due to current
implementations of
{{org.apache.ignite.internal.sql.engine.trait.TraitUtils#combinations}}: it
prepares full enumeration of all possible combinations of input traits. The
problem arises on relations with big number of inputs. In this particular case
there are 50 inputs, and every one have 2 variations of trait set, which gives
as 2^50 possible combinations. Let's introduce an upper bound for combinations
complexity for the exhaustive enumeration.
> Sql. Query optimizer hangs while preparing UNION with huge number of tables
> ---------------------------------------------------------------------------
>
> Key: IGNITE-25977
> URL: https://issues.apache.org/jira/browse/IGNITE-25977
> Project: Ignite
> Issue Type: Bug
> Components: sql ai3
> Reporter: Konstantin Orlov
> Priority: Major
> Labels: ignite-3
>
> Take a look at test case below:
> {code}
> // org.apache.ignite.internal.sql.engine.planner.UnionPlannerTest
> @Test
> void unionAllHugeNumberOfTables() throws Exception {
> int tablesCount = 50;
> IgniteTable[] tables = IntStream.range(0, tablesCount)
> .mapToObj(i -> createTestTable("TABLE_" + i))
> .toArray(IgniteTable[]::new);
> IgniteSchema schema = createSchema(tables);
> String query = IntStream.range(0, tablesCount)
> .mapToObj(i -> "SELECT \n"
> + " 'label_" + i + "' As l_name, \n"
> + " CASE WHEN EXISTS (\n"
> + " SELECT 1 FROM table_" + i + "\n"
> + " ) then true ELSE false END AS has_data")
> .collect(Collectors.joining(" UNION "));
>
> assertPlan(query, schema, node -> true);
> }
> {code}
> This test fails with {{java.lang.OutOfMemoryError}} due to current
> implementations of
> {{org.apache.ignite.internal.sql.engine.trait.TraitUtils#combinations}}: it
> prepares full enumeration of all possible combinations of input traits. The
> problem arises on relations with big number of inputs. In this particular
> case there are 50 inputs, and every one have 2 variations of trait set, which
> gives as 2^50 possible combinations. Let's introduce an upper bound for
> combinations complexity for the exhaustive enumeration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
