[jira] [Created] (IGNITE-19495) SQL onheap cache stores row on per-cache basis, but access these rows on per-index basis
Aleksey Plekhanov created IGNITE-19495:
--
Summary: SQL onheap cache stores row on per-cache basis, but
access these rows on per-index basis
Key: IGNITE-19495
URL: https://issues.apache.org/jira/browse/IGNITE-19495
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
If there are two indexes on the same cache and sqlOnHeapCache property is
enabled there can be node failures such as:
{noformat}
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
B+Tree is corrupted [groupId=-917681634, pageIds=[844420635164678],
cacheId=-917681634, cacheName=..., indexName=_key_PK, msg=Runtime failure on
row: Row@71aeb1b9[ ... ][ ]]
...
Caused by: org.apache.ignite.IgniteException: Failed to store new index row.
...
Caused by: java.lang.UnsupportedOperationException: 13 cannot be used for
inline type 19
{noformat}
Root cause: {{IndexRowCache}} is created per cache group (see
{{IndexRowCacheRegistry#onCacheRegistered}}), but stores {{IndexRowImpl}}
binded to particular index (see {{InlineIndexTree#createIndexRow}}). In case,
when another index requires the same data row, it gets row from the onheap
cache with the wrong index row handler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18832) Option to assign specific name to thin client and capturing the name in sys.client_connections
[ https://issues.apache.org/jira/browse/IGNITE-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-18832: --- Flags: (was: Important) > Option to assign specific name to thin client and capturing the name in > sys.client_connections > -- > > Key: IGNITE-18832 > URL: https://issues.apache.org/jira/browse/IGNITE-18832 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12, 2.13, 2.11.1, 2.14 >Reporter: Gangaiah >Assignee: Aleksey Plekhanov >Priority: Major > Labels: IEP-35 > Fix For: 2.16 > > > > System view client_connections doesn't have name column to capture client > name, it should have the column to track the thin client connections.. It's > difficult to pin point the client where remote address same for multiple > clients in production scenarios. > https://lists.apache.org/thread/joy6vjopq2zovkj768hnhfvmd44vhs4b -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18832) Option to assign specific name to thin client and capturing the name in sys.client_connections
[ https://issues.apache.org/jira/browse/IGNITE-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-18832: --- Ignite Flags: Release Notes Required (was: Docs Required,Release Notes Required) > Option to assign specific name to thin client and capturing the name in > sys.client_connections > -- > > Key: IGNITE-18832 > URL: https://issues.apache.org/jira/browse/IGNITE-18832 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.12, 2.13, 2.11.1, 2.14 >Reporter: Gangaiah >Assignee: Aleksey Plekhanov >Priority: Major > Labels: IEP-35 > Fix For: 2.16 > > > > System view client_connections doesn't have name column to capture client > name, it should have the column to track the thin client connections.. It's > difficult to pin point the client where remote address same for multiple > clients in production scenarios. > https://lists.apache.org/thread/joy6vjopq2zovkj768hnhfvmd44vhs4b -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19553) Metrics for operations on indexes
Aleksey Plekhanov created IGNITE-19553:
--
Summary: Metrics for operations on indexes
Key: IGNITE-19553
URL: https://issues.apache.org/jira/browse/IGNITE-19553
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Currently, we don't have metrics about performance of operations on indexes. We
have, {{io.statistics.sortedIndexes.*}} metrics about logical and physical page
reads, but without operation types and without duration. It will be heplful to
have metrics for different operations on indexes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19562) Calcite engine. Make sure all diagnostic tools work
Aleksey Plekhanov created IGNITE-19562:
--
Summary: Calcite engine. Make sure all diagnostic tools work
Key: IGNITE-19562
URL: https://issues.apache.org/jira/browse/IGNITE-19562
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics (i.e. "sql.parser.cache" and "sql.queries.user" registries)
* Events (i.e. {{EVT_SQL_QUERY_EXECUTION}})
* Long running queries warnings in log messages (see
{{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
* Performance statistics
* Tracing (tracing is a big task and should be done in a separate ticket)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19584) Calcite engine. Implement SQL metrics
Aleksey Plekhanov created IGNITE-19584: -- Summary: Calcite engine. Implement SQL metrics Key: IGNITE-19584 URL: https://issues.apache.org/jira/browse/IGNITE-19584 Project: Ignite Issue Type: Sub-task Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently, we have two group of metrics for SQL: * "sql.parser.cache" - shows count of hits/misses during SQL queries parsing * "sql.queries.user" - shows count of succesfully executed and failed metrics Parser metrics are not affected by Calcite-based engine and execution metrics count all metrics as succesfully executed. We need to implement parser metrics and fix execution metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19562) Calcite engine. Make sure all diagnostic tools work
[
https://issues.apache.org/jira/browse/IGNITE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19562:
---
Description:
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics
* Events (i.e. {{EVT_SQL_QUERY_EXECUTION}})
* Long running queries warnings in log messages (see
{{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
* Performance statistics
* Tracing
* Hiding of sensitive information in diagnostic tools
was:
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics (i.e. "sql.parser.cache" and "sql.queries.user" registries)
* Events (i.e. {{EVT_SQL_QUERY_EXECUTION}})
* Long running queries warnings in log messages (see
{{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
* Performance statistics
* Tracing (tracing is a big task and should be done in a separate ticket)
> Calcite engine. Make sure all diagnostic tools work
>
>
> Key: IGNITE-19562
> URL: https://issues.apache.org/jira/browse/IGNITE-19562
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite
>
> Calcite-based SQL engine and H2-based SQL engine use different paths to run
> queries. For H2-based engine we have a lot of diagnostic tools, perhaps some
> of them are not working for Calcite-based SQL engine.
> We need to check (write tests) and fix (in case it's not working) such
> instruments as:
> * Metrics
> * Events (i.e. {{EVT_SQL_QUERY_EXECUTION}})
> * Long running queries warnings in log messages (see
> {{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
> * Performance statistics
> * Tracing
> * Hiding of sensitive information in diagnostic tools
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19585) Calcite engine. Performance statistics
Aleksey Plekhanov created IGNITE-19585: -- Summary: Calcite engine. Performance statistics Key: IGNITE-19585 URL: https://issues.apache.org/jira/browse/IGNITE-19585 Project: Ignite Issue Type: Sub-task Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently, only root query event are written to performance statistics for Calcite-based SQL engine. We should fix it and implement physical/logical reads events for Calcite engine as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19584) Calcite engine. SQL metrics
[ https://issues.apache.org/jira/browse/IGNITE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-19584: --- Description: Currently, we have two groups (registries) of metrics for SQL: * "sql.parser.cache" - shows count of hits/misses during SQL queries parsing * "sql.queries.user" - shows count of succesfully executed and failed metrics Parser metrics are not affected by Calcite-based engine and execution metrics count all metrics as succesfully executed. We need to implement parser metrics and fix execution metrics. was: Currently, we have two group of metrics for SQL: * "sql.parser.cache" - shows count of hits/misses during SQL queries parsing * "sql.queries.user" - shows count of succesfully executed and failed metrics Parser metrics are not affected by Calcite-based engine and execution metrics count all metrics as succesfully executed. We need to implement parser metrics and fix execution metrics. Summary: Calcite engine. SQL metrics (was: Calcite engine. Implement SQL metrics) > Calcite engine. SQL metrics > --- > > Key: IGNITE-19584 > URL: https://issues.apache.org/jira/browse/IGNITE-19584 > Project: Ignite > Issue Type: Sub-task >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > > Currently, we have two groups (registries) of metrics for SQL: > * "sql.parser.cache" - shows count of hits/misses during SQL queries parsing > * "sql.queries.user" - shows count of succesfully executed and failed metrics > Parser metrics are not affected by Calcite-based engine and execution metrics > count all metrics as succesfully executed. > We need to implement parser metrics and fix execution metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19586) Calcite engine. Events
Aleksey Plekhanov created IGNITE-19586: -- Summary: Calcite engine. Events Key: IGNITE-19586 URL: https://issues.apache.org/jira/browse/IGNITE-19586 Project: Ignite Issue Type: Sub-task Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently, for Calcite-based SQL enginge only EVT_CACHE_QUERY_EXECUTED event is fired, but H2-based SQL engine fires also EVT_CACHE_QUERY_OBJECT_READ and EVT_SQL_QUERY_EXECUTION. Calcite engine should fire these events as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19562) Calcite engine. Make sure all diagnostic tools work
[
https://issues.apache.org/jira/browse/IGNITE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19562:
---
Description:
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics
* Events
* Long running queries warnings in log messages
* Performance statistics
* Tracing
* Hiding of sensitive information in diagnostic tools
was:
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics
* Events
* Long running queries warnings in log messages (see
{{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
* Performance statistics
* Tracing
* Hiding of sensitive information in diagnostic tools
> Calcite engine. Make sure all diagnostic tools work
>
>
> Key: IGNITE-19562
> URL: https://issues.apache.org/jira/browse/IGNITE-19562
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite
>
> Calcite-based SQL engine and H2-based SQL engine use different paths to run
> queries. For H2-based engine we have a lot of diagnostic tools, perhaps some
> of them are not working for Calcite-based SQL engine.
> We need to check (write tests) and fix (in case it's not working) such
> instruments as:
> * Metrics
> * Events
> * Long running queries warnings in log messages
> * Performance statistics
> * Tracing
> * Hiding of sensitive information in diagnostic tools
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19562) Calcite engine. Make sure all diagnostic tools work
[
https://issues.apache.org/jira/browse/IGNITE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19562:
---
Description:
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics
* Events
* Long running queries warnings in log messages (see
{{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
* Performance statistics
* Tracing
* Hiding of sensitive information in diagnostic tools
was:
Calcite-based SQL engine and H2-based SQL engine use different paths to run
queries. For H2-based engine we have a lot of diagnostic tools, perhaps some of
them are not working for Calcite-based SQL engine.
We need to check (write tests) and fix (in case it's not working) such
instruments as:
* Metrics
* Events (i.e. {{EVT_SQL_QUERY_EXECUTION}})
* Long running queries warnings in log messages (see
{{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
* Performance statistics
* Tracing
* Hiding of sensitive information in diagnostic tools
> Calcite engine. Make sure all diagnostic tools work
>
>
> Key: IGNITE-19562
> URL: https://issues.apache.org/jira/browse/IGNITE-19562
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite
>
> Calcite-based SQL engine and H2-based SQL engine use different paths to run
> queries. For H2-based engine we have a lot of diagnostic tools, perhaps some
> of them are not working for Calcite-based SQL engine.
> We need to check (write tests) and fix (in case it's not working) such
> instruments as:
> * Metrics
> * Events
> * Long running queries warnings in log messages (see
> {{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
> * Performance statistics
> * Tracing
> * Hiding of sensitive information in diagnostic tools
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19588) Calcite engine. Long running queries warning
Aleksey Plekhanov created IGNITE-19588:
--
Summary: Calcite engine. Long running queries warning
Key: IGNITE-19588
URL: https://issues.apache.org/jira/browse/IGNITE-19588
Project: Ignite
Issue Type: Sub-task
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
For H2-based SQL engine we have warnings in the log files, if query executed
for too long time (see {{IgniteConfiguration#setLongQueryWarningTimeout(long)}})
We need the same functionality for the Calcite-based SQL engine.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19589) Calcite engine. Tracing
Aleksey Plekhanov created IGNITE-19589: -- Summary: Calcite engine. Tracing Key: IGNITE-19589 URL: https://issues.apache.org/jira/browse/IGNITE-19589 Project: Ignite Issue Type: Sub-task Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov We need to support tracing for the Calcite-based SQL engine. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19590) Calcite engine. Hide sensitive information in diagnostic tools
Aleksey Plekhanov created IGNITE-19590: -- Summary: Calcite engine. Hide sensitive information in diagnostic tools Key: IGNITE-19590 URL: https://issues.apache.org/jira/browse/IGNITE-19590 Project: Ignite Issue Type: Sub-task Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov We need to hide sensitive information in queries exposed to diagnostic tools for the Calcite-based SQL engine, if flag IGNITE_TO_STRING_INCLUDE_SENSITIVE is not set. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19585) Calcite engine. Performance statistics
[ https://issues.apache.org/jira/browse/IGNITE-19585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-19585: --- Labels: calcite (was: ) > Calcite engine. Performance statistics > -- > > Key: IGNITE-19585 > URL: https://issues.apache.org/jira/browse/IGNITE-19585 > Project: Ignite > Issue Type: Sub-task >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: calcite > Fix For: 2.16 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently, only root query event are written to performance statistics for > Calcite-based SQL engine. > We should fix it and implement physical/logical reads events for Calcite > engine as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19586) Calcite engine. Events
[ https://issues.apache.org/jira/browse/IGNITE-19586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-19586: --- Release Note: SQL Calcite: Added events on query execution (EVT_CACHE_QUERY_OBJECT_READ, EVT_SQL_QUERY_EXECUTION) (was: SQL Calcite: Added SQL/Query events) > Calcite engine. Events > -- > > Key: IGNITE-19586 > URL: https://issues.apache.org/jira/browse/IGNITE-19586 > Project: Ignite > Issue Type: Sub-task >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently, for Calcite-based SQL enginge only EVT_CACHE_QUERY_EXECUTED event > is fired, but H2-based SQL engine fires also EVT_CACHE_QUERY_OBJECT_READ and > EVT_SQL_QUERY_EXECUTION. Calcite engine should fire these events as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19553) Metrics for operations on indexes
[
https://issues.apache.org/jira/browse/IGNITE-19553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19553:
---
Labels: ise (was: )
> Metrics for operations on indexes
> -
>
> Key: IGNITE-19553
> URL: https://issues.apache.org/jira/browse/IGNITE-19553
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, we don't have metrics about performance of operations on indexes.
> We have, {{io.statistics.sortedIndexes.*}} metrics about logical and physical
> page reads, but without operation types and without duration. It will be
> heplful to have metrics for different operations on indexes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19562) Calcite engine. Make sure all diagnostic tools work
[ https://issues.apache.org/jira/browse/IGNITE-19562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-19562: --- Labels: calcite ise (was: calcite) > Calcite engine. Make sure all diagnostic tools work > > > Key: IGNITE-19562 > URL: https://issues.apache.org/jira/browse/IGNITE-19562 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: calcite, ise > > Calcite-based SQL engine and H2-based SQL engine use different paths to run > queries. For H2-based engine we have a lot of diagnostic tools, perhaps some > of them are not working for Calcite-based SQL engine. > We need to check (write tests) and fix (in case it's not working) such > instruments as: > * Metrics > * Events > * Long running queries warnings in log messages > * Performance statistics > * Tracing > * Hiding of sensitive information in diagnostic tools -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18390) Calcite engine. Reduce count of created spools during planning
[ https://issues.apache.org/jira/browse/IGNITE-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-18390: --- Labels: calcite calcite2-required ise (was: calcite calcite2-required) > Calcite engine. Reduce count of created spools during planning > -- > > Key: IGNITE-18390 > URL: https://issues.apache.org/jira/browse/IGNITE-18390 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: calcite, calcite2-required, ise > Time Spent: 10m > Remaining Estimate: 0h > > Currently, when trying to convert traits table spool is created for each > exchange node, but not always required. Root node not required to be > rewindable, rewindability is required only on the right hand of correlated > nested loop node, but rewindability trait is propagated from the bottom nodes > (table and index scans) to the top nodes and all nodes are converted to > rewindable, this cause redundant spool creation. > Investigate posibility to reduce count of table spools to reduce planning > time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19748) Calcite engine. Support queries timeout
Aleksey Plekhanov created IGNITE-19748:
--
Summary: Calcite engine. Support queries timeout
Key: IGNITE-19748
URL: https://issues.apache.org/jira/browse/IGNITE-19748
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
We can set timeout for SQL queries:
* Using {{SqlFieldsQuery.timeout}} property (for certain query)
* Using "sql.defaultQueryTimeout" distributed property (default for all queries)
But Calcite-based SQL query engine ignore these timeouts. Only possible timeout
supported by the new engine is planning timeout. We should support execution
timeouts too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19748) Calcite engine. Support queries timeout
[
https://issues.apache.org/jira/browse/IGNITE-19748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19748:
---
Labels: calcite ise (was: calcite)
> Calcite engine. Support queries timeout
> ---
>
> Key: IGNITE-19748
> URL: https://issues.apache.org/jira/browse/IGNITE-19748
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, ise
>
> We can set timeout for SQL queries:
> * Using {{SqlFieldsQuery.timeout}} property (for certain query)
> * Using "sql.defaultQueryTimeout" distributed property (default for all
> queries)
> But Calcite-based SQL query engine ignore these timeouts. Only possible
> timeout supported by the new engine is planning timeout. We should support
> execution timeouts too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19748) Calcite engine. Support queries timeout
[
https://issues.apache.org/jira/browse/IGNITE-19748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19748:
---
Description:
We can set timeout for SQL queries:
* Using {{SqlFieldsQuery.timeout}} property (for certain query)
* Using "sql.defaultQueryTimeout" distributed property (default for all queries)
But Calcite-based SQL query engine ignore these timeouts. Only possible timeout
supported by the new engine is planning timeout. We should support execution
timeouts too.
was:
We can set timeout for SQL queries:
* Using {{SqlFieldsQuery.timeout}} property (for certain query)
* Using "sql.defaultQueryTimeout" distributed property (default for all queries)
But Calcite-based SQL query engine ignore these timeouts. Only possible timeout
supported by the new engine is planning timeout. We should support execution
timeouts too.
> Calcite engine. Support queries timeout
> ---
>
> Key: IGNITE-19748
> URL: https://issues.apache.org/jira/browse/IGNITE-19748
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite
>
> We can set timeout for SQL queries:
> * Using {{SqlFieldsQuery.timeout}} property (for certain query)
> * Using "sql.defaultQueryTimeout" distributed property (default for all
> queries)
> But Calcite-based SQL query engine ignore these timeouts. Only possible
> timeout supported by the new engine is planning timeout. We should support
> execution timeouts too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19759) Calcite engine. Review list of reserved keywords
Aleksey Plekhanov created IGNITE-19759:
--
Summary: Calcite engine. Review list of reserved keywords
Key: IGNITE-19759
URL: https://issues.apache.org/jira/browse/IGNITE-19759
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
For the calcite engine we have too strict list of reserved keywords. For
example, lexems such as "TYPE" and "OPTIONS" are reserved keywords and can't be
used as columns or table names. But "TYPE" is frequently used by users as
column name and we should exclude it from the list of reserved keywords (add it
to non-reserved keywords, see {{config.fmpp}} file \{{nonReservedKeywords}}
section). Other vendors allow to use "TYPE" as column name.
We should also review the whole list of reserved keywords (see generated
{{{}Parser.jj{}}}), perhaps some other keywords should be excluded from
reserved list too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19759) Calcite engine. Review list of reserved keywords
[
https://issues.apache.org/jira/browse/IGNITE-19759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19759:
---
Labels: ise (was: )
> Calcite engine. Review list of reserved keywords
>
>
> Key: IGNITE-19759
> URL: https://issues.apache.org/jira/browse/IGNITE-19759
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
>
> For the calcite engine we have too strict list of reserved keywords. For
> example, lexems such as "TYPE" and "OPTIONS" are reserved keywords and can't
> be used as columns or table names. But "TYPE" is frequently used by users as
> column name and we should exclude it from the list of reserved keywords (add
> it to non-reserved keywords, see {{config.fmpp}} file
> \{{nonReservedKeywords}} section). Other vendors allow to use "TYPE" as
> column name.
> We should also review the whole list of reserved keywords (see generated
> {{{}Parser.jj{}}}), perhaps some other keywords should be excluded from
> reserved list too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19767) Update Ignite dependency: Jetty
[ https://issues.apache.org/jira/browse/IGNITE-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-19767: --- Ignite Flags: Release Notes Required (was: Docs Required,Release Notes Required) > Update Ignite dependency: Jetty > --- > > Key: IGNITE-19767 > URL: https://issues.apache.org/jira/browse/IGNITE-19767 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.16 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update Jetty dependency 9.4.43.v20210629 to 9.4.51.v20230217 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19811) Continuous qieries backup acknowledge message sending fails for expired entries
Aleksey Plekhanov created IGNITE-19811:
--
Summary: Continuous qieries backup acknowledge message sending
fails for expired entries
Key: IGNITE-19811
URL: https://issues.apache.org/jira/browse/IGNITE-19811
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Expire entry event has {{null}} in topology version field (see
{{CacheContinuousQueryEntry}} constructor in the
{{CacheContinuousQueryManager#onEntryExpired}} method). When Backup acknowledge
is sending for such a message it silently (without warnings to log) fails with
NPE on {{GridDiscoveryManager#cacheGroupAffinityNodes}} ->
{{GridDiscoveryManager#resolveDiscoCache}} for {{null}} topology version (see
{{CacheContinuousQueryHandler#sendBackupAcknowledge}}).
This can lead to leaks in query entries buffer
({{CacheContinuousQueryEventBuffer#backupQ}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19818) Calcite engine. Query planning failed when cache size is too big
Aleksey Plekhanov created IGNITE-19818:
--
Summary: Calcite engine. Query planning failed when cache size is
too big
Key: IGNITE-19818
URL: https://issues.apache.org/jira/browse/IGNITE-19818
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
We use cache size as estimation for row count while planning, but use
{{cache.size()}} method, that returns int value. But if cache size is more than
{{Integer.MAX_VALUE}} we get an wrong size or even negative sometimes, which
cause assertion errors during planning.
We should fix it to {{cache.sizeLong()}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19818) Calcite engine. Query planning failed when cache size is too big
[
https://issues.apache.org/jira/browse/IGNITE-19818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19818:
---
Labels: ise (was: )
> Calcite engine. Query planning failed when cache size is too big
>
>
> Key: IGNITE-19818
> URL: https://issues.apache.org/jira/browse/IGNITE-19818
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
>
> We use cache size as estimation for row count while planning, but use
> {{cache.size()}} method, that returns int value. But if cache size is more
> than {{Integer.MAX_VALUE}} we get an wrong size or even negative sometimes,
> which cause assertion errors during planning.
> We should fix it to {{cache.sizeLong()}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19814) Calcite engine uses 0 as an inlineSize for index created by INT column
[
https://issues.apache.org/jira/browse/IGNITE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19814:
---
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
> Calcite engine uses 0 as an inlineSize for index created by INT column
> --
>
> Key: IGNITE-19814
> URL: https://issues.apache.org/jira/browse/IGNITE-19814
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Aleksey Plekhanov
>Priority: Minor
> Labels: calcite
> Attachments:
> 0001-IGNITE-19814-Add-calcite-test-for-integer-column-ind.patch
>
>
> Using the Calcite query engine the following SQL statement
> {code:sql}
> CREATE TABLE TEST (id INT, name VARCHAR, PRIMARY KEY(id))
> {code}
> creates index for INT id column with the inlineSize = 0.
> If the _key_type_ is specified in the WITH clause index is created correctly
> with the inlineSize = 5:
> {code:sql}
> CREATE TABLE TEST (id INT, name VARCHAR, PRIMARY KEY(id))
> WITH "key_type=Integer"
> {code}
> For H2 engine in both cases inlineSize is 5.
> Reproducer is attached as a unit test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19814) Calcite engine uses 0 as an inlineSize for index created by INT column
[
https://issues.apache.org/jira/browse/IGNITE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov reassigned IGNITE-19814:
--
Assignee: Aleksey Plekhanov
> Calcite engine uses 0 as an inlineSize for index created by INT column
> --
>
> Key: IGNITE-19814
> URL: https://issues.apache.org/jira/browse/IGNITE-19814
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Aleksey Plekhanov
>Priority: Minor
> Labels: calcite
> Attachments:
> 0001-IGNITE-19814-Add-calcite-test-for-integer-column-ind.patch
>
>
> Using the Calcite query engine the following SQL statement
> {code:sql}
> CREATE TABLE TEST (id INT, name VARCHAR, PRIMARY KEY(id))
> {code}
> creates index for INT id column with the inlineSize = 0.
> If the _key_type_ is specified in the WITH clause index is created correctly
> with the inlineSize = 5:
> {code:sql}
> CREATE TABLE TEST (id INT, name VARCHAR, PRIMARY KEY(id))
> WITH "key_type=Integer"
> {code}
> For H2 engine in both cases inlineSize is 5.
> Reproducer is attached as a unit test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19759) Calcite engine. Review list of reserved keywords
[
https://issues.apache.org/jira/browse/IGNITE-19759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19759:
---
Description:
For the calcite engine we have too strict list of reserved keywords. For
example, lexems such as "TYPE" and "OPTIONS" are reserved keywords and can't be
used as columns or table names. But "TYPE" is frequently used by users as
column name and we should exclude it from the list of reserved keywords (add it
to non-reserved keywords, see {{config.fmpp}} file {{nonReservedKeywords}}
section). Other vendors allow to use "TYPE" as column name.
On the other hand Calcite-based SQL engine in Ignite now allows to use some
keywords which should not be allowed as table or column names, for example,
such query executes without any problem:
{noformat}
sql("create table true (like varchar, and int, as int)");
sql("insert into true values ('1', 1, 1)");
sql("select as as as from true where like like '%' and and between and
and and");
{noformat}
Current list of reserved keywords copied from "Babel" dialect of Calcite.
Calcite has "default" dialect with default list of reserved keywords (see [1]),
this list is close to SQL stantard, but looks quite strict too.
Other vendors lists are less restrictive. For example, in SQL standard build-in
functions and all built-in types are reserved keywords, in MySQL built-in
functions are not reserved, but build-in types are reserved, in PostgreeSQL
only minimal amount of keywords required for correct parsing are reserved
(built-in functions are not reserved, built-in types are not reserved). See
comparison table [2]. Our old SQL engine based on H2 database and H2 reserved
keywords (see [3]). H2 approach is close to PostgreeSQL approach (minimal
amount of keywords are reserved). I propose to use such an approach for Ignite
too, to maximaze compatibility between our SQL engines.
[1] https://calcite.apache.org/docs/reference.html#keywords
[2] https://en.wikipedia.org/wiki/List_of_SQL_reserved_words
[3] https://www.h2database.com/html/advanced.html#keywords
was:
For the calcite engine we have too strict list of reserved keywords. For
example, lexems such as "TYPE" and "OPTIONS" are reserved keywords and can't be
used as columns or table names. But "TYPE" is frequently used by users as
column name and we should exclude it from the list of reserved keywords (add it
to non-reserved keywords, see {{config.fmpp}} file \{{nonReservedKeywords}}
section). Other vendors allow to use "TYPE" as column name.
We should also review the whole list of reserved keywords (see generated
{{{}Parser.jj{}}}), perhaps some other keywords should be excluded from
reserved list too.
> Calcite engine. Review list of reserved keywords
>
>
> Key: IGNITE-19759
> URL: https://issues.apache.org/jira/browse/IGNITE-19759
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For the calcite engine we have too strict list of reserved keywords. For
> example, lexems such as "TYPE" and "OPTIONS" are reserved keywords and can't
> be used as columns or table names. But "TYPE" is frequently used by users as
> column name and we should exclude it from the list of reserved keywords (add
> it to non-reserved keywords, see {{config.fmpp}} file {{nonReservedKeywords}}
> section). Other vendors allow to use "TYPE" as column name.
> On the other hand Calcite-based SQL engine in Ignite now allows to use some
> keywords which should not be allowed as table or column names, for example,
> such query executes without any problem:
> {noformat}
> sql("create table true (like varchar, and int, as int)");
> sql("insert into true values ('1', 1, 1)");
> sql("select as as as from true where like like '%' and and between
> and and and");
> {noformat}
> Current list of reserved keywords copied from "Babel" dialect of Calcite.
> Calcite has "default" dialect with default list of reserved keywords (see
> [1]), this list is close to SQL stantard, but looks quite strict too.
> Other vendors lists are less restrictive. For example, in SQL standard
> build-in functions and all built-in types are reserved keywords, in MySQL
> built-in functions are not reserved, but build-in types are reserved, in
> PostgreeSQL only minimal amount of keywords required for correct parsing are
> reserved (built-in functions are not reserved, built-in types are not
> reserved). See comparison table [2]. Our old SQL engine based on H2 database
> and H2 reserved keywords (see [3]). H2 approach is close to PostgreeSQL
> approach (minimal amount of keywords are reserved)
[jira] [Created] (IGNITE-19981) Calcite engine. Optimize mapping sending with query start request
Aleksey Plekhanov created IGNITE-19981: -- Summary: Calcite engine. Optimize mapping sending with query start request Key: IGNITE-19981 URL: https://issues.apache.org/jira/browse/IGNITE-19981 Project: Ignite Issue Type: Improvement Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently we send the whole fragment mapping with query start request to each node, but on the node we need only local mapping (we need only set of partition to process by current node). If there are a lot of nodes and a lot of partitions - mapping can take a lot of space -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19814) Calcite engine uses 0 as an inlineSize for index created by INT column
[
https://issues.apache.org/jira/browse/IGNITE-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-19814:
---
Labels: calcite ise (was: calcite)
> Calcite engine uses 0 as an inlineSize for index created by INT column
> --
>
> Key: IGNITE-19814
> URL: https://issues.apache.org/jira/browse/IGNITE-19814
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Aleksey Plekhanov
>Priority: Minor
> Labels: calcite, ise
> Fix For: 2.16
>
> Attachments:
> 0001-IGNITE-19814-Add-calcite-test-for-integer-column-ind.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Using the Calcite query engine the following SQL statement
> {code:sql}
> CREATE TABLE TEST (id INT, name VARCHAR, PRIMARY KEY(id))
> {code}
> creates index for INT id column with the inlineSize = 0.
> If the _key_type_ is specified in the WITH clause index is created correctly
> with the inlineSize = 5:
> {code:sql}
> CREATE TABLE TEST (id INT, name VARCHAR, PRIMARY KEY(id))
> WITH "key_type=Integer"
> {code}
> For H2 engine in both cases inlineSize is 5.
> Reproducer is attached as a unit test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20006) Calcite engine. Make table/index scan iterators yieldable
Aleksey Plekhanov created IGNITE-20006:
--
Summary: Calcite engine. Make table/index scan iterators yieldable
Key: IGNITE-20006
URL: https://issues.apache.org/jira/browse/IGNITE-20006
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Currently, index/table iterators can scan unpredictable count of cache entries
during one {{hasNext()}}/{{next()}} call. These iterators contain filter, which
applyed to each entry and row produced only for entries that satisfy filter. If
filter contains "always false" rule, one {{hasNext()}} call may scan entiry
table uninterruptably, without timeouts and yields to make another queries do
their job. We should fix this behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20006) Calcite engine. Make table/index scan iterators yieldable
[
https://issues.apache.org/jira/browse/IGNITE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-20006:
---
Labels: calcite ise (was: )
> Calcite engine. Make table/index scan iterators yieldable
> --
>
> Key: IGNITE-20006
> URL: https://issues.apache.org/jira/browse/IGNITE-20006
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, ise
>
> Currently, index/table iterators can scan unpredictable count of cache
> entries during one {{hasNext()}}/{{next()}} call. These iterators contain
> filter, which applyed to each entry and row produced only for entries that
> satisfy filter. If filter contains "always false" rule, one {{hasNext()}}
> call may scan entiry table uninterruptably, without timeouts and yields to
> make another queries do their job. We should fix this behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20010) Calcite engine. Query leaks on remote fragment initialization phase failure
Aleksey Plekhanov created IGNITE-20010: -- Summary: Calcite engine. Query leaks on remote fragment initialization phase failure Key: IGNITE-20010 URL: https://issues.apache.org/jira/browse/IGNITE-20010 Project: Ignite Issue Type: Bug Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov If any error occurs on remote fragment initialization phase (for example, LogicalRelImplementor throws an exception), query doesn't removed from running query manager -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20038) [Thin Client] Cache operations with PA enabled can fail with BufferUnderflowException
[
https://issues.apache.org/jira/browse/IGNITE-20038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-20038:
---
Affects Version/s: 2.15
2.14
> [Thin Client] Cache operations with PA enabled can fail with
> BufferUnderflowException
> ---
>
> Key: IGNITE-20038
> URL: https://issues.apache.org/jira/browse/IGNITE-20038
> Project: Ignite
> Issue Type: Task
>Affects Versions: 2.14, 2.15
> Environment:
>Reporter: Mikhail Petrov
>Assignee: Mikhail Petrov
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Cache operations with PA enabled can fail on thin clients with
> BufferUnderflowException due to broken ClientCachePartitionAwarenessGroup
> serialization.
> Exception:
> {code:java}
> java.nio.BufferUnderflowException
> at java.nio.Buffer.nextGetIndex(Buffer.java:532)
> at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:366)
> at
> org.apache.ignite.internal.binary.streams.BinaryByteBufferInputStream.readInt(BinaryByteBufferInputStream.java:111)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.readInt(BinaryReaderExImpl.java:746)
> at
> org.apache.ignite.internal.client.thin.ClientCacheAffinityMapping.readCacheKeyConfiguration(ClientCacheAffinityMapping.java:240)
> at
> org.apache.ignite.internal.client.thin.ClientCacheAffinityMapping.readResponse(ClientCacheAffinityMapping.java:197)
> at
> org.apache.ignite.internal.client.thin.ClientCacheAffinityContext.readPartitionsUpdateResponse(ClientCacheAffinityContext.java:154)
> at
> org.apache.ignite.internal.client.thin.TcpClientChannel.receive(TcpClientChannel.java:412)
> at
> org.apache.ignite.internal.client.thin.TcpClientChannel.service(TcpClientChannel.java:311)
> at
> org.apache.ignite.internal.client.thin.ThinClientAbstractPartitionAwarenessTest$TestTcpClientChannel.service(ThinClientAbstractPartitionAwarenessTest.java:345)
> at
> org.apache.ignite.internal.client.thin.ReliableChannel.lambda$affinityInfoIsUpToDate$6(ReliableChannel.java:423)
> at
> org.apache.ignite.internal.client.thin.ReliableChannel.applyOnNodeChannel(ReliableChannel.java:746)
> at
> org.apache.ignite.internal.client.thin.ReliableChannel.affinityInfoIsUpToDate(ReliableChannel.java:422)
> at
> org.apache.ignite.internal.client.thin.ReliableChannel.affinityService(ReliableChannel.java:316)
> at
> org.apache.ignite.internal.client.thin.TcpClientCache.txAwareService(TcpClientCache.java:1139)
> at
> org.apache.ignite.internal.client.thin.TcpClientCache.cacheSingleKeyOperation(TcpClientCache.java:1198)
> at
> org.apache.ignite.internal.client.thin.TcpClientCache.get(TcpClientCache.java:146)
> at
> org.apache.ignite.internal.client.thin.ThinClientPartitionAwarenessStableTopologyTest.lambda$testMultipleCacheGroupPartitionsRequest$8(ThinClientPartitionAwarenessStableTopologyTest.java:250)
> at
> org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$4(GridTestUtils.java:1229)
> at
> org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1570)
> at
> org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:88)
> {code}
> Reproducer:
> {code:java}
> /** */
> @Test
> public void test() throws Exception {
> Ignite ignite = startGrid(0);
> ignite.createCache(new
> CacheConfiguration<>("test-cache-0").setCacheMode(REPLICATED));
> ignite.createCache(new
> CacheConfiguration<>("test-cache-1").setCacheMode(PARTITIONED));
> try (IgniteClient cli = Ignition.startClient(new
> ClientConfiguration().setAddresses("127.0.0.1:10800"))) {
> ClientCacheAffinityContext affCtx =
> ((TcpIgniteClient)cli).reliableChannel().affinityContext();
> IgniteInternalFuture replCacheOpFut;
> IgniteInternalFuture partCacheOpFut;
> synchronized (affCtx.cacheKeyMapperFactoryMap) {
> partCacheOpFut = GridTestUtils.runAsync(() ->
> cli.cache("test-cache-0").get(0));
> replCacheOpFut = GridTestUtils.runAsync(() ->
> cli.cache("test-cache-1").get(0));
> GridTestUtils.waitForCondition(
> () ->
> affCtx.pendingCacheIds.containsAll(F.transform(asList("test-cache-0",
> "test-cache-1"), CU::cacheId)),
> getTestTimeout()
> );
> }
> partCacheOpFut.get();
> replCacheOpFut.get();
> }
> }
> {code}
> Explanation:
> Take a look at the ClientCachePartitionAwarenessGroup#write method. During
> its serialization we write to the buffer the variable "dfltAffinity". Then
> take a
[jira] [Created] (IGNITE-20079) Calcite engine. Write additional performance statistics info for queries
Aleksey Plekhanov created IGNITE-20079: -- Summary: Calcite engine. Write additional performance statistics info for queries Key: IGNITE-20079 URL: https://issues.apache.org/jira/browse/IGNITE-20079 Project: Ignite Issue Type: Improvement Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently, we write query SQL/query time/query page reads to performance statistics. But it will be useful to write also real plans and count of entries scanned from caches to detect problems (plan can be changed during the time, so explain plan later can show differrent value) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20006) Calcite engine. Make table/index scan iterators yieldable
[
https://issues.apache.org/jira/browse/IGNITE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-20006:
---
Ignite Flags: (was: Release Notes Required)
> Calcite engine. Make table/index scan iterators yieldable
> --
>
> Key: IGNITE-20006
> URL: https://issues.apache.org/jira/browse/IGNITE-20006
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, ise
> Fix For: 2.16
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently, index/table iterators can scan unpredictable count of cache
> entries during one {{hasNext()}}/{{next()}} call. These iterators contain
> filter, which applyed to each entry and row produced only for entries that
> satisfy filter. If filter contains "always false" rule, one {{hasNext()}}
> call may scan entiry table uninterruptably, without timeouts and yields to
> make another queries do their job. We should fix this behaviour.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20194) Calcite engine. Dependency common-codec required for some functions
Aleksey Plekhanov created IGNITE-20194: -- Summary: Calcite engine. Dependency common-codec required for some functions Key: IGNITE-20194 URL: https://issues.apache.org/jira/browse/IGNITE-20194 Project: Ignite Issue Type: Bug Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Some functions (md5, soundex) require common-codec dependency, but this dependency is not included explicitly to dependencies list and doesn't added to calcite library folder on build. Tests run with transitive dependencies and don't trigger the problem. Queries fail only when run on Ignite started from binary release package. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20353) Calcite engine. Clause 'WITH affinity_key=...' silently ignored when wrong column is specified
Aleksey Plekhanov created IGNITE-20353:
--
Summary: Calcite engine. Clause 'WITH affinity_key=...' silently
ignored when wrong column is specified
Key: IGNITE-20353
URL: https://issues.apache.org/jira/browse/IGNITE-20353
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Calcite-based SQL engine silently ignore clause WITH affinity_key=... when
wrong column is specified. H2-based engine in this case throw an error:
{noformat}
org.apache.ignite.internal.processors.query.IgniteSQLException: Affinity key
column with given name not found: test
{noformat}
Reproducer:
{code:sql}
CREATE TABLE order_items (id varchar, orderId int, sku varchar, PRIMARY KEY
(id, orderId) WITH "affinity_key=test");
{code}
Also, there is some problem with case-sensitivity, for example:
{code:sql}
CREATE TABLE order_items (id varchar, orderId int, sku varchar, PRIMARY KEY
(id, orderId) WITH "affinity_key=orderId");
{code}
Works well for H2-based engine ({{orderId}} in {{affinity_key}} converted to
{{ORDERID}} and matches {{orderId}} columns alias), but silently ignired for
Calcite-based engine ({{orderId}} in {{affinity_key}} remains without case
change). But:
{code:sql}
CREATE TABLE order_items (id varchar, orderId int, sku varchar, PRIMARY KEY
(id, orderId) WITH "affinity_key=ORDERID");
{code}
Works well for both engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20382) Calcite engine. Add metrics for CalciteQueryExecutor thread pool
Aleksey Plekhanov created IGNITE-20382: -- Summary: Calcite engine. Add metrics for CalciteQueryExecutor thread pool Key: IGNITE-20382 URL: https://issues.apache.org/jira/browse/IGNITE-20382 Project: Ignite Issue Type: Improvement Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currently, all thread-pools except CalciteQueryExecutor can be monitored via metrics, we should add this ability for CalciteQueryExecutor too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20383) Calcite engine. Convert one input of a join to the broadcast distribution
Aleksey Plekhanov created IGNITE-20383:
--
Summary: Calcite engine. Convert one input of a join to the
broadcast distribution
Key: IGNITE-20383
URL: https://issues.apache.org/jira/browse/IGNITE-20383
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Sometimes, if join inputs are not collocated it's worth to broadcast one of the
inputs, for example, query:
{code:sql}
SELECT * FROM emps WHERE emps.salary = (SELECT AVG(emps.salary) FROM emps){code}
Currently has plan:
{noformat}
IgniteProject(ID=[$0], NAME=[$1], SALARY=[$2])
IgniteNestedLoopJoin(condition=[=($2, $3)], joinType=[inner])
IgniteExchange(distribution=[single])
IgniteTableScan(table=[[PUBLIC, EMPS]])
IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[TST],
requiredColumns=[{2}], collation=[[2 ASC-nulls-first]])
{noformat}
But this plan is not optimal, since we should send entire table EMP from all
nodes to the single node. For such a query it's better to broadcast result of
the aggregation, in this case plan will be something like:
{noformat}
IgniteExchange(distribution=[single])
IgniteProject(ID=[$0], NAME=[$1], SALARY=[$2])
IgniteCorrelatedNestedLoopJoin(...)
IgniteExchange(distribution=[broadcast])
IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[SALARY_IDX])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[SALARY_IDX])
{noformat}
But currently we don't try to convert any of the join inputs to the broadcast
distribution. We should try to do this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20383) Calcite engine. Convert one input of a join to the broadcast distribution
[
https://issues.apache.org/jira/browse/IGNITE-20383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-20383:
---
Description:
Sometimes, if join inputs are not collocated it's worth to broadcast one of the
inputs, for example, query:
{code:sql}
SELECT * FROM emps WHERE emps.salary = (SELECT AVG(emps.salary) FROM emps){code}
Currently has plan:
{noformat}
IgniteProject(ID=[$0], NAME=[$1], SALARY=[$2])
IgniteNestedLoopJoin(condition=[=($2, $3)], joinType=[inner])
IgniteExchange(distribution=[single])
IgniteTableScan(table=[[PUBLIC, EMPS]])
IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[TST],
requiredColumns=[{2}], collation=[[2 ASC-nulls-first]])
{noformat}
But this plan is not optimal, since we should send entire table EMP from all
nodes to the single node. For such a query it's better to broadcast result of
the aggregation, in this case plan will be something like:
{noformat}
IgniteExchange(distribution=[single])
IgniteProject(...)
IgniteCorrelatedNestedLoopJoin(...)
IgniteExchange(distribution=[broadcast])
IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[SALARY_IDX])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[SALARY_IDX])
{noformat}
But currently we don't try to convert any of the join inputs to the broadcast
distribution. We should try to do this.
was:
Sometimes, if join inputs are not collocated it's worth to broadcast one of the
inputs, for example, query:
{code:sql}
SELECT * FROM emps WHERE emps.salary = (SELECT AVG(emps.salary) FROM emps){code}
Currently has plan:
{noformat}
IgniteProject(ID=[$0], NAME=[$1], SALARY=[$2])
IgniteNestedLoopJoin(condition=[=($2, $3)], joinType=[inner])
IgniteExchange(distribution=[single])
IgniteTableScan(table=[[PUBLIC, EMPS]])
IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[TST],
requiredColumns=[{2}], collation=[[2 ASC-nulls-first]])
{noformat}
But this plan is not optimal, since we should send entire table EMP from all
nodes to the single node. For such a query it's better to broadcast result of
the aggregation, in this case plan will be something like:
{noformat}
IgniteExchange(distribution=[single])
IgniteProject(ID=[$0], NAME=[$1], SALARY=[$2])
IgniteCorrelatedNestedLoopJoin(...)
IgniteExchange(distribution=[broadcast])
IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[SALARY_IDX])
IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[SALARY_IDX])
{noformat}
But currently we don't try to convert any of the join inputs to the broadcast
distribution. We should try to do this.
> Calcite engine. Convert one input of a join to the broadcast distribution
> -
>
> Key: IGNITE-20383
> URL: https://issues.apache.org/jira/browse/IGNITE-20383
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, ise
>
> Sometimes, if join inputs are not collocated it's worth to broadcast one of
> the inputs, for example, query:
> {code:sql}
> SELECT * FROM emps WHERE emps.salary = (SELECT AVG(emps.salary) FROM
> emps){code}
> Currently has plan:
> {noformat}
> IgniteProject(ID=[$0], NAME=[$1], SALARY=[$2])
> IgniteNestedLoopJoin(condition=[=($2, $3)], joinType=[inner])
> IgniteExchange(distribution=[single])
> IgniteTableScan(table=[[PUBLIC, EMPS]])
> IgniteReduceHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
> IgniteExchange(distribution=[single])
> IgniteMapHashAggregate(group=[{}], AVG(EMPS.SALARY)=[AVG($0)])
> IgniteIndexScan(table=[[PUBLIC, EMPS]], index=[TST],
> requiredColumns=[{2}], collation=[[2 ASC-nulls-first]])
> {noformat}
> But this plan is not optimal, since we should send entire table EMP from all
> nodes to the single node. For such a query it's better to broadcast result of
> the aggregation, in this case plan will be something like:
> {noformat}
> IgniteExchange(distribution=[single])
> IgniteProject(...)
[jira] [Resolved] (IGNITE-18330) Fix javadoc in Transaction#resume(), Transaction#suspend
[ https://issues.apache.org/jira/browse/IGNITE-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov resolved IGNITE-18330. Fix Version/s: 2.16 Resolution: Fixed [~__zz250], looks good to me. Merged to master. Thanks for the contribution! > Fix javadoc in Transaction#resume(), Transaction#suspend > > > Key: IGNITE-18330 > URL: https://issues.apache.org/jira/browse/IGNITE-18330 > Project: Ignite > Issue Type: Improvement >Reporter: Luchnikov Alexander >Assignee: bin.yin >Priority: Trivial > Labels: ise, newbie > Fix For: 2.16 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > After implementation IGNITE-5714, this api can be used with pessimistic > transactions. > Now in javadoc - Supported only for optimistic transactions.. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19981) Calcite engine. Optimize mapping sending with query start request
[ https://issues.apache.org/jira/browse/IGNITE-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-19981: --- Ignite Flags: (was: Release Notes Required) > Calcite engine. Optimize mapping sending with query start request > - > > Key: IGNITE-19981 > URL: https://issues.apache.org/jira/browse/IGNITE-19981 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: ise > Time Spent: 50m > Remaining Estimate: 0h > > Currently we send the whole fragment mapping with query start request to each > node, but on the node we need only local mapping (we need only set of > partition to process by current node). If there are a lot of nodes and a lot > of partitions - mapping can take a lot of space -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-19981) Calcite engine. Optimize mapping sending with query start request
[ https://issues.apache.org/jira/browse/IGNITE-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov resolved IGNITE-19981. Resolution: Won't Fix Looks like there is no performance boost after implementation. Benchmarks show reduced network usage, but increased CPU usage and lower overall throughput. > Calcite engine. Optimize mapping sending with query start request > - > > Key: IGNITE-19981 > URL: https://issues.apache.org/jira/browse/IGNITE-19981 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: ise > Time Spent: 50m > Remaining Estimate: 0h > > Currently we send the whole fragment mapping with query start request to each > node, but on the node we need only local mapping (we need only set of > partition to process by current node). If there are a lot of nodes and a lot > of partitions - mapping can take a lot of space -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20462) Idle_verify prints partitions hash conflicts when entries are expiring concurrently
Aleksey Plekhanov created IGNITE-20462: -- Summary: Idle_verify prints partitions hash conflicts when entries are expiring concurrently Key: IGNITE-20462 URL: https://issues.apache.org/jira/browse/IGNITE-20462 Project: Ignite Issue Type: Bug Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Background process of entries expire (ttl-cleaner-worker) always working on activated cluster, so, during idle_verify execution entries still can be expired even without workload on cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20487) Add system views for IgniteStripedThreadPoolExecutor
Aleksey Plekhanov created IGNITE-20487:
--
Summary: Add system views for IgniteStripedThreadPoolExecutor
Key: IGNITE-20487
URL: https://issues.apache.org/jira/browse/IGNITE-20487
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Currently, we have two views for {{StripedExecutor}}'s ({{stripedExecSvc}},
{{dataStreamerExecSvc}}), but we have another type of striped executor:
{{IgniteStripedThreadPoolExecutor}} ({{rebalanceStripedExecSvc}},
{{callbackExecSvc}}, calcite query task executor), it will be helpful to have
system views for these executors too.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20488) Add metrics for count of page merges/splits in BPlusTree
Aleksey Plekhanov created IGNITE-20488: -- Summary: Add metrics for count of page merges/splits in BPlusTree Key: IGNITE-20488 URL: https://issues.apache.org/jira/browse/IGNITE-20488 Project: Ignite Issue Type: Improvement Reporter: Aleksey Plekhanov It will be helpful to have such metrics as count of merges/splits in BPlusTree -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20501) Calcite engine. Memory leak in MailboxRegistryImpl#remotes on JOINs
Aleksey Plekhanov created IGNITE-20501:
--
Summary: Calcite engine. Memory leak in
MailboxRegistryImpl#remotes on JOINs
Key: IGNITE-20501
URL: https://issues.apache.org/jira/browse/IGNITE-20501
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
When JOIN relational operator is executed, downstream of JOIN can be closed if
only one side of JOIN is already drained (see last lines of
{{MergeJoinNode.InnerJoin#join}}, for example). In this case query can be
prematurely closed, and after this, message for another side of join can arrive
and register new {{Inbox}} (see {{ExchangeServiceImpl#onMessage(UUID,
QueryBatchMessage)}}), that never will be unregistered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-7346) Enable Ignite cache events per cache
[
https://issues.apache.org/jira/browse/IGNITE-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov reassigned IGNITE-7346:
-
Assignee: Aleksey Plekhanov
> Enable Ignite cache events per cache
>
>
> Key: IGNITE-7346
> URL: https://issues.apache.org/jira/browse/IGNITE-7346
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis Magda
>Assignee: Aleksey Plekhanov
> Labels: usability
> Fix For: 2.4
>
>
> Whenever a cache event from {{EventType.EVTS_CACHE}},
> {{EventType.EVTS_CACHE_REBALANCE}} or {{EventType.EVTS_CACHE_QUERY}} set is
> enabled and a listener is registered, the cluster will accumulate and send
> out events for all the deployed caches.
> Such behavior brings down performance especially if hundreds and thousands of
> caches are used while the user is interested in the monitoring of specific
> caches.
> Let's add a configuration property like
> {{CacheConfiguration.setEventsEnabled}} to enable or disable the events per
> cache. The events will be enabled by default to comply with existing
> behavior.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (IGNITE-6940) Thread Starvation monitoring
[ https://issues.apache.org/jira/browse/IGNITE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov reassigned IGNITE-6940: - Assignee: Aleksey Plekhanov (was: Andrey Kuznetsov) > Thread Starvation monitoring > > > Key: IGNITE-6940 > URL: https://issues.apache.org/jira/browse/IGNITE-6940 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Kuznetsov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: iep-7 > Fix For: 2.5 > > > Ignite Thread Pools Starvation > Description > This situation can occur if user submits tasks that recursively submit more > tasks and synchronously wait for results. Jobs arrive to worker nodes and are > queued forever since there are no free threads in public pool since all > threads are waiting for job results. > Detection and Solution > Task timeout can be set for tasks, so task gets canceled automatically. > Web Console should provide ability to cancel any task and job from UI. > Report > Timed out tasks and jobs should be reported on Web Console and reported to > logs. We need to introduce new config property to set timeout for reported > jobs. > Log record and Web Console should include: > - Master node ID > - Start time -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-6980) Automatic cancelling of hanging Ignite operations
[ https://issues.apache.org/jira/browse/IGNITE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov reassigned IGNITE-6980: - Assignee: Aleksey Plekhanov > Automatic cancelling of hanging Ignite operations > - > > Key: IGNITE-6980 > URL: https://issues.apache.org/jira/browse/IGNITE-6980 > Project: Ignite > Issue Type: Bug >Reporter: Denis Magda >Assignee: Aleksey Plekhanov >Priority: Critical > Labels: iep-7, important > Fix For: 2.5 > > > If an Ignite operation hangs due to some reason due to an internal problem or > buggy application code it needs to eventual fail after a timeout fires. > Take atomic operations case brought by Val to our attention recently: > > [http://apache-ignite-developers.2346864.n4.nabble.com/Timeouts-in-atomic-cache-td19839.html] > An application must not freeze waiting for a human being intervention if an > atomic update fails internally. > Even more, I would let all possible operation to fail after a timeout fires: > - Ignite compute computations (covered by IGNITE-6940). > - Ignite services calls. > - Atomic/transactional cache updates. > - SQL queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7527) SQL system view for current node transactions
Aleksey Plekhanov created IGNITE-7527: - Summary: SQL system view for current node transactions Key: IGNITE-7527 URL: https://issues.apache.org/jira/browse/IGNITE-7527 Project: Ignite Issue Type: Improvement Components: sql Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Implement SQL system view to show active transactions on local node. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7527) SQL system view for current node transactions
[ https://issues.apache.org/jira/browse/IGNITE-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-7527: -- Issue Type: Task (was: Improvement) > SQL system view for current node transactions > - > > Key: IGNITE-7527 > URL: https://issues.apache.org/jira/browse/IGNITE-7527 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: iep-13, sql > > Implement SQL system view to show active transactions on local node. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-5265) Eviction Rate memory metric to be implemented
[ https://issues.apache.org/jira/browse/IGNITE-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov reassigned IGNITE-5265: - Assignee: Aleksey Plekhanov (was: Ivan Rakov) > Eviction Rate memory metric to be implemented > - > > Key: IGNITE-5265 > URL: https://issues.apache.org/jira/browse/IGNITE-5265 > Project: Ignite > Issue Type: Task > Components: general >Reporter: Sergey Chugunov >Assignee: Aleksey Plekhanov >Priority: Major > > Eviction rate metric is declared on *MemoryMetrics* interface but isn't > implemented in *MemoryMetricsImpl* (current implementation returns 0 right > away). > This metric must be implemented identically to *allocationRate* metric from > the same interface (algorithm for *allocationRate* can be reused with minor > refactoring). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IGNITE-6940) Thread Starvation monitoring
[ https://issues.apache.org/jira/browse/IGNITE-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov resolved IGNITE-6940. --- Resolution: Won't Fix For now Ignite already dump threads to log when starvation is detected. Timeout for tasks and jobs can be set using IgniteCompute.withTimeout method. Task monitoring will be implemented as system view (new ticket will be created). > Thread Starvation monitoring > > > Key: IGNITE-6940 > URL: https://issues.apache.org/jira/browse/IGNITE-6940 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Kuznetsov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: iep-7 > Fix For: 2.5 > > > Ignite Thread Pools Starvation > Description > This situation can occur if user submits tasks that recursively submit more > tasks and synchronously wait for results. Jobs arrive to worker nodes and are > queued forever since there are no free threads in public pool since all > threads are waiting for job results. > Detection and Solution > Task timeout can be set for tasks, so task gets canceled automatically. > Web Console should provide ability to cancel any task and job from UI. > Report > Timed out tasks and jobs should be reported on Web Console and reported to > logs. We need to introduce new config property to set timeout for reported > jobs. > Log record and Web Console should include: > - Master node ID > - Start time -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7700) SQL system view for list of nodes
Aleksey Plekhanov created IGNITE-7700: - Summary: SQL system view for list of nodes Key: IGNITE-7700 URL: https://issues.apache.org/jira/browse/IGNITE-7700 Project: Ignite Issue Type: Task Components: sql Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Fix For: 2.5 Implement SQL system view to show list of nodes in topology. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7701) SQL system view for node attributes
Aleksey Plekhanov created IGNITE-7701: - Summary: SQL system view for node attributes Key: IGNITE-7701 URL: https://issues.apache.org/jira/browse/IGNITE-7701 Project: Ignite Issue Type: Task Components: sql Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Fix For: 2.5 Implement SQL system view to show attributes for each node in topology. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7781) JMX beans documentation
[ https://issues.apache.org/jira/browse/IGNITE-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-7781: -- Labels: documentation jmx (was: docuentation jmx) > JMX beans documentation > --- > > Key: IGNITE-7781 > URL: https://issues.apache.org/jira/browse/IGNITE-7781 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Aleksey Plekhanov >Priority: Major > Labels: documentation, jmx > Attachments: MXBeansJavaDoc.zip > > > There are about 30 JMX beans implemented in Ignite, but there is no > documentation for most of these beans, except memory metrics and brief > overview of cache metrics. > Folowing MBeans/MXBeans can be documented: > ||MBeans group||Interface||Path||Comment|| > |Eviction > metrics|org.apache.ignite.cache.eviction.fifo.FifoEvictionPolicyMBean, > org.apache.ignite.cache.eviction.igfs.IgfsPerBlockLruEvictionPolicyMXBean, > org.apache.ignite.cache.eviction.lru.LruEvictionPolicyMBean, > org.apache.ignite.cache.eviction.sorted.SortedEvictionPolicyMBean|org.apache|For > each cache, if eviction for cache is configured| > |Cache group > metrics|org.apache.ignite.mxbean.CacheGroupMetricsMXBean|org.apache..."Cache > groups".|For each group| > |Cache > metrics|org.apache.ignite.mxbean.CacheMetricsMXBean|org.apacheCacheClusterMetricsMXBeanImpl, > > org.apacheCacheLocalMetricsMXBeanImpl|For > each cache| > |Cluster > metrics|org.apache.ignite.mxbean.ClusterMetricsMXBean|org.apache...Kernal.ClusterLocalNodeMetricsMXBeanImpl, > org.apache...Kernal.ClusterMetricsMXBeanImpl| > |Data region (memory) > metrics|org.apache.ignite.mxbean.DataRegionMetricsMXBean|org.apache...DataRegionMetrics.|For > each data region| > |Data storage > metrics|org.apache.ignite.mxbean.DataStorageMetricsMXBean|org.apache..."Persistent > Store".DataStorageMetrics| > |Instance > information|org.apache.ignite.mxbean.IgniteMXBean|org.apache...Kernal.IgniteKernal| > |Ignition > state|org.apache.ignite.mxbean.IgnitionMXBean|org.apache..Kernel.Ignition| > |Thread pools metrics|org.apache.ignite.mxbean.ThreadPoolMXBean, > org.apache.ignite.mxbean.StripedExecutorMXBean|org.apache..."Thread > Pools".|For each executor| > |SPI|org.apache.ignite.spi.cache.CacheCheckpointSpiMBean, > org.apache.ignite.spi.collision.fifoqueue.FifoQueueCollisionSpiMBean, > org.apache.ignite.spi.collision.jobstealing.JobStealingCollisionSpiMBean, > org.apache.ignite.spi.collision.priorityqueue.PriorityQueueCollisionSpiMBean, > org.apache.ignite.spi.communication.tcp.TcpCommunicationSpiMBean, > org.apache.ignite.spi.deployment.local.LocalDeploymentSpiMBean, > org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpiMBean, > org.apache.ignite.spi.eventstorage.memory.MemoryEventStorageSpiMBean, > org.apache.ignite.spi.failover.always.AlwaysFailoverSpiMBean, > org.apache.ignite.spi.failover.jobstealing.JobStealingFailoverSpiMBean, > org.apache.ignite.spi.failover.never.NeverFailoverSpiMBean, > org.apache.ignite.spi.jdbc.JdbcCheckpointSpiMBean, > org.apache.ignite.spi.loadbalancing.adaptive.AdaptiveLoadBalancingSpiMBean, > org.apache.ignite.spi.loadbalancing.roundrobin.RoundRobinLoadBalancingSpiMBean, > > org.apache.ignite.spi.loadbalancing.weightedrandom.WeightedRandomLoadBalancingSpiMBean, > > org.apache.ignite.spi.sharedfs.SharedFsCheckpointSpiMBean|org.apache...SPIs.| > > JavaDoc for this beans is attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7781) JMX beans documentation
Aleksey Plekhanov created IGNITE-7781: - Summary: JMX beans documentation Key: IGNITE-7781 URL: https://issues.apache.org/jira/browse/IGNITE-7781 Project: Ignite Issue Type: Task Components: documentation Reporter: Aleksey Plekhanov Attachments: MXBeansJavaDoc.zip There are about 30 JMX beans implemented in Ignite, but there is no documentation for most of these beans, except memory metrics and brief overview of cache metrics. Folowing MBeans/MXBeans can be documented: ||MBeans group||Interface||Path||Comment|| |Eviction metrics|org.apache.ignite.cache.eviction.fifo.FifoEvictionPolicyMBean, org.apache.ignite.cache.eviction.igfs.IgfsPerBlockLruEvictionPolicyMXBean, org.apache.ignite.cache.eviction.lru.LruEvictionPolicyMBean, org.apache.ignite.cache.eviction.sorted.SortedEvictionPolicyMBean|org.apache|For each cache, if eviction for cache is configured| |Cache group metrics|org.apache.ignite.mxbean.CacheGroupMetricsMXBean|org.apache..."Cache groups".|For each group| |Cache metrics|org.apache.ignite.mxbean.CacheMetricsMXBean|org.apacheCacheClusterMetricsMXBeanImpl, org.apacheCacheLocalMetricsMXBeanImpl|For each cache| |Cluster metrics|org.apache.ignite.mxbean.ClusterMetricsMXBean|org.apache...Kernal.ClusterLocalNodeMetricsMXBeanImpl, org.apache...Kernal.ClusterMetricsMXBeanImpl| |Data region (memory) metrics|org.apache.ignite.mxbean.DataRegionMetricsMXBean|org.apache...DataRegionMetrics.|For each data region| |Data storage metrics|org.apache.ignite.mxbean.DataStorageMetricsMXBean|org.apache..."Persistent Store".DataStorageMetrics| |Instance information|org.apache.ignite.mxbean.IgniteMXBean|org.apache...Kernal.IgniteKernal| |Ignition state|org.apache.ignite.mxbean.IgnitionMXBean|org.apache..Kernel.Ignition| |Thread pools metrics|org.apache.ignite.mxbean.ThreadPoolMXBean, org.apache.ignite.mxbean.StripedExecutorMXBean|org.apache..."Thread Pools".|For each executor| |SPI|org.apache.ignite.spi.cache.CacheCheckpointSpiMBean, org.apache.ignite.spi.collision.fifoqueue.FifoQueueCollisionSpiMBean, org.apache.ignite.spi.collision.jobstealing.JobStealingCollisionSpiMBean, org.apache.ignite.spi.collision.priorityqueue.PriorityQueueCollisionSpiMBean, org.apache.ignite.spi.communication.tcp.TcpCommunicationSpiMBean, org.apache.ignite.spi.deployment.local.LocalDeploymentSpiMBean, org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpiMBean, org.apache.ignite.spi.eventstorage.memory.MemoryEventStorageSpiMBean, org.apache.ignite.spi.failover.always.AlwaysFailoverSpiMBean, org.apache.ignite.spi.failover.jobstealing.JobStealingFailoverSpiMBean, org.apache.ignite.spi.failover.never.NeverFailoverSpiMBean, org.apache.ignite.spi.jdbc.JdbcCheckpointSpiMBean, org.apache.ignite.spi.loadbalancing.adaptive.AdaptiveLoadBalancingSpiMBean, org.apache.ignite.spi.loadbalancing.roundrobin.RoundRobinLoadBalancingSpiMBean, org.apache.ignite.spi.loadbalancing.weightedrandom.WeightedRandomLoadBalancingSpiMBean, org.apache.ignite.spi.sharedfs.SharedFsCheckpointSpiMBean|org.apache...SPIs.| JavaDoc for this beans is attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7818) Incorrect assertion in PDS page eviction method
Aleksey Plekhanov created IGNITE-7818:
-
Summary: Incorrect assertion in PDS page eviction method
Key: IGNITE-7818
URL: https://issues.apache.org/jira/browse/IGNITE-7818
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
There is assertion in the method
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#removePageForReplacement:
{code:java}
assert relRmvAddr != INVALID_REL_PTR;{code}
Which seems potentially dangerous. In some rare cases, when count of
interations more then 40% of allocated pages and all processed pages are
aquired, the {{relRmvAddr}} variable will be uninitialized and
{{AssertionException}} will be thrown. But it's a correct case and page to
evict can be found later in the method {{tryToFindSequentially.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7818) Incorrect assertion in PDS page eviction method
[
https://issues.apache.org/jira/browse/IGNITE-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-7818:
--
Description:
There is an assertion in the method
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#removePageForReplacement:
{code:java}
assert relRmvAddr != INVALID_REL_PTR;{code}
Which seems potentially dangerous. In some rare cases, when count of
interations more then 40% of allocated pages and all processed pages are
aquired, the {{relRmvAddr}} variable will be uninitialized and
{{AssertionException}} will be thrown. But it's a correct case and page to
evict can be found later in the method {{tryToFindSequentially.}}
was:
There is assertion in the method
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#removePageForReplacement:
{code:java}
assert relRmvAddr != INVALID_REL_PTR;{code}
Which seems potentially dangerous. In some rare cases, when count of
interations more then 40% of allocated pages and all processed pages are
aquired, the {{relRmvAddr}} variable will be uninitialized and
{{AssertionException}} will be thrown. But it's a correct case and page to
evict can be found later in the method {{tryToFindSequentially.}}
> Incorrect assertion in PDS page eviction method
> ---
>
> Key: IGNITE-7818
> URL: https://issues.apache.org/jira/browse/IGNITE-7818
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Priority: Major
>
> There is an assertion in the method
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#removePageForReplacement:
>
> {code:java}
> assert relRmvAddr != INVALID_REL_PTR;{code}
> Which seems potentially dangerous. In some rare cases, when count of
> interations more then 40% of allocated pages and all processed pages are
> aquired, the {{relRmvAddr}} variable will be uninitialized and
> {{AssertionException}} will be thrown. But it's a correct case and page to
> evict can be found later in the method {{tryToFindSequentially.}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-7831) Throw Exceptions instead of AssertionErrors when reading from corrupted persistence
[ https://issues.apache.org/jira/browse/IGNITE-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov reassigned IGNITE-7831: - Assignee: Aleksey Plekhanov > Throw Exceptions instead of AssertionErrors when reading from corrupted > persistence > --- > > Key: IGNITE-7831 > URL: https://issues.apache.org/jira/browse/IGNITE-7831 > Project: Ignite > Issue Type: Improvement >Reporter: Ilya Lantukh >Assignee: Aleksey Plekhanov >Priority: Major > Labels: iep-14 > > There are a few places in our code where we explicitly throw AssertionErrors > due to inability to correctly read data from persistence and many more places > where we make assertions based on read values. > Assertions are used to indicate problems in internal logic, while persistence > might also get corrupted by various external reasons. It also makes uniform > handling of such issues considerably harder, because exception handling logic > in Ignite ignores Errors. If we want to improve stability and minimize > consequenses of pesistence corruption, we should replace all those > AssertionErrors and asserts with Exceptions, so that current exception > handling mechanisms could be reduce. In a number of situations it means that > instead of causing cluster-wide hang-up problematic node will be > automatically killed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-7831) Throw Exceptions instead of AssertionErrors when reading from corrupted persistence
[
https://issues.apache.org/jira/browse/IGNITE-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383714#comment-16383714
]
Aleksey Plekhanov commented on IGNITE-7831:
---
[~ilantukh], I search for assertions in the persistance implementation, there
are about 7 hundreds occurrencies, but almost all of this assertions are used
to ensure internal logic and not depend on external reasons.
Also, most of explicitly thrown AssertionErrors in the persistance
implementation are rethrown assertions with additional information.
I found only couple of cases which can potentialy match your description:
{{PageMemoryImpl#tryToRestorePage:}}
{code:java}
throw new AssertionError(String.format(
"Page is broken. Can't restore it from WAL. (grpId = %d, pageId = %X).",
fullId.groupId(), fullId.pageId()
));
{code}
{{GridTreePrinter#getChildren}}:
{code:java}
catch (IgniteCheckedException ignored) {
throw new AssertionError("Can not acquire page.");
}
{code}
But this method used only for dumping debug information in the test framework.
{{FilePageStoreManager#beforeCacheGroupStart}}:
{code:java}
assert dir.exists();
{code}
{{FilePageStoreManager#storeCacheData}}:
{code:java}
assert cacheWorkDir.exists() : "Work directory does not exist: " + cacheWorkDir;
{code}
But this method creates dir (and checks result code) before assertion.
Did you mean these cases or something else? If something else, do you have
stacktraces or any description of real cases?
> Throw Exceptions instead of AssertionErrors when reading from corrupted
> persistence
> ---
>
> Key: IGNITE-7831
> URL: https://issues.apache.org/jira/browse/IGNITE-7831
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ilya Lantukh
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: iep-14
> Fix For: 2.5
>
>
> There are a few places in our code where we explicitly throw AssertionErrors
> due to inability to correctly read data from persistence and many more places
> where we make assertions based on read values.
> Assertions are used to indicate problems in internal logic, while persistence
> might also get corrupted by various external reasons. It also makes uniform
> handling of such issues considerably harder, because exception handling logic
> in Ignite ignores Errors. If we want to improve stability and minimize
> consequenses of pesistence corruption, we should replace all those
> AssertionErrors and asserts with Exceptions, so that current exception
> handling mechanisms could be reduce. In a number of situations it means that
> instead of causing cluster-wide hang-up problematic node will be
> automatically killed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7969) Fluky failure of IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
Aleksey Plekhanov created IGNITE-7969:
-
Summary: Fluky failure of
IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
Key: IGNITE-7969
URL: https://issues.apache.org/jira/browse/IGNITE-7969
Project: Ignite
Issue Type: Bug
Affects Versions: 2.4
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Test fails on TeamCity with exception:
{noformat}
java.lang.AssertionError: Successful tryPut after failure [gridIdx=2, sucessful
puts = 50]
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:491)
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidator0(IgniteTopologyValidatorGridSplitCacheTest.java:281)
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup(IgniteTopologyValidatorGridSplitCacheTest.java:234)
...
Caused by: class org.apache.ignite.IgniteException: Failed to put entry
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:498)
... 11 more
Suppressed: class org.apache.ignite.IgniteException: Failed to put
entry [node=cache.IgniteTopologyValidatorGridSplitCacheTest0]
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:463)
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:488)
... 11 more
Suppressed: junit.framework.AssertionFailedError: expected:<1>
but was:<2>
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at junit.framework.Assert.assertEquals(Assert.java:234)
at junit.framework.Assert.assertEquals(Assert.java:241)
at
junit.framework.TestCase.assertEquals(TestCase.java:409)
at
org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:448)
... 12 more
...
{noformat}
Sometimes reproduces locally.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7969) Flaky failure of IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
[
https://issues.apache.org/jira/browse/IGNITE-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-7969:
--
Summary: Flaky failure of
IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
(was: Fluky failure of
IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup)
> Flaky failure of
> IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
> --
>
> Key: IGNITE-7969
> URL: https://issues.apache.org/jira/browse/IGNITE-7969
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: MakeTeamcityGreenAgain
>
> Test fails on TeamCity with exception:
> {noformat}
> java.lang.AssertionError: Successful tryPut after failure [gridIdx=2,
> sucessful puts = 50]
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:491)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidator0(IgniteTopologyValidatorGridSplitCacheTest.java:281)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup(IgniteTopologyValidatorGridSplitCacheTest.java:234)
> ...
> Caused by: class org.apache.ignite.IgniteException: Failed to put entry
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:498)
> ... 11 more
> Suppressed: class org.apache.ignite.IgniteException: Failed to put
> entry [node=cache.IgniteTopologyValidatorGridSplitCacheTest0]
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:463)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:488)
> ... 11 more
> Suppressed: junit.framework.AssertionFailedError: expected:<1>
> but was:<2>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at
> junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:448)
> ... 12 more
> ...
> {noformat}
> Sometimes reproduces locally.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7976) [Test failed] IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes fails on TC
Aleksey Plekhanov created IGNITE-7976:
-
Summary: [Test failed]
IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes
fails on TC
Key: IGNITE-7976
URL: https://issues.apache.org/jira/browse/IGNITE-7976
Project: Ignite
Issue Type: Bug
Affects Versions: 2.4
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Test
{{IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes}}
always fail on TeamCity due to changes by IGNITE-7869. With following error:
{noformat}
javax.cache.CacheException: Failed to parse query. Table "PERSON" not found;
SQL statement:
SELECT p._KEY, p._VAL FROM Person p WHERE p.lname=? ORDER BY p.fname [42102-195]
...
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7976) [Test failed] IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes fails on TC
[
https://issues.apache.org/jira/browse/IGNITE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-7976:
--
Description:
Test
{{IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes}}
always fail on TeamCity due to changes by IGNITE-7869. With the following error:
{noformat}
javax.cache.CacheException: Failed to parse query. Table "PERSON" not found;
SQL statement:
SELECT p._KEY, p._VAL FROM Person p WHERE p.lname=? ORDER BY p.fname [42102-195]
...
{noformat}
was:
Test
{{IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes}}
always fail on TeamCity due to changes by IGNITE-7869. With following error:
{noformat}
javax.cache.CacheException: Failed to parse query. Table "PERSON" not found;
SQL statement:
SELECT p._KEY, p._VAL FROM Person p WHERE p.lname=? ORDER BY p.fname [42102-195]
...
{noformat}
> [Test failed]
> IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes
> fails on TC
>
>
> Key: IGNITE-7976
> URL: https://issues.apache.org/jira/browse/IGNITE-7976
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: MakeTeamcityGreenAgain
>
> Test
> {{IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes}}
> always fail on TeamCity due to changes by IGNITE-7869. With the following
> error:
> {noformat}
> javax.cache.CacheException: Failed to parse query. Table "PERSON" not found;
> SQL statement:
> SELECT p._KEY, p._VAL FROM Person p WHERE p.lname=? ORDER BY p.fname
> [42102-195]
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7976) [Test failed] IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes fails on TC
[
https://issues.apache.org/jira/browse/IGNITE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404731#comment-16404731
]
Aleksey Plekhanov commented on IGNITE-7976:
---
[~dpavlov] fix is ready. Some tests on TC are still not passed (due to another
flaky tests). I hope today I will move ticket to PA.
> [Test failed]
> IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes
> fails on TC
>
>
> Key: IGNITE-7976
> URL: https://issues.apache.org/jira/browse/IGNITE-7976
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: MakeTeamcityGreenAgain
>
> Test
> {{IgnitePersistentStoreCacheGroupsTest.testClusterRestartCachesWithH2Indexes}}
> always fail on TeamCity due to changes by IGNITE-7869. With the following
> error:
> {noformat}
> javax.cache.CacheException: Failed to parse query. Table "PERSON" not found;
> SQL statement:
> SELECT p._KEY, p._VAL FROM Person p WHERE p.lname=? ORDER BY p.fname
> [42102-195]
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7969) Flaky failure of IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
[
https://issues.apache.org/jira/browse/IGNITE-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407624#comment-16407624
]
Aleksey Plekhanov commented on IGNITE-7969:
---
[~dpavlov] IGNITE-5958 is not actual anymore. There was a race in the old
implementation which was fixed by IGNITE-6491, IGNITE-6519. IGNITE-5870 don't
contain any stack traces, but I think it's the same problem as IGNITE-5958 and
ticket can be closed as duplicate.
Fix for this ticket also affect testTopologyValidator, so after merge
testTopologyValidator can be unmuted.
> Flaky failure of
> IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup
> --
>
> Key: IGNITE-7969
> URL: https://issues.apache.org/jira/browse/IGNITE-7969
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: MakeTeamcityGreenAgain
>
> Test fails on TeamCity with exception:
> {noformat}
> java.lang.AssertionError: Successful tryPut after failure [gridIdx=2,
> sucessful puts = 50]
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:491)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidator0(IgniteTopologyValidatorGridSplitCacheTest.java:281)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.testTopologyValidatorWithCacheGroup(IgniteTopologyValidatorGridSplitCacheTest.java:234)
> ...
> Caused by: class org.apache.ignite.IgniteException: Failed to put entry
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:498)
> ... 11 more
> Suppressed: class org.apache.ignite.IgniteException: Failed to put
> entry [node=cache.IgniteTopologyValidatorGridSplitCacheTest0]
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:463)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:488)
> ... 11 more
> Suppressed: junit.framework.AssertionFailedError: expected:<1>
> but was:<2>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at
> junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.ignite.internal.processors.cache.IgniteTopologyValidatorGridSplitCacheTest.tryPut(IgniteTopologyValidatorGridSplitCacheTest.java:448)
> ... 12 more
> ...
> {noformat}
> Sometimes reproduces locally.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-5870) IgniteTopologyValidatorGridSplitCacheTest#testTopologyValidator fails
[ https://issues.apache.org/jira/browse/IGNITE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov resolved IGNITE-5870. --- Resolution: Duplicate Duplicates IGNITE-5958 > IgniteTopologyValidatorGridSplitCacheTest#testTopologyValidator fails > - > > Key: IGNITE-5870 > URL: https://issues.apache.org/jira/browse/IGNITE-5870 > Project: Ignite > Issue Type: Bug >Reporter: Eduard Shangareev >Priority: Major > Labels: MakeTeamcityGreenAgain > > Success rate is 10%. > History is too short to say when it starts failing. > http://ci.ignite.apache.org/project.html?tab=testDetails&projectId=Ignite20Tests&testNameId=-318150309218360964&page=1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8033) Flaky failure TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock on TC
Aleksey Plekhanov created IGNITE-8033:
-
Summary: Flaky failure
TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock on TC
Key: IGNITE-8033
URL: https://issues.apache.org/jira/browse/IGNITE-8033
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Test TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock flakily fail on
TC. Sometimes with timeout, sometimes with the following error:
{noformat}
[2018-03-21 12:06:23,469][ERROR][main][root] Test failed.
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:55)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.Assert.assertTrue(Assert.java:31)
at junit.framework.TestCase.assertTrue(TestCase.java:201)
at
org.apache.ignite.internal.processors.cache.transactions.TxOptimisticDeadlockDetectionCrossCacheTest.doTestDeadlock(TxOptimisticDeadlockDetectionCrossCacheTest.java:186)
at
org.apache.ignite.internal.processors.cache.transactions.TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock(TxOptimisticDeadlockDetectionCrossCacheTest.java:118)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at
org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2001)
at
org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:133)
at
org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1916)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7481) Suspended transaction rollbacs incorrectly
[ https://issues.apache.org/jira/browse/IGNITE-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417310#comment-16417310 ] Aleksey Plekhanov commented on IGNITE-7481: --- [~Alexey Kuznetsov] looks good to me. > Suspended transaction rollbacs incorrectly > > > Key: IGNITE-7481 > URL: https://issues.apache.org/jira/browse/IGNITE-7481 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Kuznetsov >Assignee: Alexey Kuznetsov >Priority: Major > Fix For: 2.5 > > > When we suspend transaction, and then timeout occures , transaction would be > rollbacked incorrectly. > One of incorrect behaviors : rollback\end tx metrics are not incremented. > Thre reason is that transactionMap is cleared when we suspend transaction. > We need not clear transactionMap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-8033) Flaky failure TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock on TC
[
https://issues.apache.org/jira/browse/IGNITE-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418779#comment-16418779
]
Aleksey Plekhanov edited comment on IGNITE-8033 at 3/29/18 11:24 AM:
-
Sometimes test on TC is still failing. Ticket moved to IP for further
investigation.
was (Author: alex_pl):
Sometimes test on TC are still failed. Ticket moved to IP for further
investigation.
> Flaky failure TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock on TC
>
>
> Key: IGNITE-8033
> URL: https://issues.apache.org/jira/browse/IGNITE-8033
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.5
>
>
> Test TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock flakily fail on
> TC. Sometimes with timeout, sometimes with the following error:
> {noformat}
> [2018-03-21 12:06:23,469][ERROR][main][root] Test failed.
>
>
> junit.framework.AssertionFailedError
>
>
> at junit.framework.Assert.fail(Assert.java:55)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.Assert.assertTrue(Assert.java:31)
> at junit.framework.TestCase.assertTrue(TestCase.java:201)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxOptimisticDeadlockDetectionCrossCacheTest.doTestDeadlock(TxOptimisticDeadlockDetectionCrossCacheTest.java:186)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxOptimisticDeadlockDetectionCrossCacheTest.testDeadlock(TxOptimisticDeadlockDetectionCrossCacheTest.java:118)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2001)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:133)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1916)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-8069) IgniteOutOfMemoryException should be handled accordingly to provided failure handler
[
https://issues.apache.org/jira/browse/IGNITE-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov reassigned IGNITE-8069:
-
Assignee: Aleksey Plekhanov
> IgniteOutOfMemoryException should be handled accordingly to provided failure
> handler
>
>
> Key: IGNITE-8069
> URL: https://issues.apache.org/jira/browse/IGNITE-8069
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: iep-14
> Fix For: 2.5
>
>
> {{IgniteOutOfMemoryException}} should be handled accordingly to provided
> failure handler.
> There are a couple of places where this exception can be thrown. Need to find
> a common place(s) where this exception could be handled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-6630) Incorrect time units of average transaction commit/rollback duration cache metrics.
[
https://issues.apache.org/jira/browse/IGNITE-6630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422002#comment-16422002
]
Aleksey Plekhanov commented on IGNITE-6630:
---
[~xtern], looks good to me.
> Incorrect time units of average transaction commit/rollback duration cache
> metrics.
> ---
>
> Key: IGNITE-6630
> URL: https://issues.apache.org/jira/browse/IGNITE-6630
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Minor
> Labels: metrics, newbie
> Fix For: 2.5
>
>
> AverageTxCommitTime and AverageTxRollbackTime metrics in CacheMetrics counts
> in milliseconds instead of microseconds as pointed in javadoc.
> Simple junit reproducer:
> {code:java}
> public class CacheMetricsTxAvgTimeTest extends GridCommonAbstractTest {
> /** */
> private CacheConfiguration cacheConfiguration(String name) {
> CacheConfiguration cacheConfiguration = new
> CacheConfiguration<>(name);
> cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> cacheConfiguration.setStatisticsEnabled(true);
> return cacheConfiguration;
> }
> /** */
> public void testTxCommitDuration() throws Exception {
> try ( Ignite node = startGrid(0)) {
> IgniteCache cache =
> node.createCache(cacheConfiguration(DEFAULT_CACHE_NAME));
> try (Transaction tx = node.transactions().txStart()) {
> cache.put(1, 1);
> // Await 1 second.
> U.sleep(1_000);
> tx.commit();
> }
> // Documentation says that this metric is in microseconds.
> float commitTime = cache.metrics().getAverageTxCommitTime();
> // But this assertion will fail because it in milliseconds and
> returns only ~1000.
> assert commitTime >= 1_000_000;
> }
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-20587) SQL hints documentation. INDEX, NO_INDEX
[ https://issues.apache.org/jira/browse/IGNITE-20587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20587: --- Component/s: documentation > SQL hints documentation. INDEX, NO_INDEX > > > Key: IGNITE-20587 > URL: https://issues.apache.org/jira/browse/IGNITE-20587 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Fix For: 2.16 > > Time Spent: 5h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20587) SQL hints documentation. INDEX, NO_INDEX
[ https://issues.apache.org/jira/browse/IGNITE-20587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20587: --- Ignite Flags: (was: Docs Required,Release Notes Required) > SQL hints documentation. INDEX, NO_INDEX > > > Key: IGNITE-20587 > URL: https://issues.apache.org/jira/browse/IGNITE-20587 > Project: Ignite > Issue Type: Task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Fix For: 2.16 > > Time Spent: 5h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20587) SQL hints documentation. INDEX, NO_INDEX
[ https://issues.apache.org/jira/browse/IGNITE-20587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20587: --- Description: Add documentation for SQL hints > SQL hints documentation. INDEX, NO_INDEX > > > Key: IGNITE-20587 > URL: https://issues.apache.org/jira/browse/IGNITE-20587 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Fix For: 2.16 > > Time Spent: 5h 50m > Remaining Estimate: 0h > > Add documentation for SQL hints -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20674) Move ignite-yardstick module to ignite-extensions
Aleksey Plekhanov created IGNITE-20674: -- Summary: Move ignite-yardstick module to ignite-extensions Key: IGNITE-20674 URL: https://issues.apache.org/jira/browse/IGNITE-20674 Project: Ignite Issue Type: Improvement Components: yardstick Reporter: Aleksey Plekhanov Currently, we include ignite-yardstick module into binary release, but: - It has dependencies that never used in other Ignite modules. Sometimes it requires dependencies versions update to avoid CVEs of dependencies. - It increases size of binary release by 100Mb. - It used only by release engeneers. - It has a lot of bugs in org.yardstickframework.yardstick artifact, which can't be fixed in our repository (for example, out of the box it can be run only on java8). We should exclude it from binary release and move it to the ignite-extensions repository. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20697) Move physical records from WAL to another storage
Aleksey Plekhanov created IGNITE-20697: -- Summary: Move physical records from WAL to another storage Key: IGNITE-20697 URL: https://issues.apache.org/jira/browse/IGNITE-20697 Project: Ignite Issue Type: Improvement Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Currentrly, physycal records take most of the WAL size. But physical records in WAL files required only for crush recovery and these records are useful only for a short period of time (since last checkpoint). Size of physical records during checkpoint is more than size of all modified pages between checkpoints, since we need to store page snapshot record for each modified page and page delta records, if page is modified more than once between checkpoints. We process WAL file several times in normal workflow (without crashes): 1) We write records to WAL files 2) We copy WAL files to archive 3) We compact WAL files (remove phisical records + compress) So, totally we write all physical records twice and read physical records twice. To reduce disc workload we can move physical records to another storage and don't write them to WAL files. To provide the same crush recovery guarantees we can write modified pages twice during checkpoint. First time to some delta file and second time to the page storage. In this case we can recover any page if we crash during write to page storage from delta file (instead of WAL, as we do now). This proposal has pros and cons. Pros: - Less size of stored data (we don't store page delta files, only final state of the page) - Reduced disc workload (we store additionally write once all modified pages instead of 2 writes and 2 reads of larger amount of data) - Potentially reduced latancy (instead of writing physical records synchronously during data modification we write to WAL only logical records and physical pages will be written by checkpointer threads) Cons: - Increased checkpoint duration (we should write doubled amount of data during checkpoint) Let's try it and benchmark. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20697) Move physical records from WAL to another storage
[ https://issues.apache.org/jira/browse/IGNITE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20697: --- Description: Currentrly, physycal records take most of the WAL size. But physical records in WAL files required only for crush recovery and these records are useful only for a short period of time (since last checkpoint). Size of physical records during checkpoint is more than size of all modified pages between checkpoints, since we need to store page snapshot record for each modified page and page delta records, if page is modified more than once between checkpoints. We process WAL file several times in stable workflow (without crashes and rebalances): # We write records to WAL files # We copy WAL files to archive # We compact WAL files (remove phisical records + compress) So, totally we write all physical records twice and read physical records at least twice. To reduce disc workload we can move physical records to another storage and don't write them to WAL files. To provide the same crush recovery guarantees we can write modified pages twice during checkpoint. First time to some delta file and second time to the page storage. In this case we can recover any page if we crash during write to page storage from delta file (instead of WAL, as we do now). This proposal has pros and cons. Pros: - Less size of stored data (we don't store page delta files, only final state of the page) - Reduced disc workload (we store additionally write once all modified pages instead of 2 writes and 2 reads of larger amount of data) - Potentially reduced latancy (instead of writing physical records synchronously during data modification we write to WAL only logical records and physical pages will be written by checkpointer threads) Cons: - Increased checkpoint duration (we should write doubled amount of data during checkpoint) Let's try to implement it and benchmark. was: Currentrly, physycal records take most of the WAL size. But physical records in WAL files required only for crush recovery and these records are useful only for a short period of time (since last checkpoint). Size of physical records during checkpoint is more than size of all modified pages between checkpoints, since we need to store page snapshot record for each modified page and page delta records, if page is modified more than once between checkpoints. We process WAL file several times in normal workflow (without crashes): 1) We write records to WAL files 2) We copy WAL files to archive 3) We compact WAL files (remove phisical records + compress) So, totally we write all physical records twice and read physical records twice. To reduce disc workload we can move physical records to another storage and don't write them to WAL files. To provide the same crush recovery guarantees we can write modified pages twice during checkpoint. First time to some delta file and second time to the page storage. In this case we can recover any page if we crash during write to page storage from delta file (instead of WAL, as we do now). This proposal has pros and cons. Pros: - Less size of stored data (we don't store page delta files, only final state of the page) - Reduced disc workload (we store additionally write once all modified pages instead of 2 writes and 2 reads of larger amount of data) - Potentially reduced latancy (instead of writing physical records synchronously during data modification we write to WAL only logical records and physical pages will be written by checkpointer threads) Cons: - Increased checkpoint duration (we should write doubled amount of data during checkpoint) Let's try it and benchmark. > Move physical records from WAL to another storage > -- > > Key: IGNITE-20697 > URL: https://issues.apache.org/jira/browse/IGNITE-20697 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > > Currentrly, physycal records take most of the WAL size. But physical records > in WAL files required only for crush recovery and these records are useful > only for a short period of time (since last checkpoint). > Size of physical records during checkpoint is more than size of all modified > pages between checkpoints, since we need to store page snapshot record for > each modified page and page delta records, if page is modified more than once > between checkpoints. > We process WAL file several times in stable workflow (without crashes and > rebalances): > # We write records to WAL files > # We copy WAL files to archive > # We compact WAL files (remove phisical records + compress) > So, totally we write all physical records twice and read physical records at > least twice. > To reduce disc work
[jira] [Commented] (IGNITE-20697) Move physical records from WAL to another storage
[
https://issues.apache.org/jira/browse/IGNITE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1674#comment-1674
]
Aleksey Plekhanov commented on IGNITE-20697:
[~ktkale...@gridgain.com], sure, I have plans to create IEP and write to the
dev list, but first I want to create POC.
{quote}It also turns out that if users do not gracefully shut down the cluster
before switching to a new version of the ignite, they may experience problems
starting nodes since there will be a new data recovery mechanism.
{quote}
I suppose we will provide both mechanisms and allow user to configure it. In
the next release we can use physical records by defaul, in the following
release we can swith default to checkpoint delta file. On recovery Ignite can
decide what to do by analyzing files for corrent checkpoint.
> Move physical records from WAL to another storage
> --
>
> Key: IGNITE-20697
> URL: https://issues.apache.org/jira/browse/IGNITE-20697
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
>
> Currentrly, physycal records take most of the WAL size. But physical records
> in WAL files required only for crush recovery and these records are useful
> only for a short period of time (since last checkpoint).
> Size of physical records during checkpoint is more than size of all modified
> pages between checkpoints, since we need to store page snapshot record for
> each modified page and page delta records, if page is modified more than once
> between checkpoints.
> We process WAL file several times in stable workflow (without crashes and
> rebalances):
> # We write records to WAL files
> # We copy WAL files to archive
> # We compact WAL files (remove phisical records + compress)
> So, totally we write all physical records twice and read physical records at
> least twice.
> To reduce disc workload we can move physical records to another storage and
> don't write them to WAL files. To provide the same crush recovery guarantees
> we can write modified pages twice during checkpoint. First time to some delta
> file and second time to the page storage. In this case we can recover any
> page if we crash during write to page storage from delta file (instead of
> WAL, as we do now).
> This proposal has pros and cons.
> Pros:
> - Less size of stored data (we don't store page delta files, only final
> state of the page)
> - Reduced disc workload (we store additionally write once all modified pages
> instead of 2 writes and 2 reads of larger amount of data)
> - Potentially reduced latancy (instead of writing physical records
> synchronously during data modification we write to WAL only logical records
> and physical pages will be written by checkpointer threads)
> Cons:
> - Increased checkpoint duration (we should write doubled amount of data
> during checkpoint)
> Let's try to implement it and benchmark.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-20697) Move physical records from WAL to another storage
[
https://issues.apache.org/jira/browse/IGNITE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1674#comment-1674
]
Aleksey Plekhanov edited comment on IGNITE-20697 at 10/20/23 9:48 AM:
--
[~ktkale...@gridgain.com], sure, I have plans to create IEP and write to the
dev list, but first I want to create POC.
{quote}It also turns out that if users do not gracefully shut down the cluster
before switching to a new version of the ignite, they may experience problems
starting nodes since there will be a new data recovery mechanism.
{quote}
I suppose we will provide both mechanisms and allow user to configure it. In
the next release we can use physical records by defaul, in the following
release we can swith default to checkpoint delta file. On recovery Ignite can
decide what to do by analyzing files for current checkpoint.
was (Author: alex_pl):
[~ktkale...@gridgain.com], sure, I have plans to create IEP and write to the
dev list, but first I want to create POC.
{quote}It also turns out that if users do not gracefully shut down the cluster
before switching to a new version of the ignite, they may experience problems
starting nodes since there will be a new data recovery mechanism.
{quote}
I suppose we will provide both mechanisms and allow user to configure it. In
the next release we can use physical records by defaul, in the following
release we can swith default to checkpoint delta file. On recovery Ignite can
decide what to do by analyzing files for corrent checkpoint.
> Move physical records from WAL to another storage
> --
>
> Key: IGNITE-20697
> URL: https://issues.apache.org/jira/browse/IGNITE-20697
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
>
> Currentrly, physycal records take most of the WAL size. But physical records
> in WAL files required only for crush recovery and these records are useful
> only for a short period of time (since last checkpoint).
> Size of physical records during checkpoint is more than size of all modified
> pages between checkpoints, since we need to store page snapshot record for
> each modified page and page delta records, if page is modified more than once
> between checkpoints.
> We process WAL file several times in stable workflow (without crashes and
> rebalances):
> # We write records to WAL files
> # We copy WAL files to archive
> # We compact WAL files (remove phisical records + compress)
> So, totally we write all physical records twice and read physical records at
> least twice.
> To reduce disc workload we can move physical records to another storage and
> don't write them to WAL files. To provide the same crush recovery guarantees
> we can write modified pages twice during checkpoint. First time to some delta
> file and second time to the page storage. In this case we can recover any
> page if we crash during write to page storage from delta file (instead of
> WAL, as we do now).
> This proposal has pros and cons.
> Pros:
> - Less size of stored data (we don't store page delta files, only final
> state of the page)
> - Reduced disc workload (we store additionally write once all modified pages
> instead of 2 writes and 2 reads of larger amount of data)
> - Potentially reduced latancy (instead of writing physical records
> synchronously during data modification we write to WAL only logical records
> and physical pages will be written by checkpointer threads)
> Cons:
> - Increased checkpoint duration (we should write doubled amount of data
> during checkpoint)
> Let's try to implement it and benchmark.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-20501) Calcite engine. Memory leak in MailboxRegistryImpl#remotes on JOINs
[
https://issues.apache.org/jira/browse/IGNITE-20501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov resolved IGNITE-20501.
Fix Version/s: 2.16
Release Note: SQL Calcite: Fixed memory leak in MailboxRegistryImpl#remotes
Resolution: Fixed
[~ivandasch], thanks for the review! Merged to master.
> Calcite engine. Memory leak in MailboxRegistryImpl#remotes on JOINs
> ---
>
> Key: IGNITE-20501
> URL: https://issues.apache.org/jira/browse/IGNITE-20501
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, ise
> Fix For: 2.16
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When JOIN relational operator is executed, downstream of JOIN can be closed
> if only one side of JOIN is already drained (see last lines of
> {{MergeJoinNode.InnerJoin#join}}, for example). In this case query can be
> prematurely closed, and after this, message for another side of join can
> arrive and register new {{Inbox}} (see {{ExchangeServiceImpl#onMessage(UUID,
> QueryBatchMessage)}}), that never will be unregistered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20697) Move physical records from WAL to another storage
[ https://issues.apache.org/jira/browse/IGNITE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20697: --- Labels: ise (was: ) > Move physical records from WAL to another storage > -- > > Key: IGNITE-20697 > URL: https://issues.apache.org/jira/browse/IGNITE-20697 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: ise > > Currentrly, physycal records take most of the WAL size. But physical records > in WAL files required only for crash recovery and these records are useful > only for a short period of time (since last checkpoint). > Size of physical records during checkpoint is more than size of all modified > pages between checkpoints, since we need to store page snapshot record for > each modified page and page delta records, if page is modified more than once > between checkpoints. > We process WAL file several times in stable workflow (without crashes and > rebalances): > # We write records to WAL files > # We copy WAL files to archive > # We compact WAL files (remove phisical records + compress) > So, totally we write all physical records twice and read physical records at > least twice. > To reduce disc workload we can move physical records to another storage and > don't write them to WAL files. To provide the same crash recovery guarantees > we can write modified pages twice during checkpoint. First time to some delta > file and second time to the page storage. In this case we can recover any > page if we crash during write to page storage from delta file (instead of > WAL, as we do now). > This proposal has pros and cons. > Pros: > - Less size of stored data (we don't store page delta files, only final > state of the page) > - Reduced disc workload (we store additionally write once all modified pages > instead of 2 writes and 2 reads of larger amount of data) > - Potentially reduced latency (instead of writing physical records > synchronously during data modification we write to WAL only logical records > and physical pages will be written by checkpointer threads) > Cons: > - Increased checkpoint duration (we should write doubled amount of data > during checkpoint) > Let's try to implement it and benchmark. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20697) Move physical records from WAL to another storage
[ https://issues.apache.org/jira/browse/IGNITE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20697: --- Labels: iep-113 ise (was: ise) > Move physical records from WAL to another storage > -- > > Key: IGNITE-20697 > URL: https://issues.apache.org/jira/browse/IGNITE-20697 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: iep-113, ise > Time Spent: 10m > Remaining Estimate: 0h > > Currentrly, physycal records take most of the WAL size. But physical records > in WAL files required only for crash recovery and these records are useful > only for a short period of time (since last checkpoint). > Size of physical records during checkpoint is more than size of all modified > pages between checkpoints, since we need to store page snapshot record for > each modified page and page delta records, if page is modified more than once > between checkpoints. > We process WAL file several times in stable workflow (without crashes and > rebalances): > # We write records to WAL files > # We copy WAL files to archive > # We compact WAL files (remove phisical records + compress) > So, totally we write all physical records twice and read physical records at > least twice. > To reduce disc workload we can move physical records to another storage and > don't write them to WAL files. To provide the same crash recovery guarantees > we can write modified pages twice during checkpoint. First time to some delta > file and second time to the page storage. In this case we can recover any > page if we crash during write to page storage from delta file (instead of > WAL, as we do now). > This proposal has pros and cons. > Pros: > - Less size of stored data (we don't store page delta files, only final > state of the page) > - Reduced disc workload (we store additionally write once all modified pages > instead of 2 writes and 2 reads of larger amount of data) > - Potentially reduced latency (instead of writing physical records > synchronously during data modification we write to WAL only logical records > and physical pages will be written by checkpointer threads) > Cons: > - Increased checkpoint duration (we should write doubled amount of data > during checkpoint) > Let's try to implement it and benchmark. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20697) Move physical records from WAL to another storage
[ https://issues.apache.org/jira/browse/IGNITE-20697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-20697: --- Description: Currently, physycal records take most of the WAL size. But physical records in WAL files required only for crash recovery and these records are useful only for a short period of time (since last checkpoint). Size of physical records during checkpoint is more than size of all modified pages between checkpoints, since we need to store page snapshot record for each modified page and page delta records, if page is modified more than once between checkpoints. We process WAL file several times in stable workflow (without crashes and rebalances): # We write records to WAL files # We copy WAL files to archive # We compact WAL files (remove phisical records + compress) So, totally we write all physical records twice and read physical records at least twice. To reduce disc workload we can move physical records to another storage and don't write them to WAL files. To provide the same crash recovery guarantees we can write modified pages twice during checkpoint. First time to some delta file and second time to the page storage. In this case we can recover any page if we crash during write to page storage from delta file (instead of WAL, as we do now). This proposal has pros and cons. Pros: - Less size of stored data (we don't store page delta files, only final state of the page) - Reduced disc workload (we write all modified pages once instead of 2 writes and 2 reads of larger amount of data) - Potentially reduced latency (instead of writing physical records synchronously during data modification we write to WAL only logical records and physical pages will be written by checkpointer threads) Cons: - Increased checkpoint duration (we should write doubled amount of data during checkpoint) Let's try to implement it and benchmark. was: Currentrly, physycal records take most of the WAL size. But physical records in WAL files required only for crash recovery and these records are useful only for a short period of time (since last checkpoint). Size of physical records during checkpoint is more than size of all modified pages between checkpoints, since we need to store page snapshot record for each modified page and page delta records, if page is modified more than once between checkpoints. We process WAL file several times in stable workflow (without crashes and rebalances): # We write records to WAL files # We copy WAL files to archive # We compact WAL files (remove phisical records + compress) So, totally we write all physical records twice and read physical records at least twice. To reduce disc workload we can move physical records to another storage and don't write them to WAL files. To provide the same crash recovery guarantees we can write modified pages twice during checkpoint. First time to some delta file and second time to the page storage. In this case we can recover any page if we crash during write to page storage from delta file (instead of WAL, as we do now). This proposal has pros and cons. Pros: - Less size of stored data (we don't store page delta files, only final state of the page) - Reduced disc workload (we store additionally write once all modified pages instead of 2 writes and 2 reads of larger amount of data) - Potentially reduced latency (instead of writing physical records synchronously during data modification we write to WAL only logical records and physical pages will be written by checkpointer threads) Cons: - Increased checkpoint duration (we should write doubled amount of data during checkpoint) Let's try to implement it and benchmark. > Move physical records from WAL to another storage > -- > > Key: IGNITE-20697 > URL: https://issues.apache.org/jira/browse/IGNITE-20697 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: iep-113, ise > Time Spent: 10m > Remaining Estimate: 0h > > Currently, physycal records take most of the WAL size. But physical records > in WAL files required only for crash recovery and these records are useful > only for a short period of time (since last checkpoint). > Size of physical records during checkpoint is more than size of all modified > pages between checkpoints, since we need to store page snapshot record for > each modified page and page delta records, if page is modified more than once > between checkpoints. > We process WAL file several times in stable workflow (without crashes and > rebalances): > # We write records to WAL files > # We copy WAL files to archive > # We compact WAL files (remove phisical records + compress) >
[jira] [Created] (IGNITE-20950) Calcite engine. NPE when performance statistics is enabled after query already cached
Aleksey Plekhanov created IGNITE-20950:
--
Summary: Calcite engine. NPE when performance statistics is
enabled after query already cached
Key: IGNITE-20950
URL: https://issues.apache.org/jira/browse/IGNITE-20950
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Reproducer:
{code:java}
public void testPerformanceStatisticsEnableAfterQuery() throws Exception {
cleanPerformanceStatisticsDir();
String qry = "SELECT * FROM table(system_range(1, 1000))";
sql(grid(0), qry);
startCollectStatistics();
sql(grid(0), qry);
} {code}
Throws an exception:
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.performancestatistics.FilePerformanceStatisticsWriter.cacheIfPossible(FilePerformanceStatisticsWriter.java:540)
at
org.apache.ignite.internal.processors.performancestatistics.FilePerformanceStatisticsWriter.queryProperty(FilePerformanceStatisticsWriter.java:320)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.lambda$queryProperty$11(PerformanceStatisticsProcessor.java:207)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.write(PerformanceStatisticsProcessor.java:428)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.queryProperty(PerformanceStatisticsProcessor.java:207)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.mapAndExecutePlan(ExecutionServiceImpl.java:668)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executePlan(ExecutionServiceImpl.java:505)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.lambda$parseAndProcessQuery$4(CalciteQueryProcessor.java:495)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.processQuery(CalciteQueryProcessor.java:616)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.parseAndProcessQuery(CalciteQueryProcessor.java:495)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:389){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20950) Calcite engine. NPE when performance statistics is enabled after query already cached
[
https://issues.apache.org/jira/browse/IGNITE-20950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-20950:
---
Description:
Reproducer:
{code:java}
public void testPerformanceStatisticsEnableAfterQuery() throws Exception {
cleanPerformanceStatisticsDir();
String qry = "SELECT * FROM table(system_range(1, 1000))";
sql(grid(0), qry);
startCollectStatistics();
sql(grid(0), qry);
} {code}
Throws an exception:
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.performancestatistics.FilePerformanceStatisticsWriter.cacheIfPossible(FilePerformanceStatisticsWriter.java:540)
at
org.apache.ignite.internal.processors.performancestatistics.FilePerformanceStatisticsWriter.queryProperty(FilePerformanceStatisticsWriter.java:320)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.lambda$queryProperty$11(PerformanceStatisticsProcessor.java:207)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.write(PerformanceStatisticsProcessor.java:428)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.queryProperty(PerformanceStatisticsProcessor.java:207)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.mapAndExecutePlan(ExecutionServiceImpl.java:668)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executePlan(ExecutionServiceImpl.java:505)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.lambda$parseAndProcessQuery$4(CalciteQueryProcessor.java:495)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.processQuery(CalciteQueryProcessor.java:616)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.parseAndProcessQuery(CalciteQueryProcessor.java:495)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:389){noformat}
was:
Reproducer:
{code:java}
public void testPerformanceStatisticsEnableAfterQuery() throws Exception {
cleanPerformanceStatisticsDir();
String qry = "SELECT * FROM table(system_range(1, 1000))";
sql(grid(0), qry);
startCollectStatistics();
sql(grid(0), qry);
} {code}
Throws an exception:
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.performancestatistics.FilePerformanceStatisticsWriter.cacheIfPossible(FilePerformanceStatisticsWriter.java:540)
at
org.apache.ignite.internal.processors.performancestatistics.FilePerformanceStatisticsWriter.queryProperty(FilePerformanceStatisticsWriter.java:320)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.lambda$queryProperty$11(PerformanceStatisticsProcessor.java:207)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.write(PerformanceStatisticsProcessor.java:428)
at
org.apache.ignite.internal.processors.performancestatistics.PerformanceStatisticsProcessor.queryProperty(PerformanceStatisticsProcessor.java:207)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.mapAndExecutePlan(ExecutionServiceImpl.java:668)
at
org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executePlan(ExecutionServiceImpl.java:505)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.lambda$parseAndProcessQuery$4(CalciteQueryProcessor.java:495)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.processQuery(CalciteQueryProcessor.java:616)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.parseAndProcessQuery(CalciteQueryProcessor.java:495)
at
org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:389){noformat}
> Calcite engine. NPE when performance statistics is enabled after query
> already cached
> -
>
> Key: IGNITE-20950
> URL: https://issues.apache.org/jira/browse/IGNITE-20950
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, ise
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Reproducer:
> {code:java}
> public void testPerformanceStatisticsEnableAfterQuery() throws Exception {
> cleanPerformanceStatisticsDir();
> String qry = "SELECT * FROM table(system_range(1, 1000))";
> sql(grid(0), qry);
> startCollectStatistics();
> sql(grid(0), qry);
> } {code}
> Throws an exception:
> {noformat}
> java.lang.N
[jira] [Created] (IGNITE-21031) Calcite engine. Query fails on performance statistics in case of nested scans
Aleksey Plekhanov created IGNITE-21031:
--
Summary: Calcite engine. Query fails on performance statistics in
case of nested scans
Key: IGNITE-21031
URL: https://issues.apache.org/jira/browse/IGNITE-21031
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Nested scan can be performed by Calcite engine, for example, in case of UNION
ALL, when the first table scan is completed (and {{{}downstream().end(){}}})
method is invoked and UNION ALL operator proceed to the next table scan.
Reproducer:
{code:java}
public void testPerformanceStatisticsNestedScan() throws Exception {
sql(grid(0), "CREATE TABLE test_perf_stat_nested (a INT) WITH
template=REPLICATED");
sql(grid(0), "INSERT INTO test_perf_stat_nested VALUES (0), (1), (2), (3),
(4)");
startCollectStatistics();
sql(grid(0), "SELECT * FROM test_perf_stat_nested UNION ALL SELECT * FROM
test_perf_stat_nested");
}{code}
Fails on:
{noformat}
at
org.apache.ignite.internal.metric.IoStatisticsQueryHelper.startGatheringQueryStatistics(IoStatisticsQueryHelper.java:35)
at
org.apache.ignite.internal.processors.query.calcite.exec.tracker.PerformanceStatisticsIoTracker.startTracking(PerformanceStatisticsIoTracker.java:65)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:68)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.end(UnionAllNode.java:79)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.processNextBatch(ScanNode.java:185)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:70)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
at
org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.request(UnionAllNode.java:56)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21078) .NET: Platform cache is not updated on topology change when cache id is negative
Aleksey Plekhanov created IGNITE-21078:
--
Summary: .NET: Platform cache is not updated on topology change
when cache id is negative
Key: IGNITE-21078
URL: https://issues.apache.org/jira/browse/IGNITE-21078
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Reproducer:
{code:java}
public void TestPlatformCacheWithNegativeId()
{
InitNodes(1);
var cacheName = "negative_cache_id";
var cacheConfiguration = new CacheConfiguration(cacheName)
{
PlatformCacheConfiguration = new PlatformCacheConfiguration()
};
var cache = _ignite[0].GetOrCreateCache(cacheConfiguration);
var key = 0;
var val = new Foo(-1);
cache[key] = val;
InitNode(1);
Assert.AreEqual(val, cache[key]);
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21078) .NET: Platform cache is not updated on topology change when cache id is negative
[
https://issues.apache.org/jira/browse/IGNITE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-21078:
---
Labels: ise (was: )
> .NET: Platform cache is not updated on topology change when cache id is
> negative
>
>
> Key: IGNITE-21078
> URL: https://issues.apache.org/jira/browse/IGNITE-21078
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
>
> Reproducer:
> {code:java}
> public void TestPlatformCacheWithNegativeId()
> {
> InitNodes(1);
> var cacheName = "negative_cache_id";
> var cacheConfiguration = new CacheConfiguration(cacheName)
> {
> PlatformCacheConfiguration = new PlatformCacheConfiguration()
> };
>
> var cache = _ignite[0].GetOrCreateCache(cacheConfiguration);
> var key = 0;
> var val = new Foo(-1);
> cache[key] = val;
> InitNode(1);
> Assert.AreEqual(val, cache[key]);
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21078) .NET: Platform cache is not updated on topology change when cache id is negative
[
https://issues.apache.org/jira/browse/IGNITE-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-21078:
---
Description:
Reproducer:
{code:java}
public void TestPlatformCacheWithNegativeId()
{
InitNodes(1);
var cacheName = "negative_cache_id";
var cacheConfiguration = new CacheConfiguration(cacheName)
{
PlatformCacheConfiguration = new PlatformCacheConfiguration()
};
var cache = _ignite[0].GetOrCreateCache(cacheConfiguration);
var key = 0;
var val = new Foo(-1);
cache[key] = val;
InitNode(1);
Assert.AreEqual(val, cache[key]);
} {code}
Fails with:
{noformat}
Apache.Ignite.Core.Common.IgniteException : Java exception occurred
[class=java.lang.AssertionError, message=Affinity partition is out of range
[part=-1, partitions=1024]]
> Apache.Ignite.Core.Common.JavaException : java.lang.AssertionError:
Affinity partition is out of range [part=-1, partitions=1024]
at
org.apache.ignite.internal.processors.affinity.HistoryAffinityAssignmentImpl.get(HistoryAffinityAssignmentImpl.java:244)
at
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.primaryChanged(GridAffinityAssignmentCache.java:865)
at
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryChanged(GridCacheAffinityManager.java:403)
at
org.apache.ignite.internal.processors.platform.cache.affinity.PlatformAffinityManager.processInStreamOutLong(PlatformAffinityManager.java:62)
at
org.apache.ignite.internal.processors.platform.PlatformAbstractTarget.processInStreamOutLong(PlatformAbstractTarget.java:87)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67){noformat}
was:
Reproducer:
{code:java}
public void TestPlatformCacheWithNegativeId()
{
InitNodes(1);
var cacheName = "negative_cache_id";
var cacheConfiguration = new CacheConfiguration(cacheName)
{
PlatformCacheConfiguration = new PlatformCacheConfiguration()
};
var cache = _ignite[0].GetOrCreateCache(cacheConfiguration);
var key = 0;
var val = new Foo(-1);
cache[key] = val;
InitNode(1);
Assert.AreEqual(val, cache[key]);
} {code}
> .NET: Platform cache is not updated on topology change when cache id is
> negative
>
>
> Key: IGNITE-21078
> URL: https://issues.apache.org/jira/browse/IGNITE-21078
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
>
> Reproducer:
> {code:java}
> public void TestPlatformCacheWithNegativeId()
> {
> InitNodes(1);
> var cacheName = "negative_cache_id";
> var cacheConfiguration = new CacheConfiguration(cacheName)
> {
> PlatformCacheConfiguration = new PlatformCacheConfiguration()
> };
>
> var cache = _ignite[0].GetOrCreateCache(cacheConfiguration);
> var key = 0;
> var val = new Foo(-1);
> cache[key] = val;
> InitNode(1);
> Assert.AreEqual(val, cache[key]);
> } {code}
> Fails with:
> {noformat}
> Apache.Ignite.Core.Common.IgniteException : Java exception occurred
> [class=java.lang.AssertionError, message=Affinity partition is out of range
> [part=-1, partitions=1024]]
> > Apache.Ignite.Core.Common.JavaException : java.lang.AssertionError:
> Affinity partition is out of range [part=-1, partitions=1024]
> at
> org.apache.ignite.internal.processors.affinity.HistoryAffinityAssignmentImpl.get(HistoryAffinityAssignmentImpl.java:244)
> at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.primaryChanged(GridAffinityAssignmentCache.java:865)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryChanged(GridCacheAffinityManager.java:403)
> at
> org.apache.ignite.internal.processors.platform.cache.affinity.PlatformAffinityManager.processInStreamOutLong(PlatformAffinityManager.java:62)
> at
> org.apache.ignite.internal.processors.platform.PlatformAbstractTarget.processInStreamOutLong(PlatformAbstractTarget.java:87)
> at
> org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21082) Ignite Extensions: Exessive memory usage by performance statistics QueryHandler
Aleksey Plekhanov created IGNITE-21082: -- Summary: Ignite Extensions: Exessive memory usage by performance statistics QueryHandler Key: IGNITE-21082 URL: https://issues.apache.org/jira/browse/IGNITE-21082 Project: Ignite Issue Type: Bug Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov When processing queryProperty or queryRows events new strings are generated and written to the maps (as keys or values). Most of strings are not unique and already contained in other maps as keys or values, but as different instance. GC can't collect doublicated strings and this leads to OOM in some cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21082) Ignite Extensions: Excessive memory usage by performance statistics QueryHandler
[ https://issues.apache.org/jira/browse/IGNITE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-21082: --- Summary: Ignite Extensions: Excessive memory usage by performance statistics QueryHandler (was: Ignite Extensions: Exessive memory usage by performance statistics QueryHandler) > Ignite Extensions: Excessive memory usage by performance statistics > QueryHandler > > > Key: IGNITE-21082 > URL: https://issues.apache.org/jira/browse/IGNITE-21082 > Project: Ignite > Issue Type: Bug >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: ise > > When processing queryProperty or queryRows events new strings are generated > and written to the maps (as keys or values). Most of strings are not unique > and already contained in other maps as keys or values, but as different > instance. GC can't collect doublicated strings and this leads to OOM in some > cases. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21131) Calcite engine. OR operator with dynamic parameters can't be used for index scans
Aleksey Plekhanov created IGNITE-21131:
--
Summary: Calcite engine. OR operator with dynamic parameters can't
be used for index scans
Key: IGNITE-21131
URL: https://issues.apache.org/jira/browse/IGNITE-21131
Project: Ignite
Issue Type: Improvement
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
Calcite can compose OR operator with literals to SEARCH/SARG, and SEARCH/SARG
can be used for index scans. But we can't do this for OR operator with dynamic
parameters.
For example expression {{a IN (1, 2, 3)}} can be converted to {{a = 1 OR a = 2
OR a = 3}} and after that can be converted to {{SEARCH(a, SARG(1, 2, 3))}}, but
expression {{a IN (?, ?, ?)}} can be converted only to {{a = ? OR a = ? OR a =
?}} and can't be used for index scan.
To fix this issue we can create ranges from dynamic parameters during planning,
sort and remove intersections from these ranges in runtime.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
