[jira] [Created] (IGNITE-11829) Distribute joins fail if number of tables > 7
Stanislav Lukyanov created IGNITE-11829:
---
Summary: Distribute joins fail if number of tables > 7
Key: IGNITE-11829
URL: https://issues.apache.org/jira/browse/IGNITE-11829
Project: Ignite
Issue Type: Improvement
Components: sql
Affects Versions: 2.7
Reporter: Stanislav Lukyanov
Distributed joins fail with ArrayIndexOutOfBounds when the total number of
tables is > 7.
Example:
{code}
try (Ignite ignite =
Ignition.start("examples/config/example-ignite.xml");) {
IgniteCache cache = ignite.createCache("foo");
cache.query(new SqlFieldsQuery("CREATE TABLE Person(ID INTEGER
PRIMARY KEY, NAME VARCHAR(100));"));
cache.query(new SqlFieldsQuery("INSERT INTO Person(ID, NAME) VALUES
(1, 'Ed'), (2, 'Ann'), (3, 'Emma');"));
cache.query(new SqlFieldsQuery("SELECT *\n" +
"FROM PERSON P1\n" +
"JOIN PERSON P2 ON P1.ID = P2.ID\n" +
"JOIN PERSON P3 ON P1.ID = P3.ID\n" +
"JOIN PERSON P4 ON P1.ID = P4.ID\n" +
"JOIN PERSON P5 ON P1.ID = P5.ID\n" +
"JOIN PERSON P6 ON P1.ID = P6.ID\n" +
"JOIN PERSON P7 ON P1.ID = P7.ID\n" +
"JOIN PERSON P8 ON P1.ID =
P8.ID").setDistributedJoins(true).setEnforceJoinOrder(false));
}
{code}
throws
{code}
Exception in thread "main" javax.cache.CacheException: General error:
"java.lang.ArrayIndexOutOfBoundsException" [5-197]
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:832)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:765)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:403)
at
org.apache.ignite.examples.ExampleNodeStartup.main(ExampleNodeStartup.java:60)
Caused by: class
org.apache.ignite.internal.processors.query.IgniteSQLException: General error:
"java.lang.ArrayIndexOutOfBoundsException" [5-197]
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:454)
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parse0(QueryParser.java:156)
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parse(QueryParser.java:121)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1191)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2261)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2257)
at
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:53)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2767)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2277)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2297)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2250)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2177)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
... 3 more
Caused by: org.h2.jdbc.JdbcSQLException: General error:
"java.lang.ArrayIndexOutOfBoundsException" [5-197]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
at org.h2.message.DbException.get(DbException.java:168)
at org.h2.message.DbException.convert(DbException.java:307)
at org.h2.message.DbException.toSQLException(DbException.java:280)
at org.h2.message.TraceObject.logAndConvert(TraceObject.java:357)
at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:308)
at
org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.prepare(GridSqlQuerySplitter.java:1770)
at
org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split0(GridSqlQuerySplitter.java:299)
at
org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split(GridSqlQuerySplitter.java:219)
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:415)
... 15 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at
org.apache.ignite.internal.processors.query.h2.opt.join.CollocationModel.childFilters(CollocationModel.java:240)
at
[jira] [Created] (IGNITE-11828) Ignite Queue "operationContextPerCall" is set only during the moment of creation
Uday Kale created IGNITE-11828: -- Summary: Ignite Queue "operationContextPerCall" is set only during the moment of creation Key: IGNITE-11828 URL: https://issues.apache.org/jira/browse/IGNITE-11828 Project: Ignite Issue Type: New Feature Reporter: Uday Kale Assignee: Uday Kale -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: IgniteConfigVariationsAbstractTest subclasses do not run
Ivan F., I think that it is better to enable IgniteConfigVariationsAbstractTest subclasses first, so no new broken tests will appear. After that we can fix IgniteConfigVariationsAbstractTest subclasses one by one in separate ticket(s). вт, 30 апр. 2019 г. в 13:23, Ivan Fedotov : > > Ivan R., Ivan P., thank you for the suggestions, I took a look at them. > > According to checking CacheAtomicityMode on null in > IgniteCacheConfigVariationsAbstractTest#atomicityMode - are you sure > that checking should proceed on test level? May be it will be better to > make default cache value in case if cc.getAtomicityMode() == null > in CacheConfiguration constructor [1]? > > Also let me suggest one minor change according cache specification in > IgniteCacheReadThroughEvictionSelfTest. The same logic is used in > GridCacheAbstractSelfTest#cacheConfiguration [2]. > I think we can excract this code block in a separate static methods > (initCacheConfig, for example) in IgniteCacheReadThroughEvictionSelfTest and > invoke it in IgniteCacheReadThroughEvictionSelfTest#variationConfig and > GridCacheAbstractSelfTest#cacheConfiguration methods. > > In addition, I want to draw your attention to > IgniteContinuousQueryConfigVariationsSuite test runs [3]. > CacheContinuousQueryVariationsTest are generated dynamically. > The first batch (12 tests) works fine, but on the next runs we got NPE in > IgniteCacheConfigVariationsAbstractTest#afterTest - default cache does not > exist and we can not > invoke unwrap() on this [4]. > Seems that the problem is in > IgniteCacheConfigVariationsAbstractTest#beforeTestsStarted/afterTestsStopped > methods, cache is not properly created (or already existed cache is > destroyed). > > By the way, should I resolve these issues in the context of IGNITE-11708 or > create a separate ticket(s)? > > [1] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CacheConfiguration.java#L434 > [2] > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheAbstractSelfTest.java#L235 > [3] > https://ci.ignite.apache.org/viewLog.html?buildId=3701865=IgniteTests24Java8_RunAllNightly > [4] > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteCacheConfigVariationsAbstractTest.java#L212 > > пт, 26 апр. 2019 г. в 18:01, Ivan Rakov : > > > Ivan P., > > > > Good catch, thanks. > > > > I was wrong, test scenario is correct. The problem was in > > atomicityMode() method - it could have returned null (which was okay for > > config generation, but wasn't expected in the test code). > > Please take a look at tx_out_test_fixed.patch (attached to > > IGNITE-11708). To sum it up, both issues should be fixed now. > > > > Best Regards, > > Ivan Rakov > > > > On 26.04.2019 14:40, Павлухин Иван wrote: > > > Ivan R., > > > > > > As I can see IgniteCacheConfigVariationsFullApiTest#testGetOutTx does > > > not expect lock/unlock events due to line: > > > if (atomicityMode() == ATOMIC) > > > return lockEvtCnt.get() == 0; > > > > > > Could you please elaborate? > > > > > > пт, 26 апр. 2019 г. в 13:32, Ivan Rakov : > > >> Ivan, > > >> > > >> Seems like IgniteCacheReadThroughEvictionSelfTest is broken. Test > > >> scenario assumes that even after expiration entry will be present in > > >> IgniteCache as per it will be loaded from CacheStore. However, > > >> CacheStore is not specified in node config. I've added patch that > > >> enables cache store factory, please check IGNITE-11708 attachments. > > >> Regarding IgniteCacheConfigVariationsFullApiTest#testGetOutTx* tests: > > >> from my point of view, test scenarios make no sense. We perform get() > > >> operation from ATOMIC caches and expect that entries will be locked. I > > >> don't understand why we should lock entries on ATOMIC get, therefore I > > >> suppose to remove part of code where we listen and check > > >> EVT_CACHE_OBJECT_LOCKED/UNLOCKED events. > > >> > > >> Best Regards, > > >> Ivan Rakov > > >> > > >> On 17.04.2019 22:05, Ivan Rakov wrote: > > >>> Hi Ivan, > > >>> > > >>> I've checked your branch. Seems like these tests fail due to real > > >>> issue in functionality. > > >>> I'll take a look. > > >>> > > >>> Best Regards, > > >>> Ivan Rakov > > >>> > > >>> On 17.04.2019 13:54, Ivan Fedotov wrote: > > Hi, Igniters! > > > > During work on iep-30[1] I discovered that > > IgniteConfigVariationsAbstractTest subclasses - it is about 15_000 > > tests[2] > > - do not work. > > You can check it just run one of the tests with log output, for > > example > > ConfigVariationsTestSuiteBuilderTest#LegacyLifecycleTest#test1 [3]. > > There is no warning notification in the console. The same situation > > with > > other IgniteConfigVariationsAbstractTest subclasses - tests run, but > > they > > simply represent empty code. > > > > So,
Re: IGNITE-7285 Add default query timeout
Andrey K., > I think we should develop some kind of "Queries" options on Ignite > configuration. Quite a reasonable idea. We already have couple of query-related properties in IgniteConfiguration and we can move them (in a compatible way) to a query properties sub-aggregate. I think it is better to raise a separate topic for that. ср, 1 мая 2019 г. в 09:00, Павлухин Иван : > > Hi Saikat, > > I left a couple of comment on GitHub [1]. > > Perhaps I am missing it but what are the plans for making a default > query timeout workable by using an introduced property in query > execution flow? > > [1] https://github.com/apache/ignite/pull/6490 > > вт, 30 апр. 2019 г. в 02:50, Saikat Maitra : > > > > Hi Ivan, > > > > Yes, I checked this CacheQuery default value > > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/CacheQuery.java#L200 > > > > Also, Andrew recommended the same in review feedback. > > > > https://github.com/apache/ignite/pull/6490#discussion_r277730394 > > > > Regards, > > Saikat > > > > > > On Mon, Apr 29, 2019 at 3:18 AM Павлухин Иван wrote: > > > > > Hi Saikat, > > > > > > It a compatibility with previous versions the reason for an indefinite > > > timeout by default? > > > > > > сб, 27 апр. 2019 г. в 16:58, Saikat Maitra : > > > > > > > > Hi Alexey, Ivan, Andrew > > > > > > > > I think we can provide an option to configure defaultQueryOption at > > > > IgniteConfiguration and set the default value = 0 to imply if not set it > > > > will be execute indefinitely but then user can set this value based on > > > the > > > > application preferences and use it in addition to SQL query timeout. > > > > > > > > I have updated the PR accordingly. > > > > > > > > Please review and share if any changes required. > > > > > > > > Regards, > > > > Saikat > > > > > > > > On Wed, Apr 24, 2019 at 4:33 AM Alexey Kuznetsov > > > > wrote: > > > > > > > > > Hi Saikat and Ivan, > > > > > > > > > > I think that properties that related to SQL should not be configured > > > > > on > > > > > caches. > > > > > We already put a lot of effort to decouple SQL from caches. > > > > > > > > > > I think we should develop some kind of "Queries" options on Ignite > > > > > configuration. > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > On Wed, Apr 24, 2019 at 3:22 PM Павлухин Иван > > > wrote: > > > > > > > > > > > Hi Saikat, > > > > > > > > > > > > I think that we should have a discussion and choose a place where a > > > > > > "default query timeout" property will be configured. > > > > > > > > > > > > Generally, I think that a real (user) problem is possibility for > > > > > > queries to execute indefinitely. And I have no doubts that we can > > > > > > improve there. There could be several implementation strategies. One > > > > > > is adding a property to CacheConfiguration. But it opens various > > > > > > questions. E.g. how should it work if we execute SQL JOIN spanning > > > > > > multiple tables (caches)? Also I am concerned about queries executed > > > > > > not via cache.query() method. We have multiple alternative options, > > > > > > e.g. thin clients (IgniteClient.query) or JDBC. I believe that > > > > > > introducing a default timeout for all them is not a bad idea. > > > > > > > > > > > > What do you think? > > > > > > > > > > > > вт, 23 апр. 2019 г. в 03:01, Saikat Maitra > > >: > > > > > > > > > > > > > > Hi Ivan, > > > > > > > > > > > > > > Thank you for your email. My understanding from the jira issue was > > > it > > > > > > will > > > > > > > be cache level configuration for query default timeout. > > > > > > > > > > > > > > I need more info on the usage for this config property and if it > > > > > > > is > > > > > > shared > > > > > > > in this jira issue I can make changes or if there is a separate > > > jira > > > > > > issue > > > > > > > I can assign myself. > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Saikat > > > > > > > > > > > > > > On Mon, Apr 22, 2019 at 5:31 AM Павлухин Иван > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Saikat, > > > > > > > > > > > > > > > > I see that a configuration property is added in PR but I do not > > > see > > > > > > > > how the property is used. Was it done intentionally? > > > > > > > > > > > > > > > > Also, we need to decide whether such timeout should be > > > configured per > > > > > > > > cache or for all caches in one place. For example, we have > > > already > > > > > > > > TransactionConfiguration#setDefaultTxTimeout which is a global > > > one. > > > > > > > > > > > > > > > > Share you thoughts. > > > > > > > > > > > > > > > > вс, 21 апр. 2019 г. в 00:38, Saikat Maitra < > > > saikat.mai...@gmail.com > > > > > >: > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I have raised a PR for the below issue. > > > > > > > > > > > > > > > > > > IGNITE-7285 Add default query timeout > > > > > > > > > > > > > > > >
Re: IGNITE-7285 Add default query timeout
Hi Saikat, I left a couple of comment on GitHub [1]. Perhaps I am missing it but what are the plans for making a default query timeout workable by using an introduced property in query execution flow? [1] https://github.com/apache/ignite/pull/6490 вт, 30 апр. 2019 г. в 02:50, Saikat Maitra : > > Hi Ivan, > > Yes, I checked this CacheQuery default value > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/CacheQuery.java#L200 > > Also, Andrew recommended the same in review feedback. > > https://github.com/apache/ignite/pull/6490#discussion_r277730394 > > Regards, > Saikat > > > On Mon, Apr 29, 2019 at 3:18 AM Павлухин Иван wrote: > > > Hi Saikat, > > > > It a compatibility with previous versions the reason for an indefinite > > timeout by default? > > > > сб, 27 апр. 2019 г. в 16:58, Saikat Maitra : > > > > > > Hi Alexey, Ivan, Andrew > > > > > > I think we can provide an option to configure defaultQueryOption at > > > IgniteConfiguration and set the default value = 0 to imply if not set it > > > will be execute indefinitely but then user can set this value based on > > the > > > application preferences and use it in addition to SQL query timeout. > > > > > > I have updated the PR accordingly. > > > > > > Please review and share if any changes required. > > > > > > Regards, > > > Saikat > > > > > > On Wed, Apr 24, 2019 at 4:33 AM Alexey Kuznetsov > > > wrote: > > > > > > > Hi Saikat and Ivan, > > > > > > > > I think that properties that related to SQL should not be configured on > > > > caches. > > > > We already put a lot of effort to decouple SQL from caches. > > > > > > > > I think we should develop some kind of "Queries" options on Ignite > > > > configuration. > > > > > > > > What do you think? > > > > > > > > > > > > On Wed, Apr 24, 2019 at 3:22 PM Павлухин Иван > > wrote: > > > > > > > > > Hi Saikat, > > > > > > > > > > I think that we should have a discussion and choose a place where a > > > > > "default query timeout" property will be configured. > > > > > > > > > > Generally, I think that a real (user) problem is possibility for > > > > > queries to execute indefinitely. And I have no doubts that we can > > > > > improve there. There could be several implementation strategies. One > > > > > is adding a property to CacheConfiguration. But it opens various > > > > > questions. E.g. how should it work if we execute SQL JOIN spanning > > > > > multiple tables (caches)? Also I am concerned about queries executed > > > > > not via cache.query() method. We have multiple alternative options, > > > > > e.g. thin clients (IgniteClient.query) or JDBC. I believe that > > > > > introducing a default timeout for all them is not a bad idea. > > > > > > > > > > What do you think? > > > > > > > > > > вт, 23 апр. 2019 г. в 03:01, Saikat Maitra > >: > > > > > > > > > > > > Hi Ivan, > > > > > > > > > > > > Thank you for your email. My understanding from the jira issue was > > it > > > > > will > > > > > > be cache level configuration for query default timeout. > > > > > > > > > > > > I need more info on the usage for this config property and if it is > > > > > shared > > > > > > in this jira issue I can make changes or if there is a separate > > jira > > > > > issue > > > > > > I can assign myself. > > > > > > > > > > > > > > > > > > Regards, > > > > > > Saikat > > > > > > > > > > > > On Mon, Apr 22, 2019 at 5:31 AM Павлухин Иван > > > > > > > wrote: > > > > > > > > > > > > > Hi Saikat, > > > > > > > > > > > > > > I see that a configuration property is added in PR but I do not > > see > > > > > > > how the property is used. Was it done intentionally? > > > > > > > > > > > > > > Also, we need to decide whether such timeout should be > > configured per > > > > > > > cache or for all caches in one place. For example, we have > > already > > > > > > > TransactionConfiguration#setDefaultTxTimeout which is a global > > one. > > > > > > > > > > > > > > Share you thoughts. > > > > > > > > > > > > > > вс, 21 апр. 2019 г. в 00:38, Saikat Maitra < > > saikat.mai...@gmail.com > > > > >: > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I have raised a PR for the below issue. > > > > > > > > > > > > > > > > IGNITE-7285 Add default query timeout > > > > > > > > > > > > > > > > PR - https://github.com/apache/ignite/pull/6490 > > > > > > > > > > > > > > > > Please take a look and share feedback. > > > > > > > > > > > > > > > > Regards, > > > > > > > > Saikat > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > -- > > > > Alexey Kuznetsov > > > > > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > -- Best regards, Ivan Pavlukhin
