[jira] [Created] (IGNITE-12334) Throttle getAll operations on the grid
ABHISHEK SHUBH GUPTA created IGNITE-12334: - Summary: Throttle getAll operations on the grid Key: IGNITE-12334 URL: https://issues.apache.org/jira/browse/IGNITE-12334 Project: Ignite Issue Type: Improvement Affects Versions: 2.7.5 Reporter: ABHISHEK SHUBH GUPTA Per [this thread|http://apache-ignite-users.70518.x6.nabble.com/Throttling-getAll-tt29956.html], in a situation where a large number of keys are requested using getAll and/or the value objects are large, there is a worry about GC issues/humongous objects/OOM on the grid. It would, therefore, be beneficial to the stability of the product if there was throttling available on the ignition nodes when clients do large getAll operations. The throttling could perhaps be based on the number of keys passed on to getAll. If it exceeds threshold it could reject the request or optionally, auto-create multiple smaller batches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12333) [ML] Cleanup the ML module code
Alexey Zinoviev created IGNITE-12333: Summary: [ML] Cleanup the ML module code Key: IGNITE-12333 URL: https://issues.apache.org/jira/browse/IGNITE-12333 Project: Ignite Issue Type: Improvement Components: ml Affects Versions: 2.8 Reporter: Alexey Zinoviev Assignee: Alexey Zinoviev Fix For: 2.8 # Formatting # Abbreviation # Codestyle # Missed JavaDocs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12333) [ML] Cleanup the ML module code
[ https://issues.apache.org/jira/browse/IGNITE-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12333: - Ignite Flags: (was: Docs Required,Release Notes Required) > [ML] Cleanup the ML module code > --- > > Key: IGNITE-12333 > URL: https://issues.apache.org/jira/browse/IGNITE-12333 > Project: Ignite > Issue Type: Improvement > Components: ml >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 2.8 > > > # Formatting > # Abbreviation > # Codestyle > # Missed JavaDocs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12333) [ML] Cleanup the ML module code
[ https://issues.apache.org/jira/browse/IGNITE-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12333: - Labels: await (was: ) > [ML] Cleanup the ML module code > --- > > Key: IGNITE-12333 > URL: https://issues.apache.org/jira/browse/IGNITE-12333 > Project: Ignite > Issue Type: Improvement > Components: ml >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Labels: await > Fix For: 2.8 > > > # Formatting > # Abbreviation > # Codestyle > # Missed JavaDocs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12332) Fix flaky test GridCacheAtomicClientInvalidPartitionHandlingSelfTest#testPrimaryFullAsync
Alexei Scherbakov created IGNITE-12332: -- Summary: Fix flaky test GridCacheAtomicClientInvalidPartitionHandlingSelfTest#testPrimaryFullAsync Key: IGNITE-12332 URL: https://issues.apache.org/jira/browse/IGNITE-12332 Project: Ignite Issue Type: Bug Affects Versions: 2.7.6 Reporter: Alexei Scherbakov Fix For: 2.8 Can be reproduced locally with range = 10_000 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12331) [ML] ML Preprocessing doesn't work on SQL Tables
[
https://issues.apache.org/jira/browse/IGNITE-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12331:
-
Affects Version/s: 3.0
> [ML] ML Preprocessing doesn't work on SQL Tables
>
>
> Key: IGNITE-12331
> URL: https://issues.apache.org/jira/browse/IGNITE-12331
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 3.0
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
>
> {code:java}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version 2.0
> * (the "License"); you may not use this file except in compliance with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.ignite.examples.ml.tutorial.sql;
> import java.util.List;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.query.QueryCursor;
> import org.apache.ignite.cache.query.SqlFieldsQuery;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.internal.util.IgniteUtils;
> import org.apache.ignite.ml.dataset.feature.extractor.Vectorizer;
> import
> org.apache.ignite.ml.dataset.feature.extractor.impl.BinaryObjectVectorizer;
> import org.apache.ignite.ml.math.primitives.vector.Vector;
> import org.apache.ignite.ml.math.primitives.vector.VectorUtils;
> import org.apache.ignite.ml.preprocessing.Preprocessor;
> import org.apache.ignite.ml.preprocessing.minmaxscaling.MinMaxScalerTrainer;
> import org.apache.ignite.ml.preprocessing.normalization.NormalizationTrainer;
> import org.apache.ignite.ml.sql.SqlDatasetBuilder;
> import org.apache.ignite.ml.tree.DecisionTreeClassificationTrainer;
> import org.apache.ignite.ml.tree.DecisionTreeNode;
> /**
> * Example of using distributed {@link DecisionTreeClassificationTrainer} on
> a data stored in SQL table.
> */
> public class PreprocessingAndTrainingSQLTableExample {
> /**
> * Dummy cache name.
> */
> private static final String DUMMY_CACHE_NAME = "dummy_cache";
> /**
> * Training data.
> */
> private static final String TRAIN_DATA_RES =
> "examples/src/main/resources/datasets/titanic_train.csv";
> /**
> * Test data.
> */
> private static final String TEST_DATA_RES =
> "examples/src/main/resources/datasets/titanic_test.csv";
> /**
> * Run example.
> */
> public static void main(String[] args) {
> System.out.println(">>> Decision tree classification trainer example
> started.");
> // Start ignite grid.
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
> System.out.println(">>> Ignite grid started.");
> // Dummy cache is required to perform SQL queries.
> CacheConfiguration cacheCfg = new
> CacheConfiguration<>(DUMMY_CACHE_NAME)
> .setSqlSchema("PUBLIC");
> IgniteCache cache = null;
> try {
> cache = ignite.getOrCreateCache(cacheCfg);
> System.out.println(">>> Creating table with training
> data...");
> cache.query(new SqlFieldsQuery("create table titanic_train
> (\n" +
> "passengerid int primary key,\n" +
> "survived int,\n" +
> "pclass int,\n" +
> "name varchar(255),\n" +
> "sex varchar(255),\n" +
> "age float,\n" +
> "sibsp int,\n" +
> "parch int,\n" +
> "ticket varchar(255),\n" +
> "fare float,\n" +
> "cabin varchar(255),\n" +
> "embarked varchar(255)\n" +
> ") with \"template=partitioned\";")).getAll();
> System.out.println(">>> Filling training data...");
> cache.query(new SqlFieldsQuery("insert into titanic_train
> select * from csvread('" +
>
> IgniteUtils.resolveIgnitePath(TRAIN_DATA_RES).getAbsolutePath() +
>
[jira] [Updated] (IGNITE-12331) [ML] ML Preprocessing doesn't work on SQL Tables
[
https://issues.apache.org/jira/browse/IGNITE-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12331:
-
Fix Version/s: 3.0
> [ML] ML Preprocessing doesn't work on SQL Tables
>
>
> Key: IGNITE-12331
> URL: https://issues.apache.org/jira/browse/IGNITE-12331
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 3.0
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
> Fix For: 3.0
>
>
> {code:java}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version 2.0
> * (the "License"); you may not use this file except in compliance with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.ignite.examples.ml.tutorial.sql;
> import java.util.List;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.query.QueryCursor;
> import org.apache.ignite.cache.query.SqlFieldsQuery;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.internal.util.IgniteUtils;
> import org.apache.ignite.ml.dataset.feature.extractor.Vectorizer;
> import
> org.apache.ignite.ml.dataset.feature.extractor.impl.BinaryObjectVectorizer;
> import org.apache.ignite.ml.math.primitives.vector.Vector;
> import org.apache.ignite.ml.math.primitives.vector.VectorUtils;
> import org.apache.ignite.ml.preprocessing.Preprocessor;
> import org.apache.ignite.ml.preprocessing.minmaxscaling.MinMaxScalerTrainer;
> import org.apache.ignite.ml.preprocessing.normalization.NormalizationTrainer;
> import org.apache.ignite.ml.sql.SqlDatasetBuilder;
> import org.apache.ignite.ml.tree.DecisionTreeClassificationTrainer;
> import org.apache.ignite.ml.tree.DecisionTreeNode;
> /**
> * Example of using distributed {@link DecisionTreeClassificationTrainer} on
> a data stored in SQL table.
> */
> public class PreprocessingAndTrainingSQLTableExample {
> /**
> * Dummy cache name.
> */
> private static final String DUMMY_CACHE_NAME = "dummy_cache";
> /**
> * Training data.
> */
> private static final String TRAIN_DATA_RES =
> "examples/src/main/resources/datasets/titanic_train.csv";
> /**
> * Test data.
> */
> private static final String TEST_DATA_RES =
> "examples/src/main/resources/datasets/titanic_test.csv";
> /**
> * Run example.
> */
> public static void main(String[] args) {
> System.out.println(">>> Decision tree classification trainer example
> started.");
> // Start ignite grid.
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
> System.out.println(">>> Ignite grid started.");
> // Dummy cache is required to perform SQL queries.
> CacheConfiguration cacheCfg = new
> CacheConfiguration<>(DUMMY_CACHE_NAME)
> .setSqlSchema("PUBLIC");
> IgniteCache cache = null;
> try {
> cache = ignite.getOrCreateCache(cacheCfg);
> System.out.println(">>> Creating table with training
> data...");
> cache.query(new SqlFieldsQuery("create table titanic_train
> (\n" +
> "passengerid int primary key,\n" +
> "survived int,\n" +
> "pclass int,\n" +
> "name varchar(255),\n" +
> "sex varchar(255),\n" +
> "age float,\n" +
> "sibsp int,\n" +
> "parch int,\n" +
> "ticket varchar(255),\n" +
> "fare float,\n" +
> "cabin varchar(255),\n" +
> "embarked varchar(255)\n" +
> ") with \"template=partitioned\";")).getAll();
> System.out.println(">>> Filling training data...");
> cache.query(new SqlFieldsQuery("insert into titanic_train
> select * from csvread('" +
>
>
[jira] [Created] (IGNITE-12331) [ML] ML Preprocessing doesn't work on SQL Tables
Alexey Zinoviev created IGNITE-12331:
Summary: [ML] ML Preprocessing doesn't work on SQL Tables
Key: IGNITE-12331
URL: https://issues.apache.org/jira/browse/IGNITE-12331
Project: Ignite
Issue Type: Bug
Components: ml
Reporter: Alexey Zinoviev
Assignee: Alexey Zinoviev
{code:java}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.ignite.examples.ml.tutorial.sql;
import java.util.List;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.query.QueryCursor;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.internal.util.IgniteUtils;
import org.apache.ignite.ml.dataset.feature.extractor.Vectorizer;
import
org.apache.ignite.ml.dataset.feature.extractor.impl.BinaryObjectVectorizer;
import org.apache.ignite.ml.math.primitives.vector.Vector;
import org.apache.ignite.ml.math.primitives.vector.VectorUtils;
import org.apache.ignite.ml.preprocessing.Preprocessor;
import org.apache.ignite.ml.preprocessing.minmaxscaling.MinMaxScalerTrainer;
import org.apache.ignite.ml.preprocessing.normalization.NormalizationTrainer;
import org.apache.ignite.ml.sql.SqlDatasetBuilder;
import org.apache.ignite.ml.tree.DecisionTreeClassificationTrainer;
import org.apache.ignite.ml.tree.DecisionTreeNode;
/**
* Example of using distributed {@link DecisionTreeClassificationTrainer} on a
data stored in SQL table.
*/
public class PreprocessingAndTrainingSQLTableExample {
/**
* Dummy cache name.
*/
private static final String DUMMY_CACHE_NAME = "dummy_cache";
/**
* Training data.
*/
private static final String TRAIN_DATA_RES =
"examples/src/main/resources/datasets/titanic_train.csv";
/**
* Test data.
*/
private static final String TEST_DATA_RES =
"examples/src/main/resources/datasets/titanic_test.csv";
/**
* Run example.
*/
public static void main(String[] args) {
System.out.println(">>> Decision tree classification trainer example
started.");
// Start ignite grid.
try (Ignite ignite =
Ignition.start("examples/config/example-ignite.xml")) {
System.out.println(">>> Ignite grid started.");
// Dummy cache is required to perform SQL queries.
CacheConfiguration cacheCfg = new
CacheConfiguration<>(DUMMY_CACHE_NAME)
.setSqlSchema("PUBLIC");
IgniteCache cache = null;
try {
cache = ignite.getOrCreateCache(cacheCfg);
System.out.println(">>> Creating table with training data...");
cache.query(new SqlFieldsQuery("create table titanic_train (\n"
+
"passengerid int primary key,\n" +
"survived int,\n" +
"pclass int,\n" +
"name varchar(255),\n" +
"sex varchar(255),\n" +
"age float,\n" +
"sibsp int,\n" +
"parch int,\n" +
"ticket varchar(255),\n" +
"fare float,\n" +
"cabin varchar(255),\n" +
"embarked varchar(255)\n" +
") with \"template=partitioned\";")).getAll();
System.out.println(">>> Filling training data...");
cache.query(new SqlFieldsQuery("insert into titanic_train
select * from csvread('" +
IgniteUtils.resolveIgnitePath(TRAIN_DATA_RES).getAbsolutePath() +
"')")).getAll();
System.out.println(">>> Creating table with test data...");
cache.query(new SqlFieldsQuery("create table titanic_test (\n" +
"passengerid int primary key,\n" +
"pclass int,\n" +
"name varchar(255),\n" +
"sex varchar(255),\n" +
"age float,\n" +
"sibsp
[jira] [Commented] (IGNITE-12264) Private application data should not be lit in the logs, exceptions, ERROR, WARN etc.
[ https://issues.apache.org/jira/browse/IGNITE-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961095#comment-16961095 ] Alexei Scherbakov commented on IGNITE-12264: [~KPushenko] This feature exists for 3 years [1] Have you tried to enable it using -DIGNITE_TO_STRING_INCLUDE_SENSITIVE=false ? [1] https://issues.apache.org/jira/browse/IGNITE-4167 > Private application data should not be lit in the logs, exceptions, ERROR, > WARN etc. > > > Key: IGNITE-12264 > URL: https://issues.apache.org/jira/browse/IGNITE-12264 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.7.6 >Reporter: Pushenko Kirill >Priority: Major > > Private application data should not be lit in the logs, exceptions, ERROR, > WARN etc. > The executions contained a value in which there were cardboard numbers. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12189) Implement correct limit for TextQuery
[ https://issues.apache.org/jira/browse/IGNITE-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961071#comment-16961071 ] Yuriy Shuliha edited comment on IGNITE-12189 at 10/28/19 2:04 PM: --- [~amashenkov], [~Pavlukhin] the recent changes requested by you were implemented. Current realization still coming with 0 or less as unlimited value. Lets go with this or consider -1 to move forward this PR. CC: [~dma...@apache.org] was (Author: yuriy_shuliha): [~amashenkov], [~Pavlukhin] the recent changes requested by you whore implemented. Current realization still coming with 0 or less as unlimited value. Lets go with this or consider -1 to move forward this PR. CC: [~dma...@apache.org] > Implement correct limit for TextQuery > - > > Key: IGNITE-12189 > URL: https://issues.apache.org/jira/browse/IGNITE-12189 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Yuriy Shuliha >Assignee: Yuriy Shuliha >Priority: Major > Fix For: 2.8 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > PROBLEM > For now each server-node returns all response records to the client-node and > it may contain ~thousands, ~hundred thousands records. > Event if we need only first 10-100. Again, all the results are added to > queue in _*GridCacheQueryFutureAdapter*_ in arbitrary order by pages. > There are no any means to deliver deterministic result. > SOLUTION > Implement _*limit*_ as parameter for _*TextQuery*_ and > _*GridCacheQueryRequest*_ > It should be passed as limit parameter in Lucene's > _*IndexSearcher.search()*_ in _*GridLuceneIndex*_. > For distributed queries _*limit*_ will also trim response queue when merging > results. > Type: long > Special value: : 0 -> No limit (Integer.MAX_VALUE); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12189) Implement correct limit for TextQuery
[ https://issues.apache.org/jira/browse/IGNITE-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961071#comment-16961071 ] Yuriy Shuliha edited comment on IGNITE-12189 at 10/28/19 2:03 PM: --- [~amashenkov], [~Pavlukhin] the recent changes requested by you whore implemented. Current realization still coming with 0 or less as unlimited value. Lets go with this or consider -1 to move forward this PR. CC: [~dma...@apache.org] was (Author: yuriy_shuliha): [~amashenkov], [~Pavlukhin] the recent changes requested by you whore implemented. Current realization still coming with 0 or less as unlimited value. Lets go with this or consider -1 to move forward this PR. > Implement correct limit for TextQuery > - > > Key: IGNITE-12189 > URL: https://issues.apache.org/jira/browse/IGNITE-12189 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Yuriy Shuliha >Assignee: Yuriy Shuliha >Priority: Major > Fix For: 2.8 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > PROBLEM > For now each server-node returns all response records to the client-node and > it may contain ~thousands, ~hundred thousands records. > Event if we need only first 10-100. Again, all the results are added to > queue in _*GridCacheQueryFutureAdapter*_ in arbitrary order by pages. > There are no any means to deliver deterministic result. > SOLUTION > Implement _*limit*_ as parameter for _*TextQuery*_ and > _*GridCacheQueryRequest*_ > It should be passed as limit parameter in Lucene's > _*IndexSearcher.search()*_ in _*GridLuceneIndex*_. > For distributed queries _*limit*_ will also trim response queue when merging > results. > Type: long > Special value: : 0 -> No limit (Integer.MAX_VALUE); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12189) Implement correct limit for TextQuery
[ https://issues.apache.org/jira/browse/IGNITE-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961071#comment-16961071 ] Yuriy Shuliha commented on IGNITE-12189: - [~amashenkov], [~Pavlukhin] the recent changes requested by you whore implemented. Current realization still coming with 0 or less as unlimited value. Lets go with this or consider -1 to move forward this PR. > Implement correct limit for TextQuery > - > > Key: IGNITE-12189 > URL: https://issues.apache.org/jira/browse/IGNITE-12189 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Yuriy Shuliha >Assignee: Yuriy Shuliha >Priority: Major > Fix For: 2.8 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > PROBLEM > For now each server-node returns all response records to the client-node and > it may contain ~thousands, ~hundred thousands records. > Event if we need only first 10-100. Again, all the results are added to > queue in _*GridCacheQueryFutureAdapter*_ in arbitrary order by pages. > There are no any means to deliver deterministic result. > SOLUTION > Implement _*limit*_ as parameter for _*TextQuery*_ and > _*GridCacheQueryRequest*_ > It should be passed as limit parameter in Lucene's > _*IndexSearcher.search()*_ in _*GridLuceneIndex*_. > For distributed queries _*limit*_ will also trim response queue when merging > results. > Type: long > Special value: : 0 -> No limit (Integer.MAX_VALUE); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12330) Assertion error in CachedDeploymentInfo
Nikolay Izhikov created IGNITE-12330:
Summary: Assertion error in CachedDeploymentInfo
Key: IGNITE-12330
URL: https://issues.apache.org/jira/browse/IGNITE-12330
Project: Ignite
Issue Type: Bug
Affects Versions: 2.7.6
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov
Fix For: 2.8
The following exception occured on some production environment.
It leads to the node fail.
{noformat}
2019-09-17
18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
msg=GridCacheQueryRequest [id=4922,
cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
incMeta=false, all=false, keepBinary=true,
subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
super=GridCacheIdMessage [cacheId=-724666788]]]2019-09-17
18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
msg=GridCacheQueryRequest [id=4922,
cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
incMeta=false, all=false, keepBinary=true,
subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
super=GridCacheIdMessage [cacheId=-724666788]]]
java.lang.AssertionError: null
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-09-17
18:29:29.912[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][org.apache.ignite.Ignite]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.AssertionError]]
java.lang.AssertionError: null
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
at
[jira] [Created] (IGNITE-12329) Invalid handling of remote entries causes partition desync and transaction hanging in COMMITTING state.
Alexei Scherbakov created IGNITE-12329: -- Summary: Invalid handling of remote entries causes partition desync and transaction hanging in COMMITTING state. Key: IGNITE-12329 URL: https://issues.apache.org/jira/browse/IGNITE-12329 Project: Ignite Issue Type: Bug Affects Versions: 2.7.6 Reporter: Alexei Scherbakov Assignee: Alexei Scherbakov Fix For: 2.8 This can happen if transaction is mapped on a partition which is about to be evicted on backup. Due to bugs entry belonging to other cache may be excluded from commit or entry containing a lock can be removed without lock release causes depending transactions to hang. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12316) Extend test coverage [IGNITE-10761] GridCacheProcessor should add info about cache in exception message, if applicable.
[
https://issues.apache.org/jira/browse/IGNITE-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961017#comment-16961017
]
Vyacheslav Koptilin commented on IGNITE-12316:
--
Hello [~ktkale...@gridgain.com],
The tests look good to me!
> Extend test coverage [IGNITE-10761] GridCacheProcessor should add info about
> cache in exception message, if applicable.
> ---
>
> Key: IGNITE-12316
> URL: https://issues.apache.org/jira/browse/IGNITE-12316
> Project: Ignite
> Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Test cache operations that should fail inside tx thread
> ||*Forbidden operation inside transaction*||
> |dynamic single cache start|
> |dynamic start of multiple caches|
> |dynamic cache destroy|
> |dynamic destroy of multiple caches|
> |dynamic cache close|
> |reset cache state|
> {quote}org.apache.ignite.internal.processors.cache.GridCacheProcessorActiveTxTest#testDynamicSingleCacheStart
> org.apache.ignite.internal.processors.cache.GridCacheProcessorActiveTxTest#testDynamicStartMultipleCaches
> org.apache.ignite.internal.processors.cache.GridCacheProcessorActiveTxTest#testDynamicCacheDestroy
> org.apache.ignite.internal.processors.cache.GridCacheProcessorActiveTxTest#testDynamicDestroyMultipleCaches
> org.apache.ignite.internal.processors.cache.GridCacheProcessorActiveTxTest#testDynamicCacheClose
> org.apache.ignite.internal.processors.cache.GridCacheProcessorActiveTxTest#testResetCacheState\{quote}
> # Start cluster, start active transaction in thread
> # Assert that in *Forbidden operation inside transaction* scenarios exception
> occurs with log information about cache name and reason
> # Finish running transaction, assert that there is no exception while
> evaluating *Forbidden operation inside transaction*
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-1606) NPE during node stop due to nullified logger in TcpCommunicationSpi
[
https://issues.apache.org/jira/browse/IGNITE-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960980#comment-16960980
]
Ignite TC Bot commented on IGNITE-1606:
---
{panel:title=Branch: [pull/7004/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4731332buildTypeId=IgniteTests24Java8_RunAll]
> NPE during node stop due to nullified logger in TcpCommunicationSpi
> ---
>
> Key: IGNITE-1606
> URL: https://issues.apache.org/jira/browse/IGNITE-1606
> Project: Ignite
> Issue Type: Bug
> Components: general
>Reporter: Valentin Kulichenko
>Assignee: Dmitriy Sorokin
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Probably we should check other components as well. Not sure why we need to
> nullify logger.
> {noformat}
> Exception in thread "ignite-#101%sys-t1-1%" java.lang.NullPointerException
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1896)
> at
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1880)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1066)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1214)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1071)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1034)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.sendWithRetries(GridContinuousProcessor.java:1017)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.processNotification(GridContinuousProcessor.java:851)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.access$1500(GridContinuousProcessor.java:85)
> at
> org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.onMessage(GridContinuousProcessor.java:556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2302)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:992)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$1700(GridIoManager.java:106)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$6.run(GridIoManager.java:961)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11761) Normalize encoding for Ignite .NET test file
[ https://issues.apache.org/jira/browse/IGNITE-11761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960978#comment-16960978 ] Alexandr Shapkin commented on IGNITE-11761: --- It turned out that the PR was merged with a wrong message, without IGNITE-XXX prefix. We need to exclude this ticket from consistency check > Normalize encoding for Ignite .NET test file > > > Key: IGNITE-11761 > URL: https://issues.apache.org/jira/browse/IGNITE-11761 > Project: Ignite > Issue Type: Task >Reporter: Dmitry Pavlov >Assignee: Alexandr Shapkin >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > It is encoded in UTF-16, but all other files are UTF-8 > Idea blocks me from changing encoding because of BOM exists. > https://stackoverflow.com/questions/32986445/remove-a-bom-character-in-a-file -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12207) Inclusion of super.toString() info into some descenders of GridCacheMessage
[
https://issues.apache.org/jira/browse/IGNITE-12207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Sorokin updated IGNITE-12207:
-
Description:
Sometimes when errors related to processing of descenders of GridCacheMessage
happens, we could need information which contained at the GridCacheMessage
class, in particular deployment information, contained if depInfo field. In the
some message classes which extends GridCacheMessage, toString() method doesn't
include the 'super' part, so we haven't that information at log error messages,
as at example below:
{noformat}
2019-09-17
18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
msg=GridCacheQueryRequest [id=4922,
cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
incMeta=false, all=false, keepBinary=true,
subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
super=GridCacheIdMessage [cacheId=-724666788]]]2019-09-17
18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
msg=GridCacheQueryRequest [id=4922,
cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
incMeta=false, all=false, keepBinary=true,
subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
super=GridCacheIdMessage [cacheId=-724666788]]]
java.lang.AssertionError: null
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
at
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The assertion condition which produced error above includes the value which
obtained from GridCacheMessage.depInfo.
> Inclusion of super.toString() info into some descenders of GridCacheMessage
> ---
>
> Key: IGNITE-12207
> URL: https://issues.apache.org/jira/browse/IGNITE-12207
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.7, 2.7.6
>Reporter: Dmitriy Sorokin
>Assignee: Dmitriy Sorokin
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Sometimes when errors related to processing of descenders of GridCacheMessage
> happens, we could need information which contained at the GridCacheMessage
> class, in particular deployment information, contained if depInfo field. In
> the some message classes which extends GridCacheMessage, toString() method
> doesn't include the 'super' part, so we haven't that information at log error
> messages, as at example below:
> {noformat}
> 2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
>
[jira] [Commented] (IGNITE-7285) Add default query timeout
[
https://issues.apache.org/jira/browse/IGNITE-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960947#comment-16960947
]
Igor Sapego commented on IGNITE-7285:
-
[~ptupitsyn] [~samaitra] I confirm, we need to update C++ as well.
> Add default query timeout
> -
>
> Key: IGNITE-7285
> URL: https://issues.apache.org/jira/browse/IGNITE-7285
> Project: Ignite
> Issue Type: Improvement
> Components: cache, sql
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Saikat Maitra
>Priority: Major
> Labels: sql-stability
> Fix For: 2.8
>
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> Currently it's possible to provide timeout only on query level. It would be
> very useful to have default timeout value provided on cache startup. Let's
> add {{CacheConfiguration#defaultQueryTimeout}} configuration property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12200) More informative assertion message at constructor of CachedDeploymentInfo (GridCacheDeploymentManager class)
[
https://issues.apache.org/jira/browse/IGNITE-12200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960919#comment-16960919
]
Ignite TC Bot commented on IGNITE-12200:
{panel:title=Branch: [pull/6886/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4724401buildTypeId=IgniteTests24Java8_RunAll]
> More informative assertion message at constructor of CachedDeploymentInfo
> (GridCacheDeploymentManager class)
>
>
> Key: IGNITE-12200
> URL: https://issues.apache.org/jira/browse/IGNITE-12200
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.5, 2.7.5
>Reporter: Dmitriy Sorokin
>Assignee: Dmitriy Sorokin
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java}
> /**
> * @param sndId Sender.
> * @param ldrId Loader ID.
> * @param userVer User version.
> * @param depMode Deployment mode.
> * @param participants Participants.
> */
> private CachedDeploymentInfo(UUID sndId, IgniteUuid ldrId, String userVer,
> DeploymentMode depMode,
> Map participants) {
> assert sndId.equals(ldrId.globalId()) || participants != null;
> this.sndId = sndId;
> this.ldrId = ldrId;
> this.userVer = userVer;
> this.depMode = depMode;
> this.participants = participants == null || participants.isEmpty() ? null
> :
> new ConcurrentLinkedHashMap<>(participants);
> }
> {code}
> The code above may produce the following stacktrace, where AssertionError
> should contain more informative message for better root cause analysis:
> {noformat}
> 2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
> msg=GridCacheQueryRequest [id=4922,
> cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
> type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
> rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
> incMeta=false, all=false, keepBinary=true,
> subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
> topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
> super=GridCacheIdMessage [cacheId=-724666788]]]2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
> msg=GridCacheQueryRequest [id=4922,
> cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
> type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
> rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
> incMeta=false, all=false, keepBinary=true,
> subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
> topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
> super=GridCacheIdMessage [cacheId=-724666788]]]
> java.lang.AssertionError: null
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
[jira] [Commented] (IGNITE-12108) [IEP-35] Migrate Communication Metrics.
[
https://issues.apache.org/jira/browse/IGNITE-12108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960838#comment-16960838
]
Nikolay Izhikov commented on IGNITE-12108:
--
> I wanted to introduce metrics for Communication SPI
All I ask you regarding this PR is to avoid Nullable constructor params.
Can we do it?
> That method is deprecated and invoked in JMX only, should I really care about
> rare single get operation?
1.. Yes you should, because, many existing deployments use existing metrics.
It takes time for it to migrate on the new framework.
2. It's easy to fix, please, do it.
> What was the motivation behind your comment?
You should avoid any unnecessary changes in the codebase for the sake of review
simplicity.
{this} fixes are unnecessary. Please, revert them in this PR.
If you want to refactor it I propose the better way:
1. Discuss this code style point on the dev-list.
2. Find checkstyle rule to automatically check it.
3. Refactor all Ignite codebase in single commit to the preferred assignment
style
WDYT?
> [IEP-35] Migrate Communication Metrics.
> ---
>
> Key: IGNITE-12108
> URL: https://issues.apache.org/jira/browse/IGNITE-12108
> Project: Ignite
> Issue Type: New Feature
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: IEP-35, await
> Fix For: 2.8
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> ||*Name*||*Description*||
> |communication.tcp.outboundMessagesQueueSize|Number of messages waiting to be
> sent|
> |communication.tcp.sentBytes|Total number of bytes received by current node|
> |communication.tcp.receivedBytes|Total number of bytes sent by current node|
> |communication.tcp.sentMessagesCount|Total number of messages sent by current
> node|
> |communication.tcp.receivedMessagesCount|Total number of messages received by
> current node|
> |communication.tcp.sentMessagesByType.|Total number of messages
> with given type sent by current node|
> |communication.tcp.receivedMessagesByType.|Total number of
> messages with given type received by current node|
> |communication.tcp..sentMessagesToNode|Total number of messages sent
> by current node to the given node|
> |communication.tcp..receivedMessagesFromNode|Total number of messages
> received by current node from the given node|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12108) [IEP-35] Migrate Communication Metrics.
[
https://issues.apache.org/jira/browse/IGNITE-12108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960818#comment-16960818
]
Ivan Bessonov commented on IGNITE-12108:
[~nizhikov] can you please explain why it's a poor design and why it has to be
fixed in this particular issue?
I wanted to introduce metrics for Communication SPI, not for every other
component that uses NIO server. If you want me to do it then please create
separate issue.
> It's about unnecessary {{Map#get}} operation.
That method is deprecated and invoked in JMX only, should I really care about
rare single get operation? I don't think so.
> Please, revert unnecessary changes.
Do we have any strict rules about removing unnecessary "this." references in
code?
I mean, it's not hard for me to return everything back, but how should I
initialize new field, "this.ctx = ...;" or "ctx = ...;"? What was the
motivation behind your comment?
Please reply, thank you!
> [IEP-35] Migrate Communication Metrics.
> ---
>
> Key: IGNITE-12108
> URL: https://issues.apache.org/jira/browse/IGNITE-12108
> Project: Ignite
> Issue Type: New Feature
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: IEP-35, await
> Fix For: 2.8
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> ||*Name*||*Description*||
> |communication.tcp.outboundMessagesQueueSize|Number of messages waiting to be
> sent|
> |communication.tcp.sentBytes|Total number of bytes received by current node|
> |communication.tcp.receivedBytes|Total number of bytes sent by current node|
> |communication.tcp.sentMessagesCount|Total number of messages sent by current
> node|
> |communication.tcp.receivedMessagesCount|Total number of messages received by
> current node|
> |communication.tcp.sentMessagesByType.|Total number of messages
> with given type sent by current node|
> |communication.tcp.receivedMessagesByType.|Total number of
> messages with given type received by current node|
> |communication.tcp..sentMessagesToNode|Total number of messages sent
> by current node to the given node|
> |communication.tcp..receivedMessagesFromNode|Total number of messages
> received by current node from the given node|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12328) IgniteException "Failed to resolve nodes topology" during cache.removeAll() and constantly changing topology
[
https://issues.apache.org/jira/browse/IGNITE-12328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16960816#comment-16960816
]
Alexei Scherbakov commented on IGNITE-12328:
Step by step to reproduce:
* Start tx from a client node (A). This client node sees topology version X.
* Start another client node (B) to increment topology version on server nodes.
After that action topology version on server nodes will be X + 1.
* Discovery event of B node join should be delivered to A with a delay.
* Perform tx put (1,1) from A. Node A sees topology version X, while server
nodes see topology version X + 1.
* This put will result in telling to node A that tx should be remmaped to
version X + 1.
* Topology version for this tx on node A is set to X + 1. While Node A still
hasn’t received discovery event for node B join (X + 1 version).
* Perform tx remove (1) from A. IMPORTANT. We should use a key that is already
used in transaction. In another case tx will wait for affinity version X + 1.
* This tx remove results to assertion mentioned in ticket, because A doesn’t
see discovery event of X + 1 and exchange future corresponding to this event
and tries to get discovery cache of X + 1 that doesn’t exist on A yet.
> IgniteException "Failed to resolve nodes topology" during cache.removeAll()
> and constantly changing topology
>
>
> Key: IGNITE-12328
> URL: https://issues.apache.org/jira/browse/IGNITE-12328
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.6
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>Priority: Major
> Fix For: 2.8
>
>
> {noformat}
> [2019-09-25 13:13:58,339][ERROR][TxThread-threadNum-3] Failed to complete
> transaction.
> org.apache.ignite.IgniteException: Failed to resolve nodes topology
> [cacheGrp=cache_group_36, topVer=AffinityTopologyVersion [topVer=16,
> minorTopVer=0], history=[AffinityTopologyVersion [topVer=13, minorTopVer=0],
> AffinityTopologyVersion [topVer=14, minorTopVer=0], AffinityTopologyVersion
> [topVer=15, minorTopVer=0]], snap=Snapshot [topVer=AffinityTopologyVersion
> [topVer=15, minorTopVer=0]], locNode=TcpDiscoveryNode
> [id=6cbf7666-9a8c-4b61-8b3f-6351ef44bd4a,
> consistentId=poc-tester-client-172.25.1.21-id-0, addrs=ArrayList
> [172.25.1.21], sockAddrs=HashSet [lab21.gridgain.local/172.25.1.21:0],
> discPort=0, order=13, intOrder=0, lastExchangeTime=1569406379934, loc=true,
> ver=2.5.10#20190922-sha1:02133315, isClient=true]]
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.resolveDiscoCache(GridDiscoveryManager.java:2125)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.cacheGroupAffinityNodes(GridDiscoveryManager.java:2007)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.affinityNodes(GridCacheUtils.java:465)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map0(GridDhtColocatedLockFuture.java:939)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map(GridDhtColocatedLockFuture.java:911)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map(GridDhtColocatedLockFuture.java:811)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.lockAllAsync(GridDhtColocatedCache.java:656)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.GridDistributedCacheAdapter.txLockAsync(GridDistributedCacheAdapter.java:109)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.removeAllAsync0(GridNearTxLocal.java:1648)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.removeAllAsync(GridNearTxLocal.java:521)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter$33.inOp(GridCacheAdapter.java:2619)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:4701)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:3780)
> ~[ignite-core-2.5.10.jar:2.5.10]
> at
>
[jira] [Created] (IGNITE-12328) IgniteException "Failed to resolve nodes topology" during cache.removeAll() and constantly changing topology
Alexei Scherbakov created IGNITE-12328:
--
Summary: IgniteException "Failed to resolve nodes topology" during
cache.removeAll() and constantly changing topology
Key: IGNITE-12328
URL: https://issues.apache.org/jira/browse/IGNITE-12328
Project: Ignite
Issue Type: Bug
Affects Versions: 2.7.6
Reporter: Alexei Scherbakov
Assignee: Alexei Scherbakov
Fix For: 2.8
{noformat}
[2019-09-25 13:13:58,339][ERROR][TxThread-threadNum-3] Failed to complete
transaction.
org.apache.ignite.IgniteException: Failed to resolve nodes topology
[cacheGrp=cache_group_36, topVer=AffinityTopologyVersion [topVer=16,
minorTopVer=0], history=[AffinityTopologyVersion [topVer=13, minorTopVer=0],
AffinityTopologyVersion [topVer=14, minorTopVer=0], AffinityTopologyVersion
[topVer=15, minorTopVer=0]], snap=Snapshot [topVer=AffinityTopologyVersion
[topVer=15, minorTopVer=0]], locNode=TcpDiscoveryNode
[id=6cbf7666-9a8c-4b61-8b3f-6351ef44bd4a,
consistentId=poc-tester-client-172.25.1.21-id-0, addrs=ArrayList [172.25.1.21],
sockAddrs=HashSet [lab21.gridgain.local/172.25.1.21:0], discPort=0, order=13,
intOrder=0, lastExchangeTime=1569406379934, loc=true,
ver=2.5.10#20190922-sha1:02133315, isClient=true]]
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.resolveDiscoCache(GridDiscoveryManager.java:2125)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.cacheGroupAffinityNodes(GridDiscoveryManager.java:2007)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.affinityNodes(GridCacheUtils.java:465)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map0(GridDhtColocatedLockFuture.java:939)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map(GridDhtColocatedLockFuture.java:911)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedLockFuture.map(GridDhtColocatedLockFuture.java:811)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.lockAllAsync(GridDhtColocatedCache.java:656)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.GridDistributedCacheAdapter.txLockAsync(GridDistributedCacheAdapter.java:109)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.removeAllAsync0(GridNearTxLocal.java:1648)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.removeAllAsync(GridNearTxLocal.java:521)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$33.inOp(GridCacheAdapter.java:2619)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:4701)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:3780)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.removeAll0(GridCacheAdapter.java:2617)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.removeAll(GridCacheAdapter.java:2606)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.removeAll(IgniteCacheProxyImpl.java:1553)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.removeAll(GatewayProtectedCacheProxy.java:1026)
~[ignite-core-2.5.10.jar:2.5.10]
at
org.apache.ignite.scenario.TxBalanceTask$TxBody.doTxRemoveAll(TxBalanceTask.java:291)
~[poc-tester-0.1.0-SNAPSHOT.jar:?]
at
org.apache.ignite.scenario.TxBalanceTask$TxBody.call(TxBalanceTask.java:93)
~[poc-tester-0.1.0-SNAPSHOT.jar:?]
at
org.apache.ignite.scenario.TxBalanceTask$TxBody.call(TxBalanceTask.java:70)
~[poc-tester-0.1.0-SNAPSHOT.jar:?]
at
org.apache.ignite.scenario.internal.AbstractTxTask.doInTransaction(AbstractTxTask.java:290)
~[poc-tester-0.1.0-SNAPSHOT.jar:?]
at
org.apache.ignite.scenario.internal.AbstractTxTask.access$400(AbstractTxTask.java:56)
~[poc-tester-0.1.0-SNAPSHOT.jar:?]
at
org.apache.ignite.scenario.internal.AbstractTxTask$TxRunner.call(AbstractTxTask.java:470)
[poc-tester-0.1.0-SNAPSHOT.jar:?]
at
[jira] [Resolved] (IGNITE-12327) Cross-cache tx is mapped on wrong primary when enlisted caches have incompatible assignments.
[ https://issues.apache.org/jira/browse/IGNITE-12327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexei Scherbakov resolved IGNITE-12327. Resolution: Won't Fix Already fixed in IGNITE-12038 > Cross-cache tx is mapped on wrong primary when enlisted caches have > incompatible assignments. > - > > Key: IGNITE-12327 > URL: https://issues.apache.org/jira/browse/IGNITE-12327 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7.6 >Reporter: Alexei Scherbakov >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > This is happening when supplier node is left while rebalancing is partially > completed on demander. > Suppose we have 2 cache groups, rebalance is in progress and for first group > rebalance is done and for second group rebalance is partially done (some > partitions are still MOVING). > At this moment supplier node dies and corresponding topology version is (N,0). > New assignment is computed using current state of partitions and for first > group will be ideal and the same as for next topology (N,1) which will be > triggered after all rebalancing is completed by CacheAffinityChangeMessage. > For second group affinity will not be ideal. > If transaction is started while PME is in progress (N, 0)->(N,1), first lock > request will pass remap check if it's enslists rebalanced group. All > subsequent lock requests will use invalid topology from previous assignment. > Possible fix: return actual locked topology version from first lock request > and use it for all subsequent requests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
