[jira] [Resolved] (IGNITE-10737) Enable inspection Collections.sort(..) can be replaced with list.sort(..)
[ https://issues.apache.org/jira/browse/IGNITE-10737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov resolved IGNITE-10737. -- Fix Version/s: (was: 2.8) Resolution: Won't Fix > Enable inspection Collections.sort(..) can be replaced with list.sort(..) > - > > Key: IGNITE-10737 > URL: https://issues.apache.org/jira/browse/IGNITE-10737 > Project: Ignite > Issue Type: Task >Reporter: Maxim Muzafarov >Priority: Minor > Labels: inspections > > We need to enable inspection which reports calls to Collections.sort(list, > comparator) which could be replaced with list.sort(comparator). > # Update ignite_inspections_teamcity.xml configuration file by enabling > current inspection > # Fix all the code issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12054) Upgrade Spark module to 2.4
[
https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938765#comment-16938765
]
Alexey Zinoviev edited comment on IGNITE-12054 at 9/26/19 5:27 PM:
---
Also, in Spark was fixed bug with incorrect null handling on columns in codition
https://issues.apache.org/jira/browse/SPARK-21479
It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
previous migration from 2.2 to 2.3)
I made experiment with Spark code for version 2.3 it generates the next plan
== Parsed Logical Plan ==
'Project ['jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2]
+- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
:- 'UnresolvedRelation `jt1`
+- 'UnresolvedRelation `jt2`
== Analyzed Logical Plan ==
id1: string, val1: string, id2: string, val2: string
Project [id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25]
+- Join LeftOuter, (val1#11 = val2#25)
:- SubqueryAlias jt1
: +- Relation[id#10,val1#11] csv
+- SubqueryAlias jt2
+- Relation[id#24,val2#25] csv
== Optimized Logical Plan ==
Project [id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25]
+- Join LeftOuter, (val1#11 = val2#25)
:- Relation[id#10,val1#11] csv
+- Relation[id#24,val2#25] csv
The 2.4 generates
== Parsed Logical Plan ==
'Project ['jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2]
+- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
:- 'UnresolvedRelation `jt1`
+- 'UnresolvedRelation `jt2`
== Analyzed Logical Plan ==
id1: string, val1: string, id2: string, val2: string
Project [id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25]
+- Join LeftOuter, (val1#11 = val2#25)
:- SubqueryAlias `jt1`
: +- Relation[id#10,val1#11] csv
+- SubqueryAlias `jt2`
+- Relation[id#24,val2#25] csv
== Optimized Logical Plan ==
Project [id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25]
+- Join LeftOuter, (val1#11 = val2#25)
:- Relation[id#10,val1#11] csv
+- Filter isnotnull(val2#25)
+- Relation[id#24,val2#25] csv
The +- Filter isnotnull(val2#25) is added in optimized logical plan
But in reality it doesn't work properly (and doesn't filter in Spark), but wow!
It works for Ignite (because we honestly work with Spark plan)
If you enable next option
.config("ignite.disableSparkSQLOptimization", "true") - the behaviour will be
the same in Ignite and Spark and will not add the filter
The best approach for Spark 2.4 - disableSparkOptimization before fixing on
Spark side (I could create a bug for this)
was (Author: zaleslaw):
Also, in Spark was fixed bug with incorrect null handling on columns in codition
https://issues.apache.org/jira/browse/SPARK-21479
It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
previous migration from 2.2 to 2.3)
> Upgrade Spark module to 2.4
> ---
>
> Key: IGNITE-12054
> URL: https://issues.apache.org/jira/browse/IGNITE-12054
> Project: Ignite
> Issue Type: Task
> Components: spark
>Affects Versions: 2.7.5
>Reporter: Denis A. Magda
>Assignee: Alexey Zinoviev
>Priority: Blocker
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Users can't use APIs that are already available in Spark 2.4:
> https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite
> Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as
> a separate module that can support multiple Spark versions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12054) Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938765#comment-16938765 ] Alexey Zinoviev commented on IGNITE-12054: -- Also, in Spark was fixed bug with incorrect null handling on columns in codition https://issues.apache.org/jira/browse/SPARK-21479 It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the previous migration from 2.2 to 2.3) > Upgrade Spark module to 2.4 > --- > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: Task > Components: spark >Affects Versions: 2.7.5 >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12231) RollbackRecord's must be flushed after logging to WAL
[
https://issues.apache.org/jira/browse/IGNITE-12231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938739#comment-16938739
]
Ignite TC Bot commented on IGNITE-12231:
{panel:title=Branch: [pull/6908/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4633828buildTypeId=IgniteTests24Java8_RunAll]
> RollbackRecord's must be flushed after logging to WAL
> -
>
> Key: IGNITE-12231
> URL: https://issues.apache.org/jira/browse/IGNITE-12231
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey N. Gura
>Assignee: Andrey N. Gura
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Every record or records batch logged to WAL must call {{wal().flush()}} in
> order to save data on storage device.
> {{TxPartitionCounterStateOnePrimaryTwoBackupsHistoryRebalanceTest.testPartialPrepare_2TX_1_1}}
> test fails periodically with disabled MMAP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938736#comment-16938736
]
Sergei Ryzhov commented on IGNITE-12220:
This change is ready. [~mstepachev], could you please review?
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12054) Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937715#comment-16937715 ] Alexey Zinoviev edited comment on IGNITE-12054 at 9/26/19 3:27 PM: --- I've started the R about 2.4 version, fixed a few minor changes and posted here a few thoughts about current problems here, in Jira comments. An ExternalCatalog was refactored here [https://github.com/apache/spark/commit/f38ea00e83099a5ae8d3afdec2e896e43c2db612] and all listener properties were inherited in ExternalCatalogWithListener. Refactoring for Hive is here [https://github.com/apache/spark/pull/21122/files] Nobody yet inherited from this class on Github, the known implementations are HiveExternalCatalog and MemoryExternalCatalog (both of them doesn't support listeners and events) Also, people in Spark ML couldn't solve the same problem [http://mail-archives.apache.org/mod_mbox/spark-issues/201812.mbox/%3cjira.13144856.1520975543000.147283.1544598241...@atlassian.jira%3E] Also this question is considered in paper [https://www.waitingforcode.com/apache-spark-sql/writing-custom-external-catalog-listeners-apache-spark-sql/read] Also the semantic of Having support was changed in Spark [https://github.com/apache/spark/pull/22696/files] https://issues.apache.org/jira/browse/SPARK-25708 Now Having could be a legal operation without GroupBY was (Author: zaleslaw): I've started the R about 2.4 version, fixed a few minor changes and posted here a few thoughts about current problems here, in Jira comments. An ExternalCatalog was refactored here [https://github.com/apache/spark/commit/f38ea00e83099a5ae8d3afdec2e896e43c2db612] and all listener properties were inherited in ExternalCatalogWithListener. Nobody yet inherited from this class on Github, the known implementations are HiveExternalCatalog and MemoryExternalCatalog (both of them doesn't support listeners and events) Also, people in Spark ML couldn't solve the same problem [http://mail-archives.apache.org/mod_mbox/spark-issues/201812.mbox/%3cjira.13144856.1520975543000.147283.1544598241...@atlassian.jira%3E] Also this question is considered in paper [https://www.waitingforcode.com/apache-spark-sql/writing-custom-external-catalog-listeners-apache-spark-sql/read] Also the semantic of Having support was changed in Spark [https://github.com/apache/spark/pull/22696/files] https://issues.apache.org/jira/browse/SPARK-25708 Now Having could be a legal operation without GroupBY > Upgrade Spark module to 2.4 > --- > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: Task > Components: spark >Affects Versions: 2.7.5 >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12054) Upgrade Spark module to 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937715#comment-16937715 ] Alexey Zinoviev edited comment on IGNITE-12054 at 9/26/19 3:25 PM: --- I've started the R about 2.4 version, fixed a few minor changes and posted here a few thoughts about current problems here, in Jira comments. An ExternalCatalog was refactored here [https://github.com/apache/spark/commit/f38ea00e83099a5ae8d3afdec2e896e43c2db612] and all listener properties were inherited in ExternalCatalogWithListener. Nobody yet inherited from this class on Github, the known implementations are HiveExternalCatalog and MemoryExternalCatalog (both of them doesn't support listeners and events) Also, people in Spark ML couldn't solve the same problem [http://mail-archives.apache.org/mod_mbox/spark-issues/201812.mbox/%3cjira.13144856.1520975543000.147283.1544598241...@atlassian.jira%3E] Also this question is considered in paper [https://www.waitingforcode.com/apache-spark-sql/writing-custom-external-catalog-listeners-apache-spark-sql/read] Also the semantic of Having support was changed in Spark [https://github.com/apache/spark/pull/22696/files] https://issues.apache.org/jira/browse/SPARK-25708 Now Having could be a legal operation without GroupBY was (Author: zaleslaw): I've started the R about 2.4 version, fixed a few minor changes and posted here a few thoughts about current problems here, in Jira comments. An ExternalCatalog was refactored here [https://github.com/apache/spark/commit/f38ea00e83099a5ae8d3afdec2e896e43c2db612] and all listener properties were inherited in ExternalCatalogWithListener. Nobody yet inherited from this class on Github, the known implementations are HiveExternalCatalog and MemoryExternalCatalog (both of them doesn't support listeners and events) Also, people in Spark ML couldn't solve the same problem [http://mail-archives.apache.org/mod_mbox/spark-issues/201812.mbox/%3cjira.13144856.1520975543000.147283.1544598241...@atlassian.jira%3E] Also this question is considered in paper [https://www.waitingforcode.com/apache-spark-sql/writing-custom-external-catalog-listeners-apache-spark-sql/read] > Upgrade Spark module to 2.4 > --- > > Key: IGNITE-12054 > URL: https://issues.apache.org/jira/browse/IGNITE-12054 > Project: Ignite > Issue Type: Task > Components: spark >Affects Versions: 2.7.5 >Reporter: Denis A. Magda >Assignee: Alexey Zinoviev >Priority: Blocker > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Users can't use APIs that are already available in Spark 2.4: > https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite > Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as > a separate module that can support multiple Spark versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12233) Merge benchmarks sql index benchmarks from ignite-2.7 to master
[ https://issues.apache.org/jira/browse/IGNITE-12233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938687#comment-16938687 ] Ilya Suntsov commented on IGNITE-12233: --- https://ci.ignite.apache.org/viewLog.html?buildId=4637826=IgniteTests24Java8_RunBasicTests > Merge benchmarks sql index benchmarks from ignite-2.7 to master > --- > > Key: IGNITE-12233 > URL: https://issues.apache.org/jira/browse/IGNITE-12233 > Project: Ignite > Issue Type: Task > Components: yardstick >Reporter: Ilya Suntsov >Assignee: Ilya Suntsov >Priority: Major > > The following benchmarks are missed in master but exist in ignite-2.7: > * IgniteSqlUpdateFilteredBenchmark > * IgniteSqlInsertIndexedValue1Benchmark > * IgniteSqlInsertIndexedValue2Benchmark > * IgniteSqlInsertIndexedValue8Benchmark -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11008) JDBC Metadata: redundant spaces IS_GENERATEDCOLUMN & BUFFER_LENGTH
[
https://issues.apache.org/jira/browse/IGNITE-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938680#comment-16938680
]
Ignite TC Bot commented on IGNITE-11008:
{panel:title=Branch: [pull/6885/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4634670buildTypeId=IgniteTests24Java8_RunAll]
> JDBC Metadata: redundant spaces IS_GENERATEDCOLUMN & BUFFER_LENGTH
> --
>
> Key: IGNITE-11008
> URL: https://issues.apache.org/jira/browse/IGNITE-11008
> Project: Ignite
> Issue Type: Bug
> Components: jdbc
>Affects Versions: 2.7
>Reporter: Stepan Pilschikov
>Assignee: kcheng.mvp
>Priority: Minor
> Fix For: 2.8
>
>
> Found redundant spaces in
> org.apache.ignite.internal.jdbc.thin.JdbcThinDatabaseMetadata#getColumns
> "IS_GENERATEDCOLUMN "
> "BUFFER_LENGTH "
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11992) Improvements for new security approach
[ https://issues.apache.org/jira/browse/IGNITE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stepachev Maksim updated IGNITE-11992: -- Description: 1. The visor tasks lost permission. The method VisorQueryUtils#scheduleQueryStart makes a new thread and loses context. 3. The GridRestProcessor does tasks outside "withContext" section. As result context loses. 4. The GridRestProcessor isn't client, we can't read security subject from node attribute. We should transmit secCtx for fake nodes and secSubjId for real. was: 1. ZookeaperDiscoveryImpl doesn't implement security into itself. As a result: Caused by: class org.apache.ignite.spi.IgniteSpiException: Security context isn't certain. 2. The visor tasks lost permission. The method VisorQueryUtils#scheduleQueryStart makes a new thread and loses context. 3. The GridRestProcessor does tasks outside "withContext" section. As result context loses. 4. The GridRestProcessor isn't client, we can't read security subject from node attribute. We should transmit secCtx for fake nodes and secSubjId for real. 5. NoOpIgniteSecurityProcessor should include a disabled processor and validate it too if it is not null. It is important for a client node. For example: Into IgniteKernal#securityProcessor method createComponent return a GridSecurityProcessor. For server nodes are enabled, but for clients aren't. The clients aren't able to pass validation for this reason. 6. ATTR_SECURITY_SUBJECT was removed. It broke compatibility. > Improvements for new security approach > -- > > Key: IGNITE-11992 > URL: https://issues.apache.org/jira/browse/IGNITE-11992 > Project: Ignite > Issue Type: Improvement > Components: security >Affects Versions: 2.8 >Reporter: Stepachev Maksim >Assignee: Stepachev Maksim >Priority: Major > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > 1. The visor tasks lost permission. > The method VisorQueryUtils#scheduleQueryStart makes a new thread and loses > context. > 3. The GridRestProcessor does tasks outside "withContext" section. As result > context loses. > 4. The GridRestProcessor isn't client, we can't read security subject from > node attribute. > We should transmit secCtx for fake nodes and secSubjId for real. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12235) If for some reason Index gets corrupted then that index should be re-build without shutting down node
[ https://issues.apache.org/jira/browse/IGNITE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shivakumar updated IGNITE-12235: Description: As of now for some reason, if the Index tree gets corrupted then the node which finds the index corruption will be terminated. It will be good to handle this and re-build the index if it gets corrupted. Ex: during node restarts, if it finds any corrupted index then it will not start the node, it would be good if it deletes the corrupted index and re-build it again during re-start. was: As of now for some reason, if the Index tree gets corrupted then the node which finds the index corruption will be terminated. It will be good to handle this and re-build the index if it gets corrupted. > If for some reason Index gets corrupted then that index should be re-build > without shutting down node > -- > > Key: IGNITE-12235 > URL: https://issues.apache.org/jira/browse/IGNITE-12235 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.7 >Reporter: shivakumar >Priority: Major > > As of now for some reason, if the Index tree gets corrupted then the node > which finds the index corruption will be terminated. > It will be good to handle this and re-build the index if it gets corrupted. > Ex: > during node restarts, if it finds any corrupted index then it will not start > the node, it would be good if it deletes the corrupted index and re-build it > again during re-start. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12108) [IEP-35] Migrate Communication Metrics.
[
https://issues.apache.org/jira/browse/IGNITE-12108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938572#comment-16938572
]
Ignite TC Bot commented on IGNITE-12108:
{panel:title=Branch: [pull/6814/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4634554buildTypeId=IgniteTests24Java8_RunAll]
> [IEP-35] Migrate Communication Metrics.
> ---
>
> Key: IGNITE-12108
> URL: https://issues.apache.org/jira/browse/IGNITE-12108
> Project: Ignite
> Issue Type: New Feature
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> ||*Name*||*Description*||
> |communication.tcp.outboundMessagesQueueSize|Number of messages waiting to be
> sent|
> |communication.tcp.sentBytes|Total number of bytes received by current node|
> |communication.tcp.receivedBytes|Total number of bytes sent by current node|
> |communication.tcp.sentMessagesCount|Total number of messages sent by current
> node|
> |communication.tcp.receivedMessagesCount|Total number of messages received by
> current node|
> |communication.tcp.sentMessagesByType.|Total number of messages
> with given type sent by current node|
> |communication.tcp.receivedMessagesByType.|Total number of
> messages with given type received by current node|
> |communication.tcp..sentMessagesToNode|Total number of messages sent
> by current node to the given node|
> |communication.tcp..receivedMessagesFromNode|Total number of messages
> received by current node from the given node|
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11094) Add SSL support for ignite zookeeper SPI
[
https://issues.apache.org/jira/browse/IGNITE-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938527#comment-16938527
]
Ignite TC Bot commented on IGNITE-11094:
{panel:title=Branch: [pull/6910/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4634438buildTypeId=IgniteTests24Java8_RunAll]
> Add SSL support for ignite zookeeper SPI
>
>
> Key: IGNITE-11094
> URL: https://issues.apache.org/jira/browse/IGNITE-11094
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.7
>Reporter: Sergey Kozlov
>Assignee: Ryabov Dmitrii
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> ZK 3.5.4-beta is already supporting SSL [1]. We should add SSL support to ZK
> server connections if Zookeeper SPI used.
> 1.
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12235) If for some reason Index gets corrupted then that index should be re-build without shutting down node
shivakumar created IGNITE-12235: --- Summary: If for some reason Index gets corrupted then that index should be re-build without shutting down node Key: IGNITE-12235 URL: https://issues.apache.org/jira/browse/IGNITE-12235 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.7 Reporter: shivakumar As of now for some reason, if the Index tree gets corrupted then the node which finds the index corruption will be terminated. I will be good to handle this and re-build the index if it gets corrupted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12235) If for some reason Index gets corrupted then that index should be re-build without shutting down node
[ https://issues.apache.org/jira/browse/IGNITE-12235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shivakumar updated IGNITE-12235: Description: As of now for some reason, if the Index tree gets corrupted then the node which finds the index corruption will be terminated. It will be good to handle this and re-build the index if it gets corrupted. was: As of now for some reason, if the Index tree gets corrupted then the node which finds the index corruption will be terminated. I will be good to handle this and re-build the index if it gets corrupted. > If for some reason Index gets corrupted then that index should be re-build > without shutting down node > -- > > Key: IGNITE-12235 > URL: https://issues.apache.org/jira/browse/IGNITE-12235 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.7 >Reporter: shivakumar >Priority: Major > > As of now for some reason, if the Index tree gets corrupted then the node > which finds the index corruption will be terminated. > It will be good to handle this and re-build the index if it gets corrupted. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler
[
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938503#comment-16938503
]
Andrey N. Gura commented on IGNITE-12198:
-
[~scottmf], you didn't missing anything. This aspect should be described in the
docs.
[~Artem Budnikov] , could you please take a look? I can't find any information
about segmentation policies in the docs at all. Also "Critical Failures
Handling" page should have note about exceptional cases.
> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> stopNodeHnd);
> break; {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11008) JDBC Metadata: redundant spaces IS_GENERATEDCOLUMN & BUFFER_LENGTH
[ https://issues.apache.org/jira/browse/IGNITE-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938500#comment-16938500 ] kcheng.mvp commented on IGNITE-11008: - [~mmuzaf] Can you help me to review the PR ? > JDBC Metadata: redundant spaces IS_GENERATEDCOLUMN & BUFFER_LENGTH > -- > > Key: IGNITE-11008 > URL: https://issues.apache.org/jira/browse/IGNITE-11008 > Project: Ignite > Issue Type: Bug > Components: jdbc >Affects Versions: 2.7 >Reporter: Stepan Pilschikov >Assignee: kcheng.mvp >Priority: Minor > Fix For: 2.8 > > > Found redundant spaces in > org.apache.ignite.internal.jdbc.thin.JdbcThinDatabaseMetadata#getColumns > "IS_GENERATEDCOLUMN " > "BUFFER_LENGTH " -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergei Ryzhov updated IGNITE-12220:
---
Comment: was deleted
(was: edit SecurityPermissionSetBuilder#appendCachePermissions,
SecurityPermissionSetBuilder#appendSystemPermissions and editing tests)
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12201) distributed sql join not working as mentioned in documentation
[
https://issues.apache.org/jira/browse/IGNITE-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938482#comment-16938482
]
shivakumar commented on IGNITE-12201:
-
Hi [~Pavlukhin]
Ignite doc says non-collocated distributed join / cross join is supported by
Ignite.
But with my findings If N tables are involved in cross join then at least (N-1)
tables should be in replicated mode.
I tried creating index on each column of the table but still, I am getting the
same exception with *distributedJoins=true*
> distributed sql join not working as mentioned in documentation
> --
>
> Key: IGNITE-12201
> URL: https://issues.apache.org/jira/browse/IGNITE-12201
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.7
> Environment: Kubernetes on RHEL 7.6
>Reporter: shivakumar
>Priority: Major
> Attachments: distributed_sql_error.txt
>
>
> I am trying to do a simple cross join on two tables with non-collocated data
> (without affinity key),
> This non-collocated distributed join always fails with the error message:
>
> *"java.sql.SQLException: javax.cache.CacheException: Failed to prepare
> distributed join query: join condition does not use index "*
>
> If I create one of the tables in replicated mode and another one in
> partitioned mode this Join operation works but documentation mentions that
> Ignite supports non-collocated joins without any condition.
> And we tried with 3 tables and 1 in replicated and other 2 in partitioned
> then we observed that it failed.
> we are running the Join operations with *distributedJoins=true.*
> *We observed that if there are N tables in Join operation then (N-1) should
> be in replicated mode, is our understanding right?*
> *If our understanding is correct then to do Join operation the dimensioning
> of cluster increases by many folds which can't be used in a production
> environment.*
> *To reproduce:*
> *Ignite with 4 node cluster with native persistence enabled.*
> *create the following tables*
> {quote} {{CREATE TABLE City (}}{quote}
> {quote} {{ id LONG PRIMARY KEY, name VARCHAR)}}{quote}
> {quote} {{ WITH "backup=1";}}{quote}
> {quote} {{}}{quote}
> {quote} {{CREATE TABLE Person (}}
> {{ id LONG, name VARCHAR, city_id LONG, PRIMARY KEY (id, city_id))}}
> {{ WITH "backups=1";}}
> {{}}
> {{CREATE INDEX idx_city_name ON City (name);}}
> {{}}
> {{CREATE INDEX idx_person_name ON Person (name);}}
>
> {{INSERT INTO City (id, name) VALUES (1, 'Forest Hill');}}
> {{INSERT INTO City (id, name) VALUES (2, 'Denver');}}
> {{INSERT INTO City (id, name) VALUES (3, 'St. Petersburg');}}
> {{}}
> {{INSERT INTO Person (id, name, city_id) VALUES (1, 'John Doe', 3);}}
> {{INSERT INTO Person (id, name, city_id) VALUES (2, 'Jane Roe', 2);}}
> {{INSERT INTO Person (id, name, city_id) VALUES (3, 'Mary Major', 1);}}
> {{INSERT INTO Person (id, name, city_id) VALUES (4, 'Richard Miles', 2);}} {{
> }}{quote}
> Query to be run:
> {quote}select * from City c, Person p;{color:#66}
> {color}{quote}
> {quote}or
> {color:#80}*SELECT*{color} * *{color:#80}FROM{color}* City
> *{color:#80}AS{color}* c *{color:#80}CROSS{color}*
> *{color:#80}join{color}* Person *{color:#80}AS{color}* p;{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938426#comment-16938426
]
Ignite TC Bot commented on IGNITE-12220:
{panel:title=Branch: [pull/6904/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4637637buildTypeId=IgniteTests24Java8_RunAll]
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938425#comment-16938425
]
Ignite TC Bot commented on IGNITE-12220:
{panel:title=Branch: [pull/6904/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4637637buildTypeId=IgniteTests24Java8_RunAll]
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938419#comment-16938419
]
Ignite TC Bot commented on IGNITE-12220:
{panel:title=Branch: [pull/6904/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4637637buildTypeId=IgniteTests24Java8_RunAll]
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12220) Allow to use cache-related permissions both at system and per-cache levels
[
https://issues.apache.org/jira/browse/IGNITE-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938418#comment-16938418
]
Ignite TC Bot commented on IGNITE-12220:
{panel:title=Branch: [pull/6904/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4637637buildTypeId=IgniteTests24Java8_RunAll]
> Allow to use cache-related permissions both at system and per-cache levels
> --
>
> Key: IGNITE-12220
> URL: https://issues.apache.org/jira/browse/IGNITE-12220
> Project: Ignite
> Issue Type: Task
> Components: security
>Affects Versions: 2.7.6
>Reporter: Andrey Kuznetsov
>Assignee: Sergei Ryzhov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, {{CACHE_CREATE}} and {{CACHE_DESTROY}} permissions are enforced to
> be system-level permissions, see for instance
> {{SecurityPermissionSetBuilder#appendCachePermissions}}. This looks
> inflexible: Ignite Security implementations are not able to manage cache
> creation and deletion permissions on per-cache basis (unlike get/put/remove
> permissions). All such limitations should be found and removed in order to
> allow all {{CACHE_*}} permissions to be set both at system and per-cache
> levels.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-9638) .NET: JVM keeps track of CLR Threads, even when they are finished
[ https://issues.apache.org/jira/browse/IGNITE-9638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938372#comment-16938372 ] Pavel Tupitsyn commented on IGNITE-9638: Merged to master: 738cb67fa332239fa1fc6f31daafbc4410abb58e > .NET: JVM keeps track of CLR Threads, even when they are finished > -- > > Key: IGNITE-9638 > URL: https://issues.apache.org/jira/browse/IGNITE-9638 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.6 >Reporter: Ilya Kasnacheev >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET > Fix For: 2.8 > > Attachments: IgniteRepro.zip > > Time Spent: 40m > Remaining Estimate: 0h > > When you create a Thread in C#, JVM creates corresponding thread > "Thread-" which is visible in jstack. When C# joins this thread, it is > not removed from JVM and is kept around. This means that jstack may show > thousands of threads which are not there. Reproducer is attached. It is > presumed that memory will be exhausted eventually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9638) .NET: JVM keeps track of CLR Threads, even when they are finished
[ https://issues.apache.org/jira/browse/IGNITE-9638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-9638: --- Ignite Flags: Release Notes Required > .NET: JVM keeps track of CLR Threads, even when they are finished > -- > > Key: IGNITE-9638 > URL: https://issues.apache.org/jira/browse/IGNITE-9638 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.6 >Reporter: Ilya Kasnacheev >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET > Fix For: 2.8 > > Attachments: IgniteRepro.zip > > Time Spent: 40m > Remaining Estimate: 0h > > When you create a Thread in C#, JVM creates corresponding thread > "Thread-" which is visible in jstack. When C# joins this thread, it is > not removed from JVM and is kept around. This means that jstack may show > thousands of threads which are not there. Reproducer is attached. It is > presumed that memory will be exhausted eventually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (IGNITE-12234) .NET: Fix flaky tests
[ https://issues.apache.org/jira/browse/IGNITE-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn resolved IGNITE-12234. - Resolution: Fixed > .NET: Fix flaky tests > - > > Key: IGNITE-12234 > URL: https://issues.apache.org/jira/browse/IGNITE-12234 > Project: Ignite > Issue Type: Improvement > Components: platforms >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET > Time Spent: 20m > Remaining Estimate: 0h > > Known flaky tests: > * CacheTestSsl.TestAsyncCompletionOrder > * DataStorageMetricsTest.TestDataStorageMetrics > * PersistenceTest.TestBaselineTopology > * CancellationTest.TestTask -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12234) .NET: Fix flaky tests
[ https://issues.apache.org/jira/browse/IGNITE-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-12234: Ignite Flags: (was: Docs Required,Release Notes Required) > .NET: Fix flaky tests > - > > Key: IGNITE-12234 > URL: https://issues.apache.org/jira/browse/IGNITE-12234 > Project: Ignite > Issue Type: Improvement > Components: platforms >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Known flaky tests: > * CacheTestSsl.TestAsyncCompletionOrder > * DataStorageMetricsTest.TestDataStorageMetrics > * PersistenceTest.TestBaselineTopology > * CancellationTest.TestTask -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12234) .NET: Fix flaky tests
[ https://issues.apache.org/jira/browse/IGNITE-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-12234: Fix Version/s: 2.8 > .NET: Fix flaky tests > - > > Key: IGNITE-12234 > URL: https://issues.apache.org/jira/browse/IGNITE-12234 > Project: Ignite > Issue Type: Improvement > Components: platforms >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Known flaky tests: > * CacheTestSsl.TestAsyncCompletionOrder > * DataStorageMetricsTest.TestDataStorageMetrics > * PersistenceTest.TestBaselineTopology > * CancellationTest.TestTask -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12234) .NET: Fix flaky tests
[ https://issues.apache.org/jira/browse/IGNITE-12234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938367#comment-16938367 ] Pavel Tupitsyn commented on IGNITE-12234: - Merged to master: c5562e893693c985e5e488eed67a60c9191545c6 > .NET: Fix flaky tests > - > > Key: IGNITE-12234 > URL: https://issues.apache.org/jira/browse/IGNITE-12234 > Project: Ignite > Issue Type: Improvement > Components: platforms >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET > Time Spent: 20m > Remaining Estimate: 0h > > Known flaky tests: > * CacheTestSsl.TestAsyncCompletionOrder > * DataStorageMetricsTest.TestDataStorageMetrics > * PersistenceTest.TestBaselineTopology > * CancellationTest.TestTask -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10113) Failure during build index isn't handled by failure handler
[
https://issues.apache.org/jira/browse/IGNITE-10113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938313#comment-16938313
]
Ignite TC Bot commented on IGNITE-10113:
{panel:title=Branch: [pull/6801/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4601030buildTypeId=IgniteTests24Java8_RunAll]
> Failure during build index isn't handled by failure handler
> ---
>
> Key: IGNITE-10113
> URL: https://issues.apache.org/jira/browse/IGNITE-10113
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.6
>Reporter: Sergey Antonov
>Assignee: Ivan Bessonov
>Priority: Major
> Fix For: 2.8
>
> Attachments: StopNodeOnRebuildIndexFailureTest.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If exception will be thrown during index build, failure handler doesn't
> handle this exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
