[jira] [Commented] (IGNITE-9408) Update mesos version
[ https://issues.apache.org/jira/browse/IGNITE-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596987#comment-16596987 ] Roman Shtykh commented on IGNITE-9408: -- Updated to and tested with v 1.5.0. > Update mesos version > > > Key: IGNITE-9408 > URL: https://issues.apache.org/jira/browse/IGNITE-9408 > Project: Ignite > Issue Type: Task >Reporter: Roman Shtykh >Assignee: Roman Shtykh >Priority: Major > Fix For: 2.7 > > > 0.22.0 is being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9408) Update mesos version
[ https://issues.apache.org/jira/browse/IGNITE-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Shtykh updated IGNITE-9408: - Fix Version/s: 2.7 > Update mesos version > > > Key: IGNITE-9408 > URL: https://issues.apache.org/jira/browse/IGNITE-9408 > Project: Ignite > Issue Type: Task >Reporter: Roman Shtykh >Assignee: Roman Shtykh >Priority: Major > Fix For: 2.7 > > > 0.22.0 is being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9408) Update mesos version
[ https://issues.apache.org/jira/browse/IGNITE-9408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596986#comment-16596986 ] ASF GitHub Bot commented on IGNITE-9408: GitHub user shroman opened a pull request: https://github.com/apache/ignite/pull/4650 IGNITE-9408: Update Apache Mesos version. You can merge this pull request into a Git repository by running: $ git pull https://github.com/shroman/ignite IGNITE-9408 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4650.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4650 commit 51265ac0f058b5f6d292064ad2808697ed6d82cc Author: shroman Date: 2018-08-30T01:57:04Z IGNITE-9408: Update Apache Mesos version. > Update mesos version > > > Key: IGNITE-9408 > URL: https://issues.apache.org/jira/browse/IGNITE-9408 > Project: Ignite > Issue Type: Task >Reporter: Roman Shtykh >Assignee: Roman Shtykh >Priority: Major > > 0.22.0 is being used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9305) Wrong off-heap size is reported for a node
[
https://issues.apache.org/jira/browse/IGNITE-9305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596958#comment-16596958
]
Denis Magda commented on IGNITE-9305:
-
[~xtern], [~NSAmelchev],
Let's use a little bit modified format suggested by Nikita.
{noformat}
^-- H/N/C [hosts=1, nodes=2, CPUs=8]
^-- CPU [cur=15.73%, avg=0%, GC=0%]
^-- PageMemory [pages=25]
^-- Heap [used=71MB, free=97.99%, comm=203MB]
^-- Off-heap [used=0MB, free=99.96%, comm=220MB]:
^-- sysMemPlc region [used=0MB, free=99.98%, comm=100MB]
^-- default region [used=0MB, free=99.8%, comm=20MB]
^-- metastoreMemPlc region [used=0MB, free=99.96%, comm=100MB]
^-- Ignite persistence [used=0MB]
^-- defaul region [used=XMB]
^-- {Name} region [used=XMB]
^-- {Name} region [used=XMB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=6, qSize=0]
^-- System thread pool [active=0, idle=6, qSize=0]
{noformat}
We show the total for off-heap and persistence and also do a breakdown per
region.
> Wrong off-heap size is reported for a node
> --
>
> Key: IGNITE-9305
> URL: https://issues.apache.org/jira/browse/IGNITE-9305
> Project: Ignite
> Issue Type: Task
>Affects Versions: 2.6
>Reporter: Denis Magda
>Assignee: Pavel Pereslegin
>Priority: Blocker
> Fix For: 2.7
>
>
> Was troubleshooting an Ignite deployment today and couldn't find out from the
> logs what was the actual off-heap space used.
> Those were the given memory resoures (Ignite 2.6):
> {code}
> [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
> snapshot [ver=1, servers=1, clients=0, CPUs=64, offheap=30.0GB, heap=24.0GB]
> {code}
> And that weird stuff was reported by the node (pay attention to the last
> line):
> {code}
> [2018-08-16 15:45:50,211][INFO
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
>
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=c033026e, name=cluster_31-Dec-2017, uptime=00:38:00.257]
> ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> ^-- PageMemory [pages=6997377]
> ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is always the
> same!
> {code}
> Had to change the code by using
> {code}dataRegion.getPhysicalMemoryPages(){code} to find out that actual
> off-heap usage size was
> {code}
> >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> {code}
> The logs have to report the following instead:
> {code}
> ^-- Off-heap {Data Region 1} [used={dataRegion1.getPhysicalMemorySize()},
> free=X%, comm=dataRegion1.maxSize()]
> ^-- Off-heap {Data Region 2} [used={dataRegion2.getPhysicalMemorySize()},
> free=X%, comm=dataRegion2.maxSize()]
> {code}
> If Ignite persistence is enabled then the following extra lines have to be
> added to see the disk used space:
> {code}
> ^-- Ignite persistence {Data Region 1}:
> used={dataRegion1.getTotalAllocatedSize() -
> dataRegion1.getPhysicalMemorySize()}
> ^-- Ignite persistence {Data Region 2}
> [used={dataRegion2.getTotalAllocatedSize() -
> dataRegion2.getPhysicalMemorySize()}]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9285) [ML] Add MaxAbsScaler as a preprocessing stage
[ https://issues.apache.org/jira/browse/IGNITE-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravil Galeyev reassigned IGNITE-9285: - Assignee: Ravil Galeyev > [ML] Add MaxAbsScaler as a preprocessing stage > -- > > Key: IGNITE-9285 > URL: https://issues.apache.org/jira/browse/IGNITE-9285 > Project: Ignite > Issue Type: Sub-task > Components: ml >Reporter: Aleksey Zinoviev >Assignee: Ravil Galeyev >Priority: Major > > Add analogue of > [http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.html#sklearn.preprocessing.MaxAbsScaler] > Please look at the MinMaxScaler or Normalization packages in preprocessing > package. > Add classes if required > 1) Preprocessor > 2) Trainer > 3) custom PartitionData if shuffling is a step of algorithm > > Requirements for successful PR: > # PartitionedDataset usage > # Trainer-Model paradigm support > # Tests for Model and for Trainer (and other stuff) > # Example of usage with small, but famous dataset like IRIS, Titanic or > House Prices > # Javadocs/codestyle according guidelines -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-7783) Thin Client lib: PHP
[ https://issues.apache.org/jira/browse/IGNITE-7783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596765#comment-16596765 ] ASF GitHub Bot commented on IGNITE-7783: GitHub user ekaterina-nbl opened a pull request: https://github.com/apache/ignite/pull/4649 IGNITE-7783 PHP Thin Client You can merge this pull request into a Git repository by running: $ git pull https://github.com/nobitlost/ignite ignite-7783 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4649.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4649 commit 4b444dff62c04b97bc2cdb0676ad52880374 Author: ekaterina-nbl Date: 2018-03-20T21:12:21Z initial implementation commit d1e8014c80f698028d23e09cafd7ea190ac3e929 Author: ekaterina-nbl Date: 2018-03-20T21:32:52Z initial implementation commit f79229e233ffa7bff1e7c22f04749924af6d712a Author: ekaterina-nbl Date: 2018-03-22T09:39:32Z initial implementation commit 658d60b58840080b664e02815f4ba6aac76e5804 Author: ekaterina-nbl Date: 2018-03-22T16:52:18Z minor update commit 617cef12ad72791c7e7e67179e1b0c3f7f3ae6bb Author: ekaterina-nbl Date: 2018-03-25T13:33:27Z api spec commit d9729585f5a901977e2a2e40e86a2b715fab79fa Author: alexey-nbl Date: 2018-03-25T21:27:11Z link to api_spec added commit ea847eba62e556fa81cbf9838ffe17af60f464e4 Author: ekaterina-nbl Date: 2018-03-31T22:07:50Z error types modified commit c2ad53fe625cc3a05155eeef318421538830 Author: ekaterina-nbl Date: 2018-03-31T23:41:56Z client states commit 6d2233864b4d891361d38a7143846570bd6c0ef6 Author: ekaterina-nbl Date: 2018-04-01T13:11:27Z object types improvement commit 52fbc5f87143da068596141cb59b17b684fd2c1f Author: ekaterina-nbl Date: 2018-04-02T16:59:52Z complex objects commit cae5a28e7e0d610434debcc140738e2f48d061cf Author: ekaterina-nbl Date: 2018-04-02T20:13:00Z object types improvement commit 3165405d4eae09519dc9b5f40f162eb74a9b3c5d Author: ekaterina-nbl Date: 2018-04-02T21:21:26Z client states commit fdbb8f86b32fe4c038d38620a80921be3038f31f Author: alexey-nbl Date: 2018-04-03T08:26:04Z Ignite client states described commit 04b946885db0ea2f6fe56a75e28302641dad5f61 Author: alexey-nbl Date: 2018-04-03T09:35:49Z minor update commit f4aaf1c3f23c82ba7974b4c571a9984748e5e82e Author: alexey-nbl Date: 2018-04-03T12:47:54Z Update ObjectType.js commit e4c8279f4e83b3ed13383420ab3d1417b090a3fa Author: ekaterina-nbl Date: 2018-04-03T13:46:19Z minor updates + api spec commit ed2e4ee830ca40a28dc31958665f52fab6a0bcdd Author: alexey-nbl Date: 2018-04-04T14:34:52Z Update ObjectType.js commit 7e1666eb5df82b622a028c0ea949fa21c79e66c2 Author: alexey-nbl Date: 2018-04-04T15:14:00Z Update ObjectType.js commit 7b0270d65b597dc1b0d164e78f07c3a21c37ca67 Author: ekaterina-nbl Date: 2018-04-08T17:16:43Z sql queries, key value ops commit fe90f53fd08f77add17fbf06d014f9a9b0a11c65 Author: alexey-nbl Date: 2018-04-08T18:31:47Z Update IgniteClient.js commit 04137bf5ec3b7077e194edd0100a01bb43f7933a Author: alexey-nbl Date: 2018-04-08T18:37:04Z Update CacheConfiguration.js commit b63ad5980718da1b0c44fa4ca138c6e85e47aef3 Author: alexey-nbl Date: 2018-04-08T19:11:11Z Update ObjectType.js commit 756b908c9dc38ae497e4d7d740f836dabed93e48 Author: alexey-nbl Date: 2018-04-08T22:41:54Z Update CacheClient.js commit e96ffee17298dd25d26a7029738132478271cf92 Author: ekaterina-nbl Date: 2018-04-08T23:23:33Z object array, minor updates commit c050e671f74232c4efc41f51c2018d08b152cbbc Author: alexey-nbl Date: 2018-04-09T21:04:35Z Update CacheClient.js commit 25052a4e93d6fcc0b0c4789fdd5a1eb85413b5a2 Author: alexey-nbl Date: 2018-04-09T22:43:50Z Update ObjectType.js commit c516b4147c10398d4f34d78a8830dd0fcb5f28f4 Author: alexey-nbl Date: 2018-04-10T11:50:13Z Update CacheClient.js commit 0e9924a0df8a1d41718fb929698b8e4416f2efa2 Author: ekaterina-nbl Date: 2018-04-10T13:21:16Z cache key value operations tests commit e09ee0e7969bffd6f4cfd11579f6d0ff9b486c99 Author: ekaterina-nbl Date: 2018-04-10T13:21:50Z Merge branch 'master' of https://github.com/nobitlost/ignite commit 15d2abce41b4f1c9d4d5fecd44a99a5393177482 Author: alexey-nbl Date: 2018-04-10T14:38:57Z Update Query.js > Thin Client lib: PHP > > > Key: IGNITE-7783 > URL: https://issues.apache.org/jira/browse/IGNITE-7783 > Project: Ignite > Issue Type: New Feature > Components: thin client >Reporter: Alexey Kosenchuk >Assignee: ekaterina.vergizova >Priority: Major >
[jira] [Assigned] (IGNITE-9398) Reduce time on processing CustomDiscoveryMessage by discovery message worker
[
https://issues.apache.org/jira/browse/IGNITE-9398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Kovalenko reassigned IGNITE-9398:
---
Assignee: Pavel Kovalenko
> Reduce time on processing CustomDiscoveryMessage by discovery message worker
>
>
> Key: IGNITE-9398
> URL: https://issues.apache.org/jira/browse/IGNITE-9398
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.4, 2.5, 2.6
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Major
> Fix For: 2.7
>
>
> Processing discovery CustomMessage may take significant values of time
> (0.5-0.7 seconds) before sending to next node in the topology. This
> significantly accumulates the total time of PME if topology has multiple
> nodes.
> Let X = time of processing discovery message by discovery-msg-worker on each
> node before sending to next node.
> Let N = number of nodes in the topology.
> Then the minimal total time of exchange will be:
> T = N * X
> We shouldn't make heavy actions when process discovery message. Best solution
> will be separated thread that will do it, while discovery-msg-worker will
> just pass a message to that thread and send a message immediately to another
> node in topology.
> This affects both TcpDiscoverySpi and ZkDiscoverySpi.
> {noformat}
> [11:59:33,134][INFO][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Enqueued
> message type = TcpDiscoveryCustomEventMessage id =
> e4b542b6561-a38dfe31-dcfd-430b-acb3-5a531db4197e time = 0
> [11:59:33,537][INFO][tcp-disco-msg-worker-#2][GridSnapshotAwareClusterStateProcessorImpl]
> Received activate request with BaselineTopology[id=0]
> [11:59:33,549][INFO][tcp-disco-msg-worker-#2][GridSnapshotAwareClusterStateProcessorImpl]
> Started state transition: true
> [11:59:33,752][INFO][exchange-worker-#62][time] Started exchange init
> [topVer=AffinityTopologyVersion [topVer=110, minorTopVer=1], crd=true,
> evt=DISCOVERY_CUSTOM_EVT, evtNode=a38dfe31-dcfd-430b-acb3-5a531db4197e,
> customEvt=ChangeGlobalStateMessage
> [id=cea542b6561-47395de6-c204-4576-a0a3-99ec53d41ac3,
> reqId=5b651439-7a6a-43fc-9cb0-d646c3380576,
> initiatingNodeId=a38dfe31-dcfd-430b-acb3-5a531db4197e, activate=true,
> baselineTopology=BaselineTopology [id=0, branchingHash=-69412111965,
> branchingType='New BaselineTopology', baselineNodes=[node42, node43, node44,
> node45, node46, node47, node48, node49, node50, node51, node52, node53,
> node54, node55, node56, node57, node58, node59, node1, node4, node5, node2,
> node3, node8, node9, node6, node7, node60, node61, node62, node63, node64,
> node65, node66, node67, node68, node69, node70, node71, node72, node73,
> node74, node75, node76, node77, node78, node79, node80, node81, node82,
> node83, node84, node85, node86, node87, node88, node89, node90, node91,
> node92, node93, node94, node95, node96, node97, node10, node98, node11,
> node99, node12, node13, node14, node15, node16, node100, node17, node18,
> node19, node108, node107, node106, node105, node104, node103, node102,
> node101, node109, node20, node21, node22, node23, node24, node25, node26,
> node27, node28, node29, node110, node30, node31, node32, node33, node34,
> node35, node36, node37, node38, node39, node40, node41]],
> forceChangeBaselineTopology=false, timestamp=1535101173015], allowMerge=false]
> [11:59:33,753][INFO][exchange-worker-#62][GridDhtPartitionsExchangeFuture]
> Start activation process [nodeId=1906b9c3-73f4-4c30-85cc-cf6b99c3bab9,
> client=false, topVer=AffinityTopologyVersion [topVer=110, minorTopVer=1]]
> [11:59:33,756][INFO][exchange-worker-#62][FilePageStoreManager] Resolved page
> store work directory:
> /storage/ssd/avolkov/tiden/snapshots-180824-114937/test_pitr/ignite.server.1/work/db/node1
> [11:59:33,756][INFO][exchange-worker-#62][FileWriteAheadLogManager] Resolved
> write ahead log work directory:
> /storage/ssd/avolkov/tiden/snapshots-180824-114937/test_pitr/ignite.server.1/work/db/wal/node1
> [11:59:33,756][INFO][exchange-worker-#62][FileWriteAheadLogManager] Resolved
> write ahead log archive directory:
> /storage/ssd/avolkov/tiden/snapshots-180824-114937/test_pitr/ignite.server.1/work/db/wal/archive/node1
> [11:59:33,757][INFO][exchange-worker-#62][FileWriteAheadLogManager] Started
> write-ahead log manager [mode=LOG_ONLY]
> [11:59:33,763][INFO][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Processed
> message type = TcpDiscoveryCustomEventMessage id =
> e4b542b6561-a38dfe31-dcfd-430b-acb3-5a531db4197e time = 629
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-9426: --- Affects Version/s: 2.6 > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.6 >Reporter: Dmitriy Govorukhin >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure future performance improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-9426: --- Description: Need to create JMH and Yardstick benchmarks for the atomic sequence in order to be able to measure performance improvements. was: Need to create JMH and Yardstick benchmarks for the atomic sequence in order to be able to measure future performance improvements. > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.6 >Reporter: Dmitriy Govorukhin >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure performance improvements. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-9426: --- Description: Need to create JMH and Yardstick benchmarks for the atomic sequence in order to be able to measure future performance improvements. was:Need to create JMH and Yardstick benchmarks for the atomic sequence in order to be able to measure future performance improvements > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.6 >Reporter: Dmitriy Govorukhin >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure future performance improvements. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-9426: --- Ignite Flags: (was: Docs Required) > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Reporter: Dmitriy Govorukhin >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure future performance improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-9426: --- Fix Version/s: 2.7 > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Reporter: Dmitriy Govorukhin >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure future performance improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-9426: --- Issue Type: Improvement (was: Bug) > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Reporter: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure future performance improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9426) IgniteAtomicSequence benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin reassigned IGNITE-9426: -- Assignee: Dmitriy Govorukhin > IgniteAtomicSequence benchmarks > --- > > Key: IGNITE-9426 > URL: https://issues.apache.org/jira/browse/IGNITE-9426 > Project: Ignite > Issue Type: Improvement >Reporter: Dmitriy Govorukhin >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > > Need to create JMH and Yardstick benchmarks for the atomic sequence in order > to be able to measure future performance improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9426) IgniteAtomicSequence benchmarks
Dmitriy Govorukhin created IGNITE-9426: -- Summary: IgniteAtomicSequence benchmarks Key: IGNITE-9426 URL: https://issues.apache.org/jira/browse/IGNITE-9426 Project: Ignite Issue Type: Bug Reporter: Dmitriy Govorukhin Need to create JMH and Yardstick benchmarks for the atomic sequence in order to be able to measure future performance improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9341) Notify metastorage listeners right before start of discovery processor
[ https://issues.apache.org/jira/browse/IGNITE-9341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596606#comment-16596606 ] ASF GitHub Bot commented on IGNITE-9341: GitHub user sk0x50 opened a pull request: https://github.com/apache/ignite/pull/4648 IGNITE-9341 You can merge this pull request into a Git repository by running: $ git pull https://github.com/sk0x50/ignite ignite-9341 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4648.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4648 commit db188e822a485cbdc24929f88e304c140caa5bc6 Author: Slava Koptilin Date: 2018-08-29T17:00:16Z IGNITE-9341: test changes > Notify metastorage listeners right before start of discovery processor > -- > > Key: IGNITE-9341 > URL: https://issues.apache.org/jira/browse/IGNITE-9341 > Project: Ignite > Issue Type: Improvement > Components: general >Reporter: Ivan Rakov >Priority: Major > Fix For: 2.8 > > > onReadyForRead() is called only for inheritors of > MetastorageLifecycleListener interface which are started prior to > GridCacheProcessor. Listeners are notified at the moment of > ReadOnlyMetastorage initialization, which in turn occur during > GridCacheDatabaseSharedManager start. > We can split ReadOnlyMetastorage initialization and notification of listeners > - this will allow all components to subscribe for read-only metastorage ready > event. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9326) IgniteCacheFailedUpdateResponseTest hangs in master
[ https://issues.apache.org/jira/browse/IGNITE-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596604#comment-16596604 ] Ilya Lantukh commented on IGNITE-9326: -- Looks good. > IgniteCacheFailedUpdateResponseTest hangs in master > --- > > Key: IGNITE-9326 > URL: https://issues.apache.org/jira/browse/IGNITE-9326 > Project: Ignite > Issue Type: Test >Reporter: Alexey Goncharuk >Assignee: Alexey Goncharuk >Priority: Major > Labels: MakeTeamcityGreenAgain > Fix For: 2.7 > > > The test started to hang after IGNITE-8926 was merged. > The reason is that entry processor result started to be lazily serialized > during the message send, which results in a failure handler invocation. > However, the test checks that the exception is rethrown to a user. > The issue affects only ATOMIC caches. > One of the possible fixes is to marshal the result after the topology lock is > released. > Muting test in master for now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9273) IgnitePdsDynamicCacheTest.testRestartAndCreate is very flaky in master
[ https://issues.apache.org/jira/browse/IGNITE-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596598#comment-16596598 ] Alexey Goncharuk commented on IGNITE-9273: -- Test fails because logical records are not written for LOCAL caches. Adding logical records for LOCAL caches and adding more tests. > IgnitePdsDynamicCacheTest.testRestartAndCreate is very flaky in master > --- > > Key: IGNITE-9273 > URL: https://issues.apache.org/jira/browse/IGNITE-9273 > Project: Ignite > Issue Type: Bug >Reporter: Ilya Kasnacheev >Assignee: Alexey Goncharuk >Priority: Major > Labels: MakeTeamcityGreenAgain > > Such as: > https://ci.ignite.apache.org/viewLog.html?buildId=1655341=buildResultsDiv=IgniteTests24Java8_Pds1 > https://ci.ignite.apache.org/viewLog.html?buildId=1655880=buildResultsDiv=IgniteTests24Java8_Pds1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9326) IgniteCacheFailedUpdateResponseTest hangs in master
[
https://issues.apache.org/jira/browse/IGNITE-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596586#comment-16596586
]
Alexey Goncharuk commented on IGNITE-9326:
--
[~ilantukh], can you review the fix? I moved the entry processor result
serialization for ATOMIC caches outside topology read lock, but before leaving
the {{getAllAsyncInternal}} method so the exception is properly handled.
TC run is attached, MTCGA reports no new failures.
> IgniteCacheFailedUpdateResponseTest hangs in master
> ---
>
> Key: IGNITE-9326
> URL: https://issues.apache.org/jira/browse/IGNITE-9326
> Project: Ignite
> Issue Type: Test
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Major
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.7
>
>
> The test started to hang after IGNITE-8926 was merged.
> The reason is that entry processor result started to be lazily serialized
> during the message send, which results in a failure handler invocation.
> However, the test checks that the exception is rethrown to a user.
> The issue affects only ATOMIC caches.
> One of the possible fixes is to marshal the result after the topology lock is
> released.
> Muting test in master for now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
[
https://issues.apache.org/jira/browse/IGNITE-9423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko resolved IGNITE-9423.
Resolution: Duplicate
> unreliable test:
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions, training with batch size {1}]
> ---
>
> Key: IGNITE-9423
> URL: https://issues.apache.org/jira/browse/IGNITE-9423
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
>Priority: Major
> Fix For: 2.7
>
>
> [Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
> complains that
> {{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
> unreliable:
> {quote}*This test looks flaky:*
> Frequent test status changes: 39 changes out of 128 invocations
> Test status change in build without changes: from successful to failed
> [View test history
> »|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
> Output for most recent test failure: {noformat}
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase
> [Data divided on 4 partitions, training with batch size \{1}]
> java.lang.AssertionError: expected:<0.0> but was:<1.0>
> at
> org.apache.ignite.ml.svm.SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase
> (SVMMultiClassTrainerTest.java:53){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9425) NPE on index rebuild
[
https://issues.apache.org/jira/browse/IGNITE-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596500#comment-16596500
]
ASF GitHub Bot commented on IGNITE-9425:
GitHub user dkarachentsev opened a pull request:
https://github.com/apache/ignite/pull/4647
IGNITE-9425 - Fix NPE on index rebuild
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9425
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4647.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4647
commit d26fb535865765b48fe955b3d95c544bb1ae1885
Author: dkarachentsev
Date: 2018-08-29T15:43:36Z
IGNITE-9425 - Fix NPE on index rebuild
> NPE on index rebuild
>
>
> Key: IGNITE-9425
> URL: https://issues.apache.org/jira/browse/IGNITE-9425
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.6
>Reporter: Dmitry Karachentsev
>Assignee: Dmitry Karachentsev
>Priority: Major
> Fix For: 2.7
>
>
> Summary:
> This issue is seen when we have two caches mapped to two different regions
> (one with persistence and one without persistence)
> The client-side config should have the cache definitions
> The server-side config should have the data regions only
> Affinity should be defined on the cache without persistence
> We need to populate the data into both the caches and then restart the server
> and clients
> The issue will be seen when the client reconnects.
> Workaround:
> Add the cache configurations (for the caches without persistence) to the
> server-side config.
> Steps to reproduce:
> Ignite server
> - region1 (with persistence)
> - region2 (without persistence)
> client
> - cache1a from region1 – with custom affinity
> - cache2a fom region2 – with custom affinity
> 1. Populate data in both cache1a and cache2a.
> 2. Restart ignite server. It knows about cache1a from the persistent store.
> It doesn’t know about cache2a.
> 3. Restart client. When it connects to ignite, the server sees a nullpointer
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$11.apply(GridCacheDatabaseSharedManager.java:1243)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$11.apply(GridCacheDatabaseSharedManager.java:1239)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:353)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.rebuildIndexesIfNeeded(GridCacheDatabaseSharedManager.java:1239)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:1711)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:126)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:729)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2419)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2299)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9425) NPE on index rebuild
Dmitry Karachentsev created IGNITE-9425:
---
Summary: NPE on index rebuild
Key: IGNITE-9425
URL: https://issues.apache.org/jira/browse/IGNITE-9425
Project: Ignite
Issue Type: Bug
Affects Versions: 2.6
Reporter: Dmitry Karachentsev
Assignee: Dmitry Karachentsev
Fix For: 2.7
Summary:
This issue is seen when we have two caches mapped to two different regions (one
with persistence and one without persistence)
The client-side config should have the cache definitions
The server-side config should have the data regions only
Affinity should be defined on the cache without persistence
We need to populate the data into both the caches and then restart the server
and clients
The issue will be seen when the client reconnects.
Workaround:
Add the cache configurations (for the caches without persistence) to the
server-side config.
Steps to reproduce:
Ignite server
- region1 (with persistence)
- region2 (without persistence)
client
- cache1a from region1 – with custom affinity
- cache2a fom region2 – with custom affinity
1. Populate data in both cache1a and cache2a.
2. Restart ignite server. It knows about cache1a from the persistent store. It
doesn’t know about cache2a.
3. Restart client. When it connects to ignite, the server sees a nullpointer
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$11.apply(GridCacheDatabaseSharedManager.java:1243)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$11.apply(GridCacheDatabaseSharedManager.java:1239)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:353)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.rebuildIndexesIfNeeded(GridCacheDatabaseSharedManager.java:1239)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:1711)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:126)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:451)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:729)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2419)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2299)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-8886) Simultaneous using of BinaryWriter и BinaryRawWriter leads to BinaryObjectException and OOM
[
https://issues.apache.org/jira/browse/IGNITE-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596457#comment-16596457
]
Ilya Kasnacheev commented on IGNITE-8886:
-
[~ezhuravl] please review proposed fix.
> Simultaneous using of BinaryWriter и BinaryRawWriter leads to
> BinaryObjectException and OOM
> ---
>
> Key: IGNITE-8886
> URL: https://issues.apache.org/jira/browse/IGNITE-8886
> Project: Ignite
> Issue Type: Bug
> Components: binary
>Affects Versions: 2.5
>Reporter: Roman Guseinov
>Assignee: Ilya Kasnacheev
>Priority: Major
> Labels: binary
> Attachments: BinarylizableInvalidFlag.java, BinarylizableOOM.java
>
>
> When we use BinaryWriter and BinaryRawWriter simultaneously inside
> writeBinary method we can get the following exceptions in the process of
> deserializing objects:
> 1. class org.apache.ignite.binary.BinaryObjectException: Invalid flag value:
> 115
> {code:java}
> Exception in thread "main" javax.cache.CacheException: class
> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1302)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1732)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:910)
>at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:608)
>at
> org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag.main(BinarylizableInvalidFlag.java:33)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7322)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:171)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4563)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4537)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1350)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:907)
>... 2 more
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:909)
>at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>at
> org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:798)
>at
> org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:177)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:67)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:125)
>at
> org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1764)
>at
> org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1752)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.setResult(GridPartitionedSingleGetFuture.java:679)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.localGet(GridPartitionedSingleGetFuture.java:461)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.mapKeyToNode(GridPartitionedSingleGetFuture.java:342)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:216)
>at
>
[jira] [Assigned] (IGNITE-9411) Lock: make sure lock timeouts works fine
[ https://issues.apache.org/jira/browse/IGNITE-9411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Pavlukhin reassigned IGNITE-9411: -- Assignee: Ivan Pavlukhin > Lock: make sure lock timeouts works fine > > > Key: IGNITE-9411 > URL: https://issues.apache.org/jira/browse/IGNITE-9411 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Vladimir Ozerov >Assignee: Ivan Pavlukhin >Priority: Major > Fix For: 2.7 > > > In SQL it is not uncommon that locks are taken in arbitrary order, what may > lead to deadlocks. Fair deadlock detector is good solution in monolithic > databases - just analyze dependency graph and kill one of conflicting > transactions. > We have a ticket to implement distributed deadlock detector in Ignite [1]. > However, this solution is rather complex and may bring some overhead. > For now it is better to rely on some timeout (global or per-transaction), and > rollback TX when it fails to lock certain entry for a long time. Probably we > already have this feature in some form. Need to add verify that it works and > add more tests if needed. > [1] https://issues.apache.org/jira/browse/IGNITE-9322 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9145) [ML] Add different strategies to index labels in StringEncoderTrainer
[ https://issues.apache.org/jira/browse/IGNITE-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Zinoviev updated IGNITE-9145: - Fix Version/s: (was: 2.7) > [ML] Add different strategies to index labels in StringEncoderTrainer > - > > Key: IGNITE-9145 > URL: https://issues.apache.org/jira/browse/IGNITE-9145 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Aleksey Zinoviev >Assignee: Aleksey Zinoviev >Priority: Major > > The main idea to add a few strategies of indexing: sorting and so on. > Currently it supports only one strategy (most popular with zero and less > popular with the max index size). > There are can be a few options > * 'frequencyDesc': descending order by label frequency (most frequent label > assigned 0) > * 'frequencyAsc': ascending order by label frequency (least frequent label > assigned 0) > * 'alphabetDesc': descending alphabetical order > * 'alphabetAsc': ascending alphabetical order > > Please, update the method **transformFrequenciesToEncodingValues and add the > strategy as a parameter of trainer. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9404) Ignite start hangs infinitely when sync preloading is cancelled
[
https://issues.apache.org/jira/browse/IGNITE-9404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Pavlukhin updated IGNITE-9404:
---
Attachment: dump1.txt
> Ignite start hangs infinitely when sync preloading is cancelled
> ---
>
> Key: IGNITE-9404
> URL: https://issues.apache.org/jira/browse/IGNITE-9404
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.6
>Reporter: Ivan Pavlukhin
>Priority: Major
> Attachments: IgniteStartStopRace.java, dump1.txt,
> ignite-26252679.0.log
>
>
> This case fires very rarely in {{TcpDiscoveryRestartTest.testRestart}}.
> Caches with {{SYNC}} rebalance mode are affected (system internal cache uses
> such mode). When _starting_ instance is trying to preload partitions for such
> cache from another instance which _stops_ around the same time, first
> instance could hang infinitely. When {{SYNC}} rebalance mode is enabled
> starting instance awaits on _preload future_ in start routine. In another
> thread it starts fetching partitions and receives an error from _stopping_
> instance and cancels _rebalance future_ but _preload future_ is not cancelled.
> It is a bit tricky to reproduce case fairly. It is possible to reproduce the
> case using attached reproducer with adding some delay to
> {{GridDhtPartitionDemander.RebalanceFuture#checkIsDone(boolean)}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9404) Ignite start hangs infinitely when sync preloading is cancelled
[
https://issues.apache.org/jira/browse/IGNITE-9404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Pavlukhin updated IGNITE-9404:
---
Attachment: ignite-26252679.0.log
> Ignite start hangs infinitely when sync preloading is cancelled
> ---
>
> Key: IGNITE-9404
> URL: https://issues.apache.org/jira/browse/IGNITE-9404
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.6
>Reporter: Ivan Pavlukhin
>Priority: Major
> Attachments: IgniteStartStopRace.java, dump1.txt,
> ignite-26252679.0.log
>
>
> This case fires very rarely in {{TcpDiscoveryRestartTest.testRestart}}.
> Caches with {{SYNC}} rebalance mode are affected (system internal cache uses
> such mode). When _starting_ instance is trying to preload partitions for such
> cache from another instance which _stops_ around the same time, first
> instance could hang infinitely. When {{SYNC}} rebalance mode is enabled
> starting instance awaits on _preload future_ in start routine. In another
> thread it starts fetching partitions and receives an error from _stopping_
> instance and cancels _rebalance future_ but _preload future_ is not cancelled.
> It is a bit tricky to reproduce case fairly. It is possible to reproduce the
> case using attached reproducer with adding some delay to
> {{GridDhtPartitionDemander.RebalanceFuture#checkIsDone(boolean)}} method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
[
https://issues.apache.org/jira/browse/IGNITE-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Zinoviev updated IGNITE-9421:
-
Affects Version/s: (was: 2.6)
> ML Examples: LogisticRegressionSGDTrainerExample example result not correct
> ---
>
> Key: IGNITE-9421
> URL: https://issues.apache.org/jira/browse/IGNITE-9421
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Reporter: Stepan Pilschikov
>Assignee: Aleksey Zinoviev
>Priority: Major
> Fix For: 2.7
>
>
> Running
> org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
> example
> Output:
> {code}
> >>> Absolute amount of errors 100
> >>> Accuracy 0.0
> >>> Confusion matrix is [[50, 50], [0, 0]]
> >>> -
> >>> Logistic regression model over partitioned dataset usage example
> >>> completed.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
[
https://issues.apache.org/jira/browse/IGNITE-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596420#comment-16596420
]
ASF GitHub Bot commented on IGNITE-9421:
GitHub user zaleslaw opened a pull request:
https://github.com/apache/ignite/pull/4646
IGNITE-9421: fixed model output
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9421
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4646.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4646
commit 82279ec8fdb3bf7192af48783937c89d1867dce9
Author: zaleslaw
Date: 2018-08-29T14:38:47Z
IGNITE-9421: fixed model output
> ML Examples: LogisticRegressionSGDTrainerExample example result not correct
> ---
>
> Key: IGNITE-9421
> URL: https://issues.apache.org/jira/browse/IGNITE-9421
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Stepan Pilschikov
>Assignee: Aleksey Zinoviev
>Priority: Major
> Fix For: 2.7
>
>
> Running
> org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
> example
> Output:
> {code}
> >>> Absolute amount of errors 100
> >>> Accuracy 0.0
> >>> Confusion matrix is [[50, 50], [0, 0]]
> >>> -
> >>> Logistic regression model over partitioned dataset usage example
> >>> completed.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9424) Partition equal to -1 during insert to atomic cache
[
https://issues.apache.org/jira/browse/IGNITE-9424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596375#comment-16596375
]
ASF GitHub Bot commented on IGNITE-9424:
GitHub user akalash opened a pull request:
https://github.com/apache/ignite/pull/4645
IGNITE-9424 set partition to KeyCacheObject after reading from page
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9424
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4645.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4645
commit 02d9eb573966ca1a5cf6b258b2fd9bfefc0c7054
Author: Anton Kalashnikov
Date: 2018-08-29T14:03:28Z
IGNITE-9424 set partition to KeyCacheObject after reading from page
> Partition equal to -1 during insert to atomic cache
> ---
>
> Key: IGNITE-9424
> URL: https://issues.apache.org/jira/browse/IGNITE-9424
> Project: Ignite
> Issue Type: Test
>Reporter: Anton Kalashnikov
>Assignee: Anton Kalashnikov
>Priority: Major
>
> Reproduced by IgnitePdsThreadInterruptionTest.testInterruptsOnWALWrite
> {noformat}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
> (retry update if possible).: [31108]
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1261)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1740)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1090)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:817)
> at
> org.apache.ignite.internal.processors.cache.persistence.db.file.IgnitePdsThreadInterruptionTest$3.run(IgnitePdsThreadInterruptionTest.java:208)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
> Failed to update keys (retry update if possible).: [31108]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:253)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:303)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:300)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.map(GridDhtAtomicAbstractUpdateFuture.java:394)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1865)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1664)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:483)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:443)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1153)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:611)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2430)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2407)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1087)
> ... 3 more
> Suppressed: class org.apache.ignite.IgniteCheckedException:
[jira] [Created] (IGNITE-9424) Partition equal to -1 during insert to atomic cache
Anton Kalashnikov created IGNITE-9424:
-
Summary: Partition equal to -1 during insert to atomic cache
Key: IGNITE-9424
URL: https://issues.apache.org/jira/browse/IGNITE-9424
Project: Ignite
Issue Type: Test
Reporter: Anton Kalashnikov
Assignee: Anton Kalashnikov
Reproduced by IgnitePdsThreadInterruptionTest.testInterruptsOnWALWrite
{noformat}
org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys
(retry update if possible).: [31108]
at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1261)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1740)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1090)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:817)
at
org.apache.ignite.internal.processors.cache.persistence.db.file.IgnitePdsThreadInterruptionTest$3.run(IgnitePdsThreadInterruptionTest.java:208)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
Failed to update keys (retry update if possible).: [31108]
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:253)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:303)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:300)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.map(GridDhtAtomicAbstractUpdateFuture.java:394)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1865)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1664)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:483)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:443)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1153)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:611)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2430)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2407)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1087)
... 3 more
Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to
update keys.
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.UpdateErrors.addFailedKey(UpdateErrors.java:108)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKey(GridNearAtomicUpdateResponse.java:329)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2623)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1942)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1776)
... 13 more
Suppressed: class
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
Runtime failure on search row:
org.apache.ignite.internal.processors.cache.tree.SearchRow@371d7ce1
at
[jira] [Commented] (IGNITE-8886) Simultaneous using of BinaryWriter и BinaryRawWriter leads to BinaryObjectException and OOM
[
https://issues.apache.org/jira/browse/IGNITE-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596351#comment-16596351
]
ASF GitHub Bot commented on IGNITE-8886:
GitHub user alamar opened a pull request:
https://github.com/apache/ignite/pull/4643
IGNITE-8886 Fix position calculation for mixed raw binary objects.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-8886
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4643.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4643
commit e16262cca84be1e7c883cc1bc6af7dc1eeec1c32
Author: Ilya Kasnacheev
Date: 2018-08-29T13:48:02Z
IGNITE-8886 Fix position calculation for mixed raw binary objects.
> Simultaneous using of BinaryWriter и BinaryRawWriter leads to
> BinaryObjectException and OOM
> ---
>
> Key: IGNITE-8886
> URL: https://issues.apache.org/jira/browse/IGNITE-8886
> Project: Ignite
> Issue Type: Bug
> Components: binary
>Affects Versions: 2.5
>Reporter: Roman Guseinov
>Priority: Major
> Labels: binary
> Attachments: BinarylizableInvalidFlag.java, BinarylizableOOM.java
>
>
> When we use BinaryWriter and BinaryRawWriter simultaneously inside
> writeBinary method we can get the following exceptions in the process of
> deserializing objects:
> 1. class org.apache.ignite.binary.BinaryObjectException: Invalid flag value:
> 115
> {code:java}
> Exception in thread "main" javax.cache.CacheException: class
> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1302)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1732)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:910)
>at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:608)
>at
> org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag.main(BinarylizableInvalidFlag.java:33)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7322)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:171)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4563)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4537)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1350)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:907)
>... 2 more
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:909)
>at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>at
> org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:798)
>at
> org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:177)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:67)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:125)
>at
> org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1764)
>at
> org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1752)
>at
>
[jira] [Assigned] (IGNITE-8886) Simultaneous using of BinaryWriter и BinaryRawWriter leads to BinaryObjectException and OOM
[
https://issues.apache.org/jira/browse/IGNITE-8886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Kasnacheev reassigned IGNITE-8886:
---
Assignee: Ilya Kasnacheev
> Simultaneous using of BinaryWriter и BinaryRawWriter leads to
> BinaryObjectException and OOM
> ---
>
> Key: IGNITE-8886
> URL: https://issues.apache.org/jira/browse/IGNITE-8886
> Project: Ignite
> Issue Type: Bug
> Components: binary
>Affects Versions: 2.5
>Reporter: Roman Guseinov
>Assignee: Ilya Kasnacheev
>Priority: Major
> Labels: binary
> Attachments: BinarylizableInvalidFlag.java, BinarylizableOOM.java
>
>
> When we use BinaryWriter and BinaryRawWriter simultaneously inside
> writeBinary method we can get the following exceptions in the process of
> deserializing objects:
> 1. class org.apache.ignite.binary.BinaryObjectException: Invalid flag value:
> 115
> {code:java}
> Exception in thread "main" javax.cache.CacheException: class
> org.apache.ignite.IgniteCheckedException: Failed to deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1302)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1732)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:910)
>at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:608)
>at
> org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag.main(BinarylizableInvalidFlag.java:33)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7322)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:171)
>at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get0(GridCacheAdapter.java:4563)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4537)
>at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1350)
>at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.get(IgniteCacheProxyImpl.java:907)
>... 2 more
> Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
> deserialize object
> [typeName=org.apache.ignite.reproducers.fd7550.BinarylizableInvalidFlag$TestBean]
>at
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:909)
>at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>at
> org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:798)
>at
> org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:143)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinary(CacheObjectUtils.java:177)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectUtils.unwrapBinaryIfNeeded(CacheObjectUtils.java:67)
>at
> org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:125)
>at
> org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1764)
>at
> org.apache.ignite.internal.processors.cache.GridCacheContext.unwrapBinaryIfNeeded(GridCacheContext.java:1752)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.setResult(GridPartitionedSingleGetFuture.java:679)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.localGet(GridPartitionedSingleGetFuture.java:461)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.mapKeyToNode(GridPartitionedSingleGetFuture.java:342)
>at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.map(GridPartitionedSingleGetFuture.java:216)
>at
>
[jira] [Comment Edited] (IGNITE-9116) .NET: LINQ: CacheConfiguration.SqlSchema is ignored
[
https://issues.apache.org/jira/browse/IGNITE-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596311#comment-16596311
]
Pavel Tupitsyn edited comment on IGNITE-9116 at 8/29/18 1:16 PM:
-
GitHub user ptupitsyn opened a pull request:
https://github.com/apache/ignite/pull/4642
IGNITE-9116 .NET: LINQ: Use CacheConfiguration.SqlSchema when generating SQL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ptupitsyn/ignite ignite-9116
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4642.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4642
was (Author: githubbot):
GitHub user ptupitsyn opened a pull request:
https://github.com/apache/ignite/pull/4642
IGNITE-9116 .NET: LINQ: Use CacheConfiguration.SqlSchema when generating SQL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ptupitsyn/ignite ignite-9116
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4642.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4642
commit c0638d786dfa17d68ad5e48f041b2188c74132f7
Author: Pavel Tupitsyn
Date: 2018-08-13T08:42:40Z
Fix LINQ provider SQL schema inferrence
commit 4038a9b2da1f63bc8b27e367b27ccf13727ffac8
Author: Pavel Tupitsyn
Date: 2018-08-13T08:46:10Z
Add tests
commit 09e6839397aec6700670a533e1d7450f053d3034
Author: Pavel Tupitsyn
Date: 2018-08-13T10:59:46Z
Fix quoted identifiers handling
commit dd294b871eee0b9257d213d9fe52ac113bda6a77
Author: Pavel Tupitsyn
Date: 2018-08-13T11:02:42Z
Fix quoted identifiers handling
commit bdd87f22b2ee214159721ccd38933a2a9be284ab
Author: Pavel Tupitsyn
Date: 2018-08-13T16:55:59Z
Fix quoted identifiers handling
commit 43623d411f0c253f08a691055f4589c8ae39822e
Author: Pavel Tupitsyn
Date: 2018-08-13T17:04:31Z
Fixing tests
commit fa034fa8f1b298d6eb459065f8b4ab88ec57fced
Author: Pavel Tupitsyn
Date: 2018-08-13T17:06:26Z
TODOs
commit d60fe10fe89f4d681ade6ac418326e5c1f63ae09
Author: Pavel Tupitsyn
Date: 2018-08-29T08:55:06Z
NormalizeSchemaName
commit d440cecff1fbdcbd9c6f2d36dfd82cb218119c87
Author: Pavel Tupitsyn
Date: 2018-08-29T08:59:09Z
NormalizeSchemaName
commit d22c5a71c539d2515db292be9ed0be107e75154f
Author: Pavel Tupitsyn
Date: 2018-08-29T09:08:27Z
Fixing tests
commit fe7adfbc66b76d320a35e3e8bb43c23a79f6ba59
Author: Pavel Tupitsyn
Date: 2018-08-29T12:27:44Z
Fixing tests
commit cff6ec094bd3b2446bc98d5f6c575024159c8ee3
Author: Pavel Tupitsyn
Date: 2018-08-29T12:47:58Z
Fixing tests
commit be910ff16b1a75a9a30c831107a1489b3ed55bb5
Author: Pavel Tupitsyn
Date: 2018-08-29T12:58:06Z
Fixing tests
commit 432c5d7eae18cb3030aa624cebcea0f81efda046
Author: Pavel Tupitsyn
Date: 2018-08-29T13:08:03Z
Merge remote-tracking branch 'origin/master' into ignite-9116
> .NET: LINQ: CacheConfiguration.SqlSchema is ignored
> ---
>
> Key: IGNITE-9116
> URL: https://issues.apache.org/jira/browse/IGNITE-9116
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.5, 2.6
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET
> Fix For: 2.7
>
>
> {{CacheConfiguration.SqlSchema}} and {{CacheClientConfiguration.SqlSchema}}
> are ignored by LINQ provider, schema name is inferred only from cache name.
> See {{ExpressionWalker.GetTableNameWithSchema}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (IGNITE-9116) .NET: LINQ: CacheConfiguration.SqlSchema is ignored
[
https://issues.apache.org/jira/browse/IGNITE-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-9116:
---
Comment: was deleted
(was: Done: https://github.com/apache/ignite/pull/4642)
> .NET: LINQ: CacheConfiguration.SqlSchema is ignored
> ---
>
> Key: IGNITE-9116
> URL: https://issues.apache.org/jira/browse/IGNITE-9116
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.5, 2.6
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET
> Fix For: 2.7
>
>
> {{CacheConfiguration.SqlSchema}} and {{CacheClientConfiguration.SqlSchema}}
> are ignored by LINQ provider, schema name is inferred only from cache name.
> See {{ExpressionWalker.GetTableNameWithSchema}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9116) .NET: LINQ: CacheConfiguration.SqlSchema is ignored
[
https://issues.apache.org/jira/browse/IGNITE-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596313#comment-16596313
]
Pavel Tupitsyn commented on IGNITE-9116:
Done: https://github.com/apache/ignite/pull/4642
> .NET: LINQ: CacheConfiguration.SqlSchema is ignored
> ---
>
> Key: IGNITE-9116
> URL: https://issues.apache.org/jira/browse/IGNITE-9116
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.5, 2.6
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET
> Fix For: 2.7
>
>
> {{CacheConfiguration.SqlSchema}} and {{CacheClientConfiguration.SqlSchema}}
> are ignored by LINQ provider, schema name is inferred only from cache name.
> See {{ExpressionWalker.GetTableNameWithSchema}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9116) .NET: LINQ: CacheConfiguration.SqlSchema is ignored
[
https://issues.apache.org/jira/browse/IGNITE-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596311#comment-16596311
]
ASF GitHub Bot commented on IGNITE-9116:
GitHub user ptupitsyn opened a pull request:
https://github.com/apache/ignite/pull/4642
IGNITE-9116 .NET: LINQ: Use CacheConfiguration.SqlSchema when generating SQL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ptupitsyn/ignite ignite-9116
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4642.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4642
commit c0638d786dfa17d68ad5e48f041b2188c74132f7
Author: Pavel Tupitsyn
Date: 2018-08-13T08:42:40Z
Fix LINQ provider SQL schema inferrence
commit 4038a9b2da1f63bc8b27e367b27ccf13727ffac8
Author: Pavel Tupitsyn
Date: 2018-08-13T08:46:10Z
Add tests
commit 09e6839397aec6700670a533e1d7450f053d3034
Author: Pavel Tupitsyn
Date: 2018-08-13T10:59:46Z
Fix quoted identifiers handling
commit dd294b871eee0b9257d213d9fe52ac113bda6a77
Author: Pavel Tupitsyn
Date: 2018-08-13T11:02:42Z
Fix quoted identifiers handling
commit bdd87f22b2ee214159721ccd38933a2a9be284ab
Author: Pavel Tupitsyn
Date: 2018-08-13T16:55:59Z
Fix quoted identifiers handling
commit 43623d411f0c253f08a691055f4589c8ae39822e
Author: Pavel Tupitsyn
Date: 2018-08-13T17:04:31Z
Fixing tests
commit fa034fa8f1b298d6eb459065f8b4ab88ec57fced
Author: Pavel Tupitsyn
Date: 2018-08-13T17:06:26Z
TODOs
commit d60fe10fe89f4d681ade6ac418326e5c1f63ae09
Author: Pavel Tupitsyn
Date: 2018-08-29T08:55:06Z
NormalizeSchemaName
commit d440cecff1fbdcbd9c6f2d36dfd82cb218119c87
Author: Pavel Tupitsyn
Date: 2018-08-29T08:59:09Z
NormalizeSchemaName
commit d22c5a71c539d2515db292be9ed0be107e75154f
Author: Pavel Tupitsyn
Date: 2018-08-29T09:08:27Z
Fixing tests
commit fe7adfbc66b76d320a35e3e8bb43c23a79f6ba59
Author: Pavel Tupitsyn
Date: 2018-08-29T12:27:44Z
Fixing tests
commit cff6ec094bd3b2446bc98d5f6c575024159c8ee3
Author: Pavel Tupitsyn
Date: 2018-08-29T12:47:58Z
Fixing tests
commit be910ff16b1a75a9a30c831107a1489b3ed55bb5
Author: Pavel Tupitsyn
Date: 2018-08-29T12:58:06Z
Fixing tests
commit 432c5d7eae18cb3030aa624cebcea0f81efda046
Author: Pavel Tupitsyn
Date: 2018-08-29T13:08:03Z
Merge remote-tracking branch 'origin/master' into ignite-9116
> .NET: LINQ: CacheConfiguration.SqlSchema is ignored
> ---
>
> Key: IGNITE-9116
> URL: https://issues.apache.org/jira/browse/IGNITE-9116
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.5, 2.6
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET
> Fix For: 2.7
>
>
> {{CacheConfiguration.SqlSchema}} and {{CacheClientConfiguration.SqlSchema}}
> are ignored by LINQ provider, schema name is inferred only from cache name.
> See {{ExpressionWalker.GetTableNameWithSchema}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9305) Wrong off-heap size is reported for a node
[
https://issues.apache.org/jira/browse/IGNITE-9305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596303#comment-16596303
]
Amelchev Nikita commented on IGNITE-9305:
-
Hello, [~xtern], [~dmagda],
I suggest more readable format for off-heap data regions:
{noformat}
^-- H/N/C [hosts=1, nodes=2, CPUs=8]
^-- CPU [cur=15.73%, avg=0%, GC=0%]
^-- PageMemory [pages=25]
^-- Heap [used=71MB, free=97.99%, comm=203MB]
^-- Off-heap [used=0MB, free=99.96%, comm=220MB]
^-- Off-heap data regions:
^-- sysMemPlc [used=0MB, free=99.98%, comm=100MB]
^-- default [used=0MB, free=99.8%, comm=20MB]
^-- metastoreMemPlc [used=0MB, free=99.96%, comm=100MB]
^-- Ignite persistence region: default [used=0MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=6, qSize=0]
^-- System thread pool [active=0, idle=6, qSize=0]
{noformat}
Any thoughts?
> Wrong off-heap size is reported for a node
> --
>
> Key: IGNITE-9305
> URL: https://issues.apache.org/jira/browse/IGNITE-9305
> Project: Ignite
> Issue Type: Task
>Affects Versions: 2.6
>Reporter: Denis Magda
>Assignee: Pavel Pereslegin
>Priority: Blocker
> Fix For: 2.7
>
>
> Was troubleshooting an Ignite deployment today and couldn't find out from the
> logs what was the actual off-heap space used.
> Those were the given memory resoures (Ignite 2.6):
> {code}
> [2018-08-16 15:07:49,961][INFO ][main][GridDiscoveryManager] Topology
> snapshot [ver=1, servers=1, clients=0, CPUs=64, offheap=30.0GB, heap=24.0GB]
> {code}
> And that weird stuff was reported by the node (pay attention to the last
> line):
> {code}
> [2018-08-16 15:45:50,211][INFO
> ][grid-timeout-worker-#135%cluster_31-Dec-2017%][IgniteKernal%cluster_31-Dec-2017]
>
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=c033026e, name=cluster_31-Dec-2017, uptime=00:38:00.257]
> ^-- H/N/C [hosts=1, nodes=1, CPUs=64]
> ^-- CPU [cur=0.03%, avg=5.54%, GC=0%]
> ^-- PageMemory [pages=6997377]
> ^-- Heap [used=9706MB, free=61.18%, comm=22384MB]
> ^-- Non heap [used=144MB, free=-1%, comm=148MB] - this line is always the
> same!
> {code}
> Had to change the code by using
> {code}dataRegion.getPhysicalMemoryPages(){code} to find out that actual
> off-heap usage size was
> {code}
> >>> Physical Memory Size: 28651614208 => 27324 MB, 26 GB
> {code}
> The logs have to report the following instead:
> {code}
> ^-- Off-heap {Data Region 1} [used={dataRegion1.getPhysicalMemorySize()},
> free=X%, comm=dataRegion1.maxSize()]
> ^-- Off-heap {Data Region 2} [used={dataRegion2.getPhysicalMemorySize()},
> free=X%, comm=dataRegion2.maxSize()]
> {code}
> If Ignite persistence is enabled then the following extra lines have to be
> added to see the disk used space:
> {code}
> ^-- Ignite persistence {Data Region 1}:
> used={dataRegion1.getTotalAllocatedSize() -
> dataRegion1.getPhysicalMemorySize()}
> ^-- Ignite persistence {Data Region 2}
> [used={dataRegion2.getTotalAllocatedSize() -
> dataRegion2.getPhysicalMemorySize()}]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9301) Support method compute withNoResultCache in .Net
[ https://issues.apache.org/jira/browse/IGNITE-9301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596301#comment-16596301 ] Pavel Tupitsyn commented on IGNITE-9301: Oh, I'm late to the party, thanks everyone for handling this! > Support method compute withNoResultCache in .Net > > > Key: IGNITE-9301 > URL: https://issues.apache.org/jira/browse/IGNITE-9301 > Project: Ignite > Issue Type: Task > Components: platforms >Affects Versions: 2.6 >Reporter: Dmitriy Pavlov >Assignee: Aleksei Zaitsev >Priority: Major > Labels: .net > Fix For: 2.7 > > > During https://issues.apache.org/jira/browse/IGNITE-6284 implementation new > method was added - > org.apache.ignite.IgniteCompute#withNoResultCache > but this method was not supported in .NET API version. > It is required to add correct support to .NET. > Please remove method name from UnneededMethods in > modules/platforms/dotnet/Apache.Ignite.Core.Tests/ApiParity/ComputeParityTest.cs > once issue is done -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
[
https://issues.apache.org/jira/browse/IGNITE-9423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9423:
---
Description:
[Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
complains that
{{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
unreliable:
{quote}*This test looks flaky:*
Frequent test status changes: 39 changes out of 128 invocations
Test status change in build without changes: from successful to failed
[View test history
»|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
Output for most recent test failure: {noformat}
SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase
[Data divided on 4 partitions, training with batch size \{1}]
java.lang.AssertionError: expected:<0.0> but was:<1.0>
at
org.apache.ignite.ml.svm.SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase
(SVMMultiClassTrainerTest.java:53){noformat}
was:
[Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
complains that
{{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
unreliable:
{quote}*This test looks flaky:*
Frequent test status changes: 39 changes out of 128 invocations
Test status change in build without changes: from successful to failed
[View test history
»|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
Output for most recent test failure: {noformat}
SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on
4 partitions,
training with batch size \{1}]
java.lang.AssertionError: expected:<0.0> but was:<1.0>
at org.apache.ignite.ml.svm.SVMMultiClassTrainerTest
.testTrainWithTheLinearlySeparableCase(SVMMultiClassTrainerTest.java:53){noformat}
> unreliable test:
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions, training with batch size {1}]
> ---
>
> Key: IGNITE-9423
> URL: https://issues.apache.org/jira/browse/IGNITE-9423
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
>Priority: Major
> Fix For: 2.7
>
>
> [Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
> complains that
> {{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
> unreliable:
> {quote}*This test looks flaky:*
> Frequent test status changes: 39 changes out of 128 invocations
> Test status change in build without changes: from successful to failed
> [View test history
> »|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
> Output for most recent test failure: {noformat}
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase
> [Data divided on 4 partitions, training with batch size \{1}]
> java.lang.AssertionError: expected:<0.0> but was:<1.0>
> at
> org.apache.ignite.ml.svm.SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase
> (SVMMultiClassTrainerTest.java:53){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9270) Design thread per partition model
[ https://issues.apache.org/jira/browse/IGNITE-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9270: Ignite Flags: (was: Docs Required) > Design thread per partition model > - > > Key: IGNITE-9270 > URL: https://issues.apache.org/jira/browse/IGNITE-9270 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Major > Labels: thread-per-partition > Fix For: 2.7 > > > A new model of executing cache partition operations (READ, CREATE, UPDATE, > DELETE) should satisfy following conditions > 1) All modify operations (CREATE, UPDATE, DELETE) on some partition must be > performed by the same thread. > 2) Read operations can be executed by any thread. > 3) Ordering of modify operations on primary and backup nodes should be same. > We should investigate performance if we choose dedicated executor service for > such operations, or we can use a messaging model to use network threads to > perform such operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
[
https://issues.apache.org/jira/browse/IGNITE-9423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9423:
---
Fix Version/s: 2.7
> unreliable test:
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions, training with batch size {1}]
> ---
>
> Key: IGNITE-9423
> URL: https://issues.apache.org/jira/browse/IGNITE-9423
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
>Priority: Major
> Fix For: 2.7
>
>
> [Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
> complains that
> {{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
> unreliable:
> {quote}*This test looks flaky:*
> Frequent test status changes: 39 changes out of 128 invocations
> Test status change in build without changes: from successful to failed
> [View test history
> »|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
> Output for most recent test failure: {noformat}
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions,
> training with batch size \{1}]
> java.lang.AssertionError: expected:<0.0> but was:<1.0>
> at org.apache.ignite.ml.svm.SVMMultiClassTrainerTest
> .testTrainWithTheLinearlySeparableCase(SVMMultiClassTrainerTest.java:53){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9271) Implement transaction commit using thread per partition model
[ https://issues.apache.org/jira/browse/IGNITE-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9271: Ignite Flags: (was: Docs Required) > Implement transaction commit using thread per partition model > - > > Key: IGNITE-9271 > URL: https://issues.apache.org/jira/browse/IGNITE-9271 > Project: Ignite > Issue Type: Sub-task > Components: cache >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Major > Labels: thread-per-partition > Fix For: 2.7 > > > Currently, we perform commit of a transaction from sys thread and do write > operations with multiple partitions. > We should delegate such operations to an appropriate thread and wait for > results. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
[
https://issues.apache.org/jira/browse/IGNITE-9423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko reassigned IGNITE-9423:
--
Assignee: Oleg Ignatenko
> unreliable test:
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions, training with batch size {1}]
> ---
>
> Key: IGNITE-9423
> URL: https://issues.apache.org/jira/browse/IGNITE-9423
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
>Priority: Major
> Fix For: 2.7
>
>
> [Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
> complains that
> {{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
> unreliable:
> {quote}*This test looks flaky:*
> Frequent test status changes: 39 changes out of 128 invocations
> Test status change in build without changes: from successful to failed
> [View test history
> »|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
> Output for most recent test failure: {noformat}
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions,
> training with batch size \{1}]
> java.lang.AssertionError: expected:<0.0> but was:<1.0>
> at org.apache.ignite.ml.svm.SVMMultiClassTrainerTest
> .testTrainWithTheLinearlySeparableCase(SVMMultiClassTrainerTest.java:53){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9084) Trash in WAL after node stop may affect WAL rebalance
[
https://issues.apache.org/jira/browse/IGNITE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Kovalenko updated IGNITE-9084:
Ignite Flags: (was: Docs Required)
> Trash in WAL after node stop may affect WAL rebalance
> -
>
> Key: IGNITE-9084
> URL: https://issues.apache.org/jira/browse/IGNITE-9084
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.6
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Major
> Fix For: 2.7
>
>
> During iteration over WAL we can face with trash in WAL segment, which can
> remains after node restart. We should handle this situation in WAL rebalance
> iterator and gracefully stop iteration process.
> {noformat}
> [2018-07-25
> 17:18:21,152][ERROR][sys-#25385%persistence.IgnitePdsTxHistoricalRebalancingTest0%][GridCacheIoManager]
> Failed to process message [senderId=f0d35df7-ff93-4b6c-b699-45f3e7c3,
> messageType=class
> o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandMessage]
> class org.apache.ignite.IgniteException: Failed to read WAL record at
> position: 19346739 size: 67108864
> at
> org.apache.ignite.internal.util.lang.GridIteratorAdapter.next(GridIteratorAdapter.java:38)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$WALHistoricalIterator.advance(GridCacheOffheapManager.java:1033)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$WALHistoricalIterator.next(GridCacheOffheapManager.java:948)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$WALHistoricalIterator.nextX(GridCacheOffheapManager.java:917)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$WALHistoricalIterator.nextX(GridCacheOffheapManager.java:842)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.IgniteRebalanceIteratorImpl.nextX(IgniteRebalanceIteratorImpl.java:130)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.IgniteRebalanceIteratorImpl.next(IgniteRebalanceIteratorImpl.java:185)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.IgniteRebalanceIteratorImpl.next(IgniteRebalanceIteratorImpl.java:37)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:348)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:370)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:380)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:365)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1613)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to read WAL
> record at position: 19346739 size: 67108864
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.handleRecordException(AbstractWalRecordsIterator.java:263)
> at
>
[jira] [Commented] (IGNITE-9388) mesos IgniteProvider tries to access obolete ignite.run or download from slow archive
[ https://issues.apache.org/jira/browse/IGNITE-9388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596293#comment-16596293 ] Roman Shtykh commented on IGNITE-9388: -- Confirmed downloading at the private mesos cluster. > mesos IgniteProvider tries to access obolete ignite.run or download from slow > archive > - > > Key: IGNITE-9388 > URL: https://issues.apache.org/jira/browse/IGNITE-9388 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.6 >Reporter: Roman Shtykh >Assignee: Roman Shtykh >Priority: Major > Fix For: 2.7 > > > Although it downloads from the archive, for Japan, for instance, it takes 2-3 > hours. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9086) Error during commit transaction on primary node may lead to breaking transaction data integrity
[ https://issues.apache.org/jira/browse/IGNITE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9086: Ignite Flags: (was: Docs Required) > Error during commit transaction on primary node may lead to breaking > transaction data integrity > --- > > Key: IGNITE-9086 > URL: https://issues.apache.org/jira/browse/IGNITE-9086 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.4, 2.5, 2.6 >Reporter: Pavel Kovalenko >Priority: Major > Fix For: 2.7 > > > Transaction properties are PESSIMISTIC, REPEATABLE READ. > If primary partitions participating in the transaction are spread across > several nodes and commit is failed on some of the primary nodes while other > primary nodes have committed transaction it may lead to breaking transaction > data integrity. A data become inconsistent even after rebalance when the node > with failed commit returns back to the cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
[
https://issues.apache.org/jira/browse/IGNITE-9423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9423:
---
Ignite Flags: (was: Docs Required)
> unreliable test:
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions, training with batch size {1}]
> ---
>
> Key: IGNITE-9423
> URL: https://issues.apache.org/jira/browse/IGNITE-9423
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
>Priority: Major
> Fix For: 2.7
>
>
> [Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
> complains that
> {{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
> unreliable:
> {quote}*This test looks flaky:*
> Frequent test status changes: 39 changes out of 128 invocations
> Test status change in build without changes: from successful to failed
> [View test history
> »|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
> Output for most recent test failure: {noformat}
> SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided
> on 4 partitions,
> training with batch size \{1}]
> java.lang.AssertionError: expected:<0.0> but was:<1.0>
> at org.apache.ignite.ml.svm.SVMMultiClassTrainerTest
> .testTrainWithTheLinearlySeparableCase(SVMMultiClassTrainerTest.java:53){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9088) Add ability to dump persistence after particular test
[ https://issues.apache.org/jira/browse/IGNITE-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9088: Ignite Flags: (was: Docs Required) > Add ability to dump persistence after particular test > - > > Key: IGNITE-9088 > URL: https://issues.apache.org/jira/browse/IGNITE-9088 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Major > Fix For: 2.7 > > > Sometimes it's needed to analyze persistence after a particular test finish > on TeamCity. > We need to add an ability to dump persistence dirs/files to the specified > directory on test running host for further analysis. > This should be managed by a property. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9206) Node can't join to ring if all existing nodes have stopped and another new node joined ahead
[
https://issues.apache.org/jira/browse/IGNITE-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Kovalenko updated IGNITE-9206:
Ignite Flags: (was: Docs Required)
> Node can't join to ring if all existing nodes have stopped and another new
> node joined ahead
>
>
> Key: IGNITE-9206
> URL: https://issues.apache.org/jira/browse/IGNITE-9206
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Priority: Major
> Fix For: 2.7
>
>
> TcpDiscovery SPI problem.
> Situation:
> Existing cluster with nodes 1 and 2. Nodes 1 and 2 are stopping.
> 1) Node 3 joins to cluster and sends JoinMessage to node 2.
> 2) Node 2 is stopping and unable to handle JoinMessage from node 3. Node 3
> choose node 1 as the next node to send the message.
> 3) Node 3 sends JoinMessage to node 1.
> 4) Node 4 joins to cluster.
> 5) Node 1 is stopping and unable to handle JoinMessage from node 3.
> 6) Node 4 sees that there are no alive nodes in the ring at the time and
> become the first node in the topology.
> 7) Node 3 sends JoinMessage to Node 4 and this process repeats again and
> again without any success.
> At Node 4 logs we can see that remote connection from Node 3 is established
> but no active actions have performed. Node 3 leaves in CONNECTING state
> forever. At the same time Node 4 thinks that Node 3 is already in the ring.
> Failed test:
> GridCacheReplicatedDataStructuresFailoverSelfTest#testAtomicSequenceConstantTopologyChange
> Link to TC:
> https://ci.ignite.apache.org/viewLog.html?buildId=1594376=buildResultsDiv=IgniteTests24Java8_DataStructures
> Shrinked log:
> {code:java}
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,733][INFO ][main][root] >>>
> Stopping grid
> [name=replicated.GridCacheReplicatedDataStructuresFailoverSelfTest0,
> id=3e2c94bd-8e98-4dd9-8d1a-befbfe00]
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,739][INFO
> ][thread-replicated.GridCacheReplicatedDataStructuresFailoverSelfTest7][root]
> Start node: replicated.GridCacheReplicatedDataStructuresFailoverSelfTest7
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,740][INFO
> ][tcp-disco-msg-worker-#2146%replicated.GridCacheReplicatedDataStructuresFailoverSelfTest6%][TcpDiscoverySpi]
> New next node [newNext=TcpDiscoveryNode
> [id=3e2c94bd-8e98-4dd9-8d1a-befbfe00, addrs=ArrayList [127.0.0.1],
> sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1,
> lastExchangeTime=1533416953738, loc=false, ver=2.7.0#20180803-sha1:3ab8bbad,
> isClient=false]]
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,741][INFO
> ][tcp-disco-srvr-#2100%replicated.GridCacheReplicatedDataStructuresFailoverSelfTest0%][TcpDiscoverySpi]
> TCP discovery accepted incoming connection [rmtAddr=/127.0.0.1,
> rmtPort=50099]
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,741][INFO
> ][tcp-disco-srvr-#2100%replicated.GridCacheReplicatedDataStructuresFailoverSelfTest0%][TcpDiscoverySpi]
> TCP discovery spawning a new thread for connection [rmtAddr=/127.0.0.1,
> rmtPort=50099]
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,743][INFO
> ][tcp-disco-sock-reader-#2151%replicated.GridCacheReplicatedDataStructuresFailoverSelfTest0%][TcpDiscoverySpi]
> Started serving remote node connection [rmtAddr=/127.0.0.1:50099,
> rmtPort=50099]
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,746][INFO
> ][thread-replicated.GridCacheReplicatedDataStructuresFailoverSelfTest7][GridCacheReplicatedDataStructuresFailoverSelfTest7]
>
> [00:09:13] : [Step 3/4]
> [00:09:13] : [Step 3/4] >>>__
> [00:09:13] : [Step 3/4] >>> / _/ ___/ |/ / _/_ __/ __/
> [00:09:13] : [Step 3/4] >>> _/ // (7 7// / / / / _/
> [00:09:13] : [Step 3/4] >>> /___/\___/_/|_/___/ /_/ /___/
> [00:09:13] : [Step 3/4] >>>
> [00:09:13] : [Step 3/4] >>> ver. 2.7.0-SNAPSHOT#20180803-sha1:3ab8bbad
> [00:09:13] : [Step 3/4] >>> 2018 Copyright(C) Apache Software Foundation
> [00:09:13] : [Step 3/4] >>>
> [00:09:13] : [Step 3/4] >>> Ignite documentation: http://ignite.apache.org
> [00:09:13] : [Step 3/4]
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,746][INFO
> ][thread-replicated.GridCacheReplicatedDataStructuresFailoverSelfTest7][GridCacheReplicatedDataStructuresFailoverSelfTest7]
> Config URL: n/a
> [00:09:13] : [Step 3/4] [2018-08-04 21:09:13,747][INFO
> ][thread-replicated.GridCacheReplicatedDataStructuresFailoverSelfTest7][GridCacheReplicatedDataStructuresFailoverSelfTest7]
> IgniteConfiguration
> [igniteInstanceName=replicated.GridCacheReplicatedDataStructuresFailoverSelfTest7,
> pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, stripedPoolSize=8,
> sysPoolSize=8, mgmtPoolSize=4,
[jira] [Updated] (IGNITE-9185) Collect and check update counters visited during WAL rebalance
[ https://issues.apache.org/jira/browse/IGNITE-9185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9185: Ignite Flags: (was: Docs Required) > Collect and check update counters visited during WAL rebalance > -- > > Key: IGNITE-9185 > URL: https://issues.apache.org/jira/browse/IGNITE-9185 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.5 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Critical > Fix For: 2.7 > > > Currently we don't check what update counters we visit during WAL iteration > and what data we send to a demander node. There can be situation, that we met > last requested update counter in WAL and stop rebalance process, while due to > possible DataRecord's reordering we miss some updates after. > If rebalance process breaks due to end of WAL, but not all data records were > visit, we can easily check what records are missed, cancel rebalance and > print useful information to log for further debug. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9422) All client node fails with ZKDiscovery enabled.
Stanilovsky Evgeny created IGNITE-9422:
--
Summary: All client node fails with ZKDiscovery enabled.
Key: IGNITE-9422
URL: https://issues.apache.org/jira/browse/IGNITE-9422
Project: Ignite
Issue Type: Improvement
Components: zookeeper
Affects Versions: 2.6
Reporter: Stanilovsky Evgeny
Found problem with using ZKDiscovery:
{code:java}
2018-08-28 17:43:17,953 INFO
[org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl]
(zk-DPL_GRID%DplGridNodeName-EventThread) Newer version of existing
BinaryMetadata[type
Id=557511097,
typeName=com.sbt.bm.ucp.published.api.retail.PublishedIndividual_DPL_PROXY] is
received from node 33384a02-5cbb-4dd6-8318-6ee34c88a400; updating it locally
2018-08-28 17:43:17,954 ERROR
[org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl]
(zk-DPL_GRID%DplGridNodeName-EventThread) Fatal error in ZookeeperDiscovery.
Stopping the node in orde
r to prevent cluster wide instability.: java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.onJoiningNodeDataReceived(CacheObjectBinaryProcessorImpl.java:1001)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:909)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processBulkJoin(ZookeeperDiscoveryImpl.java:2792)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2638)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2610)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9422) All client node fails with ZKDiscovery enabled.
[
https://issues.apache.org/jira/browse/IGNITE-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stanilovsky Evgeny updated IGNITE-9422:
---
Ignite Flags: (was: Docs Required)
> All client node fails with ZKDiscovery enabled.
> ---
>
> Key: IGNITE-9422
> URL: https://issues.apache.org/jira/browse/IGNITE-9422
> Project: Ignite
> Issue Type: Improvement
> Components: zookeeper
>Affects Versions: 2.6
>Reporter: Stanilovsky Evgeny
>Priority: Major
>
> Found problem with using ZKDiscovery:
> {code:java}
> 2018-08-28 17:43:17,953 INFO
> [org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl]
> (zk-DPL_GRID%DplGridNodeName-EventThread) Newer version of existing
> BinaryMetadata[type
> Id=557511097,
> typeName=com.sbt.bm.ucp.published.api.retail.PublishedIndividual_DPL_PROXY]
> is received from node 33384a02-5cbb-4dd6-8318-6ee34c88a400; updating it
> locally
> 2018-08-28 17:43:17,954 ERROR
> [org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl]
> (zk-DPL_GRID%DplGridNodeName-EventThread) Fatal error in ZookeeperDiscovery.
> Stopping the node in orde
> r to prevent cluster wide instability.: java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.onJoiningNodeDataReceived(CacheObjectBinaryProcessorImpl.java:1001)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:909)
> at
> org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processBulkJoin(ZookeeperDiscoveryImpl.java:2792)
> at
> org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2638)
> at
> org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2610)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9422) All client node fails with ZKDiscovery enabled.
[
https://issues.apache.org/jira/browse/IGNITE-9422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stanilovsky Evgeny updated IGNITE-9422:
---
Description:
Found problem with using ZKDiscovery:
{code:java}
2018-08-28 17:43:17,953 INFO
[org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl]
(zk-D_GRID%DplGridNodeName-EventThread) Newer version of existing
BinaryMetadata[type
Id=557511097, typeName=PublishedIndividual_D_PROXY] is received from node
33384a02-5cbb-4dd6-8318-6ee34c88a400; updating it locally
2018-08-28 17:43:17,954 ERROR
[org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl]
(zk-D_GRID%DplGridNodeName-EventThread) Fatal error in ZookeeperDiscovery.
Stopping the node in orde
r to prevent cluster wide instability.: java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.onJoiningNodeDataReceived(CacheObjectBinaryProcessorImpl.java:1001)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:909)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processBulkJoin(ZookeeperDiscoveryImpl.java:2792)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2638)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2610)
{code}
was:
Found problem with using ZKDiscovery:
{code:java}
2018-08-28 17:43:17,953 INFO
[org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl]
(zk-DPL_GRID%DplGridNodeName-EventThread) Newer version of existing
BinaryMetadata[type
Id=557511097,
typeName=com.sbt.bm.ucp.published.api.retail.PublishedIndividual_DPL_PROXY] is
received from node 33384a02-5cbb-4dd6-8318-6ee34c88a400; updating it locally
2018-08-28 17:43:17,954 ERROR
[org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl]
(zk-DPL_GRID%DplGridNodeName-EventThread) Fatal error in ZookeeperDiscovery.
Stopping the node in orde
r to prevent cluster wide instability.: java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.onJoiningNodeDataReceived(CacheObjectBinaryProcessorImpl.java:1001)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:909)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processBulkJoin(ZookeeperDiscoveryImpl.java:2792)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2638)
at
org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2610)
{code}
> All client node fails with ZKDiscovery enabled.
> ---
>
> Key: IGNITE-9422
> URL: https://issues.apache.org/jira/browse/IGNITE-9422
> Project: Ignite
> Issue Type: Improvement
> Components: zookeeper
>Affects Versions: 2.6
>Reporter: Stanilovsky Evgeny
>Priority: Major
>
> Found problem with using ZKDiscovery:
> {code:java}
> 2018-08-28 17:43:17,953 INFO
> [org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl]
> (zk-D_GRID%DplGridNodeName-EventThread) Newer version of existing
> BinaryMetadata[type
> Id=557511097, typeName=PublishedIndividual_D_PROXY] is received from node
> 33384a02-5cbb-4dd6-8318-6ee34c88a400; updating it locally
> 2018-08-28 17:43:17,954 ERROR
> [org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl]
> (zk-D_GRID%DplGridNodeName-EventThread) Fatal error in ZookeeperDiscovery.
> Stopping the node in orde
> r to prevent cluster wide instability.: java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.onJoiningNodeDataReceived(CacheObjectBinaryProcessorImpl.java:1001)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:909)
> at
> org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processBulkJoin(ZookeeperDiscoveryImpl.java:2792)
> at
> org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2638)
> at
> org.apache.ignite.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.processNewEvents(ZookeeperDiscoveryImpl.java:2610)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9420) Move logical recovery phase outside of PME
[ https://issues.apache.org/jira/browse/IGNITE-9420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9420: Ignite Flags: (was: Docs Required) > Move logical recovery phase outside of PME > -- > > Key: IGNITE-9420 > URL: https://issues.apache.org/jira/browse/IGNITE-9420 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.5 >Reporter: Pavel Kovalenko >Priority: Major > Fix For: 2.7 > > > Currently, we perform logical recovery in PME here > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#restoreState > We should move logical recovery before discovery manager will start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9418) Avoid initialize file page store manager for caches during PME synchronously
[ https://issues.apache.org/jira/browse/IGNITE-9418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9418: Ignite Flags: (was: Docs Required) > Avoid initialize file page store manager for caches during PME synchronously > > > Key: IGNITE-9418 > URL: https://issues.apache.org/jira/browse/IGNITE-9418 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.5 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Critical > Fix For: 2.7 > > > Currently, we do creation for partition and index files during PME for > starting caches at the beginning of > org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#readCheckpointAndRestoreMemory > method. > We should avoid this because sometimes it took a long time as we perform > writing to disk. > If the cache was registered during PME we should initialize page store lazy > or asynchronously. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9419) Avoid saving cache configuration synchronously during PME
[ https://issues.apache.org/jira/browse/IGNITE-9419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kovalenko updated IGNITE-9419: Ignite Flags: (was: Docs Required) > Avoid saving cache configuration synchronously during PME > - > > Key: IGNITE-9419 > URL: https://issues.apache.org/jira/browse/IGNITE-9419 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.5 >Reporter: Pavel Kovalenko >Priority: Major > Fix For: 2.7 > > > Currently, we save cache configuration during PME at the activation phase > here > org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.CachesInfo#updateCachesInfo > . We should avoid this, as it performs operations to a disk. We should save > it asynchronously or lazy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9423) unreliable test: SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on 4 partitions, training with batch size {1}]
Oleg Ignatenko created IGNITE-9423:
--
Summary: unreliable test:
SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on
4 partitions, training with batch size {1}]
Key: IGNITE-9423
URL: https://issues.apache.org/jira/browse/IGNITE-9423
Project: Ignite
Issue Type: Task
Components: ml
Affects Versions: 2.6
Reporter: Oleg Ignatenko
[Teamcity here|https://ci.ignite.apache.org/viewLog.html?buildId=1755119;]
complains that
{{SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase}} is
unreliable:
{quote}*This test looks flaky:*
Frequent test status changes: 39 changes out of 128 invocations
Test status change in build without changes: from successful to failed
[View test history
»|https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=testDetails=5118678405024105955#analysis]{quote}
Output for most recent test failure: {noformat}
SVMMultiClassTrainerTest.testTrainWithTheLinearlySeparableCase[Data divided on
4 partitions,
training with batch size \{1}]
java.lang.AssertionError: expected:<0.0> but was:<1.0>
at org.apache.ignite.ml.svm.SVMMultiClassTrainerTest
.testTrainWithTheLinearlySeparableCase(SVMMultiClassTrainerTest.java:53){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9413) [ML] Learning rate optimization for GDB.
[ https://issues.apache.org/jira/browse/IGNITE-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Platonov reassigned IGNITE-9413: --- Assignee: Alexey Platonov > [ML] Learning rate optimization for GDB. > > > Key: IGNITE-9413 > URL: https://issues.apache.org/jira/browse/IGNITE-9413 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Major > Fix For: 2.7 > > > We need to support learning rate optimization while training for MSE-loss and > Log-loss -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9414) [ML] Using sparce vectors in Tree-based algorithms.
[ https://issues.apache.org/jira/browse/IGNITE-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Platonov reassigned IGNITE-9414: --- Assignee: Alexey Platonov > [ML] Using sparce vectors in Tree-based algorithms. > --- > > Key: IGNITE-9414 > URL: https://issues.apache.org/jira/browse/IGNITE-9414 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Minor > Fix For: 2.7 > > > We need to support sparce vectors in DecisionTrees, RF, GDB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9415) [ML] Using sparce vectors in LSQR and MLP
[ https://issues.apache.org/jira/browse/IGNITE-9415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Platonov reassigned IGNITE-9415: --- Assignee: Alexey Platonov > [ML] Using sparce vectors in LSQR and MLP > - > > Key: IGNITE-9415 > URL: https://issues.apache.org/jira/browse/IGNITE-9415 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Minor > Fix For: 2.7 > > > We need to investigate and apply sparce vectors support in BLAS for LSQR and > MLP (or implement own version) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9412) [ML] GDB convergence by error support.
[ https://issues.apache.org/jira/browse/IGNITE-9412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Platonov reassigned IGNITE-9412: --- Assignee: Alexey Platonov > [ML] GDB convergence by error support. > -- > > Key: IGNITE-9412 > URL: https://issues.apache.org/jira/browse/IGNITE-9412 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Major > Fix For: 2.7 > > > We need to support early training interruption when GDB has small error rate > on learning sample -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9348) ML examples improvements, follow-up to IGNITE-9297
[ https://issues.apache.org/jira/browse/IGNITE-9348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596254#comment-16596254 ] ASF GitHub Bot commented on IGNITE-9348: GitHub user oignatenko opened a pull request: https://github.com/apache/ignite/pull/4641 IGNITE-9348 ML examples improvements You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9348 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4641.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4641 commit 39c41f7e5b40e1825a40902b1add318fbe7c7cdc Author: Oleg Ignatenko Date: 2018-08-22T14:28:29Z IGNITE-9348 ML examples improvements - wip -- verified with diffs overview, executing the examples and studying execution logs commit 78b498252943cfc5ddab9dd7ebaeeea129643bd0 Author: Oleg Ignatenko Date: 2018-08-22T15:25:57Z IGNITE-9348 ML examples improvements - wip -- verified with diffs overview, executing the examples and studying execution logs commit 09d666d1b16293f8116e4021b9379b84203896cb Author: Oleg Ignatenko Date: 2018-08-22T15:34:52Z Merge branch 'master-ml' into ignite-9348 commit 11732c09637c1cecc3c0d7e035dfab711bf2751f Author: Oleg Ignatenko Date: 2018-08-23T15:09:21Z IGNITE-9348 ML examples improvements - wip (logging improved) -- verified with diffs overview, executing the examples and studying execution logs > ML examples improvements, follow-up to IGNITE-9297 > -- > > Key: IGNITE-9348 > URL: https://issues.apache.org/jira/browse/IGNITE-9348 > Project: Ignite > Issue Type: Task > Components: ml >Affects Versions: 2.6 >Reporter: Oleg Ignatenko >Assignee: Oleg Ignatenko >Priority: Major > Fix For: 2.7 > > > After IGNITE-9297 was resolved, ML examples were discussed with some folks > ([~skozlov], [~avolkov], [~chief]) in order to check if some other > improvements may be desired. This ticket is to address obtained feedback. > List of points that look worth taking care of includes (but is not limited > to): > * some examples javadocs still looks like missing details (eg random forest) > * some mentions of algorithms (eg kNN, gradient boosting etc) are better to > be linked to respective introductory articles in order to help less > experienced users easier grasp the examples > * presence of some examples doesn't look justified (eg LocalDatasetExample) > * logging in some examples looks sub-optimal, either too brief or too lengthy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9348) ML examples improvements, follow-up to IGNITE-9297
[
https://issues.apache.org/jira/browse/IGNITE-9348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596253#comment-16596253
]
Oleg Ignatenko commented on IGNITE-9348:
One more problem noticed is in {{LogisticRegressionSGDTrainerExample}}: it
currently shows extremely poor prediction results, this will be addressed per
separate ticket IGNITE-9421.
> ML examples improvements, follow-up to IGNITE-9297
> --
>
> Key: IGNITE-9348
> URL: https://issues.apache.org/jira/browse/IGNITE-9348
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Assignee: Oleg Ignatenko
>Priority: Major
> Fix For: 2.7
>
>
> After IGNITE-9297 was resolved, ML examples were discussed with some folks
> ([~skozlov], [~avolkov], [~chief]) in order to check if some other
> improvements may be desired. This ticket is to address obtained feedback.
> List of points that look worth taking care of includes (but is not limited
> to):
> * some examples javadocs still looks like missing details (eg random forest)
> * some mentions of algorithms (eg kNN, gradient boosting etc) are better to
> be linked to respective introductory articles in order to help less
> experienced users easier grasp the examples
> * presence of some examples doesn't look justified (eg LocalDatasetExample)
> * logging in some examples looks sub-optimal, either too brief or too lengthy
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
[
https://issues.apache.org/jira/browse/IGNITE-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596243#comment-16596243
]
Oleg Ignatenko commented on IGNITE-9421:
per preliminary discussion with [~zaleslaw] this is most likely bug in the
model, it should round results. This looks plausible because very similar
example {{LogRegressionMultiClassClassificationExample}} using a similar model
runs fine, meaning that other model is implemented correctly.
> ML Examples: LogisticRegressionSGDTrainerExample example result not correct
> ---
>
> Key: IGNITE-9421
> URL: https://issues.apache.org/jira/browse/IGNITE-9421
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Stepan Pilschikov
>Assignee: Aleksey Zinoviev
>Priority: Major
> Fix For: 2.7
>
>
> Running
> org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
> example
> Output:
> {code}
> >>> Absolute amount of errors 100
> >>> Accuracy 0.0
> >>> Confusion matrix is [[50, 50], [0, 0]]
> >>> -
> >>> Logistic regression model over partitioned dataset usage example
> >>> completed.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
[
https://issues.apache.org/jira/browse/IGNITE-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9421:
---
Ignite Flags: (was: Docs Required)
> ML Examples: LogisticRegressionSGDTrainerExample example result not correct
> ---
>
> Key: IGNITE-9421
> URL: https://issues.apache.org/jira/browse/IGNITE-9421
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Stepan Pilschikov
>Priority: Major
> Fix For: 2.7
>
>
> Running
> org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
> example
> Output:
> {code}
> >>> Absolute amount of errors 100
> >>> Accuracy 0.0
> >>> Confusion matrix is [[50, 50], [0, 0]]
> >>> -
> >>> Logistic regression model over partitioned dataset usage example
> >>> completed.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
[
https://issues.apache.org/jira/browse/IGNITE-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko reassigned IGNITE-9421:
--
Assignee: Aleksey Zinoviev
> ML Examples: LogisticRegressionSGDTrainerExample example result not correct
> ---
>
> Key: IGNITE-9421
> URL: https://issues.apache.org/jira/browse/IGNITE-9421
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Stepan Pilschikov
>Assignee: Aleksey Zinoviev
>Priority: Major
> Fix For: 2.7
>
>
> Running
> org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
> example
> Output:
> {code}
> >>> Absolute amount of errors 100
> >>> Accuracy 0.0
> >>> Confusion matrix is [[50, 50], [0, 0]]
> >>> -
> >>> Logistic regression model over partitioned dataset usage example
> >>> completed.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
[
https://issues.apache.org/jira/browse/IGNITE-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9421:
---
Fix Version/s: 2.7
> ML Examples: LogisticRegressionSGDTrainerExample example result not correct
> ---
>
> Key: IGNITE-9421
> URL: https://issues.apache.org/jira/browse/IGNITE-9421
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Stepan Pilschikov
>Priority: Major
> Fix For: 2.7
>
>
> Running
> org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
> example
> Output:
> {code}
> >>> Absolute amount of errors 100
> >>> Accuracy 0.0
> >>> Confusion matrix is [[50, 50], [0, 0]]
> >>> -
> >>> Logistic regression model over partitioned dataset usage example
> >>> completed.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9421) ML Examples: LogisticRegressionSGDTrainerExample example result not correct
Stepan Pilschikov created IGNITE-9421:
-
Summary: ML Examples: LogisticRegressionSGDTrainerExample example
result not correct
Key: IGNITE-9421
URL: https://issues.apache.org/jira/browse/IGNITE-9421
Project: Ignite
Issue Type: Bug
Components: ml
Affects Versions: 2.6
Reporter: Stepan Pilschikov
Running
org.apache.ignite.examples.ml.regression.logistic.binary.LogisticRegressionSGDTrainerExample
example
Output:
{code}
>>> Absolute amount of errors 100
>>> Accuracy 0.0
>>> Confusion matrix is [[50, 50], [0, 0]]
>>> -
>>> Logistic regression model over partitioned dataset usage example completed.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-5136) GridLogThrottle memory leak
[ https://issues.apache.org/jira/browse/IGNITE-5136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596236#comment-16596236 ] ASF GitHub Bot commented on IGNITE-5136: Github user SomeFire closed the pull request at: https://github.com/apache/ignite/pull/3236 > GridLogThrottle memory leak > --- > > Key: IGNITE-5136 > URL: https://issues.apache.org/jira/browse/IGNITE-5136 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Stanilovsky Evgeny >Assignee: Ryabov Dmitrii >Priority: Major > Fix For: 2.7 > > > class GridLogThrottle stores throttle info into map and noone clears it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9420) Move logical recovery phase outside of PME
Pavel Kovalenko created IGNITE-9420: --- Summary: Move logical recovery phase outside of PME Key: IGNITE-9420 URL: https://issues.apache.org/jira/browse/IGNITE-9420 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.5 Reporter: Pavel Kovalenko Fix For: 2.7 Currently, we perform logical recovery in PME here org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#restoreState We should move logical recovery before discovery manager will start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9419) Avoid saving cache configuration synchronously during PME
Pavel Kovalenko created IGNITE-9419: --- Summary: Avoid saving cache configuration synchronously during PME Key: IGNITE-9419 URL: https://issues.apache.org/jira/browse/IGNITE-9419 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.5 Reporter: Pavel Kovalenko Fix For: 2.7 Currently, we save cache configuration during PME at the activation phase here org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.CachesInfo#updateCachesInfo . We should avoid this, as it performs operations to a disk. We should save it asynchronously or lazy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9418) Avoid initialize file page store manager for caches during PME synchronously
Pavel Kovalenko created IGNITE-9418: --- Summary: Avoid initialize file page store manager for caches during PME synchronously Key: IGNITE-9418 URL: https://issues.apache.org/jira/browse/IGNITE-9418 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.5 Reporter: Pavel Kovalenko Assignee: Pavel Kovalenko Fix For: 2.7 Currently, we do creation for partition and index files during PME for starting caches at the beginning of org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager#readCheckpointAndRestoreMemory method. We should avoid this because sometimes it took a long time as we perform writing to disk. If the cache was registered during PME we should initialize page store lazy or asynchronously. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9417) IgniteSqlInsertIndexedValueBenchamrks failed with more than one driver
Ilya Suntsov created IGNITE-9417:
Summary: IgniteSqlInsertIndexedValueBenchamrks failed with more
than one driver
Key: IGNITE-9417
URL: https://issues.apache.org/jira/browse/IGNITE-9417
Project: Ignite
Issue Type: Bug
Reporter: Ilya Suntsov
I guess that we need to handle errors like bellow and continue to work OR each
driver must insert it's own key range:
{noformat}
ERROR: Shutting down benchmark driver to unexpected exception.
Type '--help' for usage.
javax.cache.CacheException: Duplicate key during INSERT [key=52]
<-->at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:676)
<-->at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:615)
<-->at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:385)
<-->at
org.apache.ignite.yardstick.cache.dml.IgniteSqlInsertIndexedValue1Benchmark.test(IgniteSqlInsertIndexedValue1Benchmark.java:38)
<-->at
org.yardstickframework.impl.BenchmarkRunner$2.run(BenchmarkRunner.java:178)
<-->at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.processors.query.IgniteSQLException: Duplicate key
during INSERT [key=52]
<-->at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.doInsert(DmlStatementsProcessor.java:803)
<-->at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.processDmlSelectResult(DmlStatementsProcessor.java:581)
<-->at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.executeUpdateStatement(DmlStatementsProcessor.java:539)
<-->at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFields(DmlStatementsProcessor.java:171)
<-->at
org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFieldsDistributed(DmlStatementsProcessor.java:345)
<-->at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.doRunPrepared(IgniteH2Indexing.java:1791)
<-->at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1755)
<-->at
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2107)
<-->at
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2102)
<-->at
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
<-->at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2650)
<-->at
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2116)
<-->at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:664)
<-->... 5 more
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9396) ML Examples: can't run examples, not enough dependencies in pom.xml
[
https://issues.apache.org/jira/browse/IGNITE-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596190#comment-16596190
]
Oleg Ignatenko edited comment on IGNITE-9396 at 8/29/18 10:57 AM:
--
I think I managed to reproduce the problem on my machine using IntelliJ Idea
(Ultimate 2018.1).
For that, I first checked what is the documented way to build examples per
[README.txt|https://github.com/apache/ignite/blob/master/examples/README.txt]:
"to start running you simply need to import provided `pom.xml` file into your
favourite IDE." In order to reproduce clean start I removed everything that was
cashed in my local maven repo and invalidated Idea caches. After that I tried
to build and launch an example and got compile errors similar to reported here
(specific example tested was {{KMeansClusterizationExample}} but it probably
doesn't really matter).
To find out how this can be corrected I decided to re-check how we build
[examples at
Teamcity|https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Examples]
because this way has proven to be clean and reliably working. Checking build
settings at Teamcity it turned out that prior to building examples they run a
maven (install) build for the Ignite. I tried the same on my machine and after
I did the install build of Ignite as described in
[DEVNOTES.txt|https://github.com/apache/ignite/blob/master/DEVNOTES.txt]
examples successfully loaded and built in my IDE and started working as
expected.
So far it looks like root issue is that README instructions in examples are
somewhat incomplete, possibly these can be expanded as follows (text for adding
is in _italic_ font):
{quote}...to start running you simply need to import provided `pom.xml` file
into your favourite IDE.
_Note working with examples assumes that you have Ignite already installed. If
you use source bundle this means it has built as described in DEVNOTES.txt in
Ignite project root._{quote}
[~dmagda] kindly agreed to discuss these matters on Thursday.
was (Author: oignatenko):
I think I managed to reproduce the problem on my machine using IntelliJ Idea
(Ultimate 2018.1).
For that, I first checked what is the documented way to build examples per
{{README.txt}}: "to start running you simply need to import provided `pom.xml`
file into your favourite IDE." In order to reproduce clean start I removed
everything that was cashed in my local maven repo and invalidated Idea caches.
After that I tried to build and launch an example and got compile errors
similar to reported here (specific example tested was
{{KMeansClusterizationExample}} but it probably doesn't really matter).
To find out how this can be corrected I decided to re-check how we build
[examples at
Teamcity|https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Examples]
because this way has proven to be clean and reliably working. Checking build
settings at Teamcity it turned out that prior to building examples they run a
maven (install) build for the Ignite. I tried the same on my machine and after
I did the install build of Ignite as described in
[DEVNOTES.txt|https://github.com/apache/ignite/blob/master/DEVNOTES.txt]
examples successfully loaded and built in my IDE and started working as
expected.
So far it looks like root issue is that README instructions in examples are
somewhat incomplete, possibly these can be expanded as follows (text for adding
is in _italic_ font):
{quote}...to start running you simply need to import provided `pom.xml` file
into your favourite IDE.
_Note working with examples assumes that you have Ignite already installed. If
you use source bundle this means it has built as described in DEVNOTES.txt in
Ignite project root._{quote}
[~dmagda] kindly agreed to discuss these matters on Thursday.
> ML Examples: can't run examples, not enough dependencies in pom.xml
> ---
>
> Key: IGNITE-9396
> URL: https://issues.apache.org/jira/browse/IGNITE-9396
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.7
>Reporter: Stepan Pilschikov
>Assignee: Oleg Ignatenko
>Priority: Major
>
> Trying to run ml-examples and can't compile example project because several
> dependencies didn't added in pom.xml
> Execution:
> Using JB IDEA Run->Run class to execute current class
> or
> `java -classpath [all classpaths which are listed in pom file] class name`
> or compile with maven also failed
> `mvn clean compile`
> Exception:
> Can't compile project. not enough dependencies
> Missing dependencies:
> {code:xml}
>
> com.github.fommil.netlib
> core
>
>
> org.springframework.data
> spring-data-commons
>
>
> org.springframework
>
[jira] [Updated] (IGNITE-7405) [ML] Distributed MLP cleanup/refactoring phase 2
[ https://issues.apache.org/jira/browse/IGNITE-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak updated IGNITE-7405: --- Summary: [ML] Distributed MLP cleanup/refactoring phase 2 (was: Distributed MLP cleanup/refactoring phase 2) > [ML] Distributed MLP cleanup/refactoring phase 2 > > > Key: IGNITE-7405 > URL: https://issues.apache.org/jira/browse/IGNITE-7405 > Project: Ignite > Issue Type: Bug > Components: ml >Affects Versions: 2.4 >Reporter: Artem Malykh >Assignee: Artem Malykh >Priority: Major > Fix For: 2.7 > > > All refactoring which is not done in IGNITE-7350. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-8867) [ML] Bagging on learning sample
[ https://issues.apache.org/jira/browse/IGNITE-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak updated IGNITE-8867: --- Summary: [ML] Bagging on learning sample (was: Bootstrapping for learning sample) > [ML] Bagging on learning sample > --- > > Key: IGNITE-8867 > URL: https://issues.apache.org/jira/browse/IGNITE-8867 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Oleg Ignatenko >Priority: Major > Fix For: 2.7 > > > Need to implement bootstrapping algorithm in Bagging-classifier -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9416) [ML] Update distribution approach for TensorFlow module
[ https://issues.apache.org/jira/browse/IGNITE-9416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak updated IGNITE-9416: --- Fix Version/s: 2.7 > [ML] Update distribution approach for TensorFlow module > --- > > Key: IGNITE-9416 > URL: https://issues.apache.org/jira/browse/IGNITE-9416 > Project: Ignite > Issue Type: Improvement > Components: ml >Affects Versions: 2.7 >Reporter: Anton Dmitriev >Priority: Major > Fix For: 2.7 > > > So far we package "ignite-tf.sh" into zip file in "ignite-tensorflow" module. > We need to define right approach to do it so that users are able to use it > easily the same way as other Ignite command line tools. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9414) [ML] Using sparce vectors in Tree-based algorithms.
[ https://issues.apache.org/jira/browse/IGNITE-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak updated IGNITE-9414: --- Priority: Minor (was: Major) > [ML] Using sparce vectors in Tree-based algorithms. > --- > > Key: IGNITE-9414 > URL: https://issues.apache.org/jira/browse/IGNITE-9414 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Priority: Minor > Fix For: 2.7 > > > We need to support sparce vectors in DecisionTrees, RF, GDB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9396) ML Examples: can't run examples, not enough dependencies in pom.xml
[
https://issues.apache.org/jira/browse/IGNITE-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596190#comment-16596190
]
Oleg Ignatenko commented on IGNITE-9396:
I think I managed to reproduce the problem on my machine using IntelliJ Idea
(Ultimate 2018.1).
For that, I first checked what is the documented way to build examples per
{{README.txt}}: "to start running you simply need to import provided `pom.xml`
file into your favourite IDE." In order to reproduce clean start I removed
everything that was cashed in my local maven repo and invalidated Idea caches.
After that I tried to build and launch an example and got compile errors
similar to reported here (specific example tested was
{{KMeansClusterizationExample}} but it probably doesn't really matter).
To find out how this can be corrected I decided to re-check how we build
[examples at
Teamcity|https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Examples]
because this way has proven to be clean and reliably working. Checking build
settings at Teamcity it turned out that prior to building examples they run a
maven (install) build for the Ignite. I tried the same on my machine and after
I did the install build of Ignite as described in
[DEVNOTES.txt|https://github.com/apache/ignite/blob/master/DEVNOTES.txt]
examples successfully loaded and built in my IDE and started working as
expected.
So far it looks like root issue is that README instructions in examples are
somewhat incomplete, possibly these can be expanded as follows (text for adding
is in _italic_ font):
{quote}...to start running you simply need to import provided `pom.xml` file
into your favourite IDE.
_Note working with examples assumes that you have Ignite already installed. If
you use source bundle this means it has built as described in DEVNOTES.txt in
Ignite project root._{quote}
[~dmagda] kindly agreed to discuss these matters on Thursday.
> ML Examples: can't run examples, not enough dependencies in pom.xml
> ---
>
> Key: IGNITE-9396
> URL: https://issues.apache.org/jira/browse/IGNITE-9396
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.7
>Reporter: Stepan Pilschikov
>Assignee: Oleg Ignatenko
>Priority: Major
>
> Trying to run ml-examples and can't compile example project because several
> dependencies didn't added in pom.xml
> Execution:
> Using JB IDEA Run->Run class to execute current class
> or
> `java -classpath [all classpaths which are listed in pom file] class name`
> or compile with maven also failed
> `mvn clean compile`
> Exception:
> Can't compile project. not enough dependencies
> Missing dependencies:
> {code:xml}
>
> com.github.fommil.netlib
> core
>
>
> org.springframework.data
> spring-data-commons
>
>
> org.springframework
> spring-beans
>
>
> org.springframework
> spring-context
>
>
> com.h2database
> h2
>
> {code}
> com.github.fommil.netlib, springframework.* and h2 need to run -
> org.apache.ignite.examples.ml.dataset.AlgorithmSpecificDatasetExample
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9416) [ML] Update distribution approach for TensorFlow module
Anton Dmitriev created IGNITE-9416: -- Summary: [ML] Update distribution approach for TensorFlow module Key: IGNITE-9416 URL: https://issues.apache.org/jira/browse/IGNITE-9416 Project: Ignite Issue Type: Improvement Components: ml Affects Versions: 2.7 Reporter: Anton Dmitriev So far we package "ignite-tf.sh" into zip file in "ignite-tensorflow" module. We need to define right approach to do it so that users are able to use it easily the same way as other Ignite command line tools. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9415) [ML] Using sparce vectors in LSQR and MLP
Yury Babak created IGNITE-9415: -- Summary: [ML] Using sparce vectors in LSQR and MLP Key: IGNITE-9415 URL: https://issues.apache.org/jira/browse/IGNITE-9415 Project: Ignite Issue Type: Improvement Components: ml Reporter: Yury Babak Fix For: 2.7 We need to investigate and apply sparce vectors support in BLAS for LSQR and MLP (or implement own version) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9415) [ML] Using sparce vectors in LSQR and MLP
[ https://issues.apache.org/jira/browse/IGNITE-9415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak updated IGNITE-9415: --- Priority: Minor (was: Major) > [ML] Using sparce vectors in LSQR and MLP > - > > Key: IGNITE-9415 > URL: https://issues.apache.org/jira/browse/IGNITE-9415 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Priority: Minor > Fix For: 2.7 > > > We need to investigate and apply sparce vectors support in BLAS for LSQR and > MLP (or implement own version) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9414) [ML] Using sparce vectors in Tree-based algorithms.
Yury Babak created IGNITE-9414: -- Summary: [ML] Using sparce vectors in Tree-based algorithms. Key: IGNITE-9414 URL: https://issues.apache.org/jira/browse/IGNITE-9414 Project: Ignite Issue Type: Improvement Components: ml Reporter: Yury Babak Fix For: 2.7 We need to support sparce vectors in DecisionTrees, RF, GDB -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9413) [ML] Learning rate optimization for GDB.
Yury Babak created IGNITE-9413: -- Summary: [ML] Learning rate optimization for GDB. Key: IGNITE-9413 URL: https://issues.apache.org/jira/browse/IGNITE-9413 Project: Ignite Issue Type: Improvement Components: ml Reporter: Yury Babak Fix For: 2.7 We need to support learning rate optimization while training for MSE-loss and Log-loss -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9412) [ML] GDB convergence by error support.
Yury Babak created IGNITE-9412: -- Summary: [ML] GDB convergence by error support. Key: IGNITE-9412 URL: https://issues.apache.org/jira/browse/IGNITE-9412 Project: Ignite Issue Type: Improvement Components: ml Reporter: Yury Babak Fix For: 2.7 We need to support early training interruption when GDB has small error rate on learning sample -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9411) Lock: make sure lock timeouts works fine
Vladimir Ozerov created IGNITE-9411: --- Summary: Lock: make sure lock timeouts works fine Key: IGNITE-9411 URL: https://issues.apache.org/jira/browse/IGNITE-9411 Project: Ignite Issue Type: Task Components: mvcc Reporter: Vladimir Ozerov Fix For: 2.7 In SQL it is not uncommon that locks are taken in arbitrary order, what may lead to deadlocks. Fair deadlock detector is good solution in monolithic databases - just analyze dependency graph and kill one of conflicting transactions. We have a ticket to implement distributed deadlock detector in Ignite [1]. However, this solution is rather complex and may bring some overhead. For now it is better to rely on some timeout (global or per-transaction), and rollback TX when it fails to lock certain entry for a long time. Probably we already have this feature in some form. Need to add verify that it works and add more tests if needed. [1] https://issues.apache.org/jira/browse/IGNITE-9322 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9410) MVCC: add support to thin clients
Vladimir Ozerov created IGNITE-9410: --- Summary: MVCC: add support to thin clients Key: IGNITE-9410 URL: https://issues.apache.org/jira/browse/IGNITE-9410 Project: Ignite Issue Type: Task Components: mvcc, thin client Reporter: Vladimir Ozerov Fix For: 2.8 Currently only ODBC and JDBC drivers support transactions. We need to add transactional API to .NET, Java, CPP, NodeJS and Python clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[
https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-7384:
Fix Version/s: 2.7
> MVCC TX: Support historical rebalance
> -
>
> Key: IGNITE-7384
> URL: https://issues.apache.org/jira/browse/IGNITE-7384
> Project: Ignite
> Issue Type: Bug
> Components: mvcc
>Reporter: Igor Seliverstov
>Assignee: Roman Kondakov
>Priority: Major
> Fix For: 2.7
>
>
> In case a node returns to topology it requests a delta instead of full
> partition, WAL-based iterator is used there
> ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}})
> WAL-based iterator doesn't contain MVCC versions which causes issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7991) MVCC TX Crash recovery
[ https://issues.apache.org/jira/browse/IGNITE-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-7991: Fix Version/s: 2.7 > MVCC TX Crash recovery > -- > > Key: IGNITE-7991 > URL: https://issues.apache.org/jira/browse/IGNITE-7991 > Project: Ignite > Issue Type: Task > Components: cache, mvcc >Reporter: Igor Seliverstov >Priority: Major > Fix For: 2.7 > > > Implement crash recovery for MVCC enabled caches -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9224) MVCC SQL: Cache metrics
[
https://issues.apache.org/jira/browse/IGNITE-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov reassigned IGNITE-9224:
---
Assignee: (was: Ivan Pavlukhin)
> MVCC SQL: Cache metrics
> ---
>
> Key: IGNITE-9224
> URL: https://issues.apache.org/jira/browse/IGNITE-9224
> Project: Ignite
> Issue Type: Improvement
> Components: mvcc, sql
>Reporter: Ivan Pavlukhin
>Priority: Major
> Fix For: 2.7
>
>
> We need to support {{CacheMetrics}} API for MVCC caches. Semantics should
> conform JCache specification. Only committed stores and removes should be
> counted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9265) MVCC TX: Two rows with the same key in one MERGE statement produce an exception
[
https://issues.apache.org/jira/browse/IGNITE-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov reassigned IGNITE-9265:
---
Assignee: (was: Andrew Mashenkov)
> MVCC TX: Two rows with the same key in one MERGE statement produce an
> exception
> ---
>
> Key: IGNITE-9265
> URL: https://issues.apache.org/jira/browse/IGNITE-9265
> Project: Ignite
> Issue Type: Bug
> Components: mvcc
>Reporter: Igor Seliverstov
>Priority: Major
>
> In case the operation like {{MERGE INTO INTEGER (_key, _val) KEY(_key) VALUES
> (1,1),(1,2)}} is called an exception is occurred.
> Correct behavior: each next update on the same key overwrites pervious one
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7764) MVCC TX: Locking protocol for Key-Value API
[ https://issues.apache.org/jira/browse/IGNITE-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-7764: Fix Version/s: 2.7 > MVCC TX: Locking protocol for Key-Value API > --- > > Key: IGNITE-7764 > URL: https://issues.apache.org/jira/browse/IGNITE-7764 > Project: Ignite > Issue Type: New Feature > Components: mvcc, sql >Reporter: Roman Kondakov >Assignee: Igor Seliverstov >Priority: Major > Fix For: 2.7 > > > We need to implement an MVCC-compatible locking protocol for Key-Value API. > At the moment during transactions with KV operations if entry we are going to > change is unlocked we do not check if it has been changed by the previous > transaction. See IGNITE-6935. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9224) MVCC SQL: Cache metrics
[
https://issues.apache.org/jira/browse/IGNITE-9224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-9224:
Fix Version/s: 2.7
> MVCC SQL: Cache metrics
> ---
>
> Key: IGNITE-9224
> URL: https://issues.apache.org/jira/browse/IGNITE-9224
> Project: Ignite
> Issue Type: Improvement
> Components: mvcc, sql
>Reporter: Ivan Pavlukhin
>Assignee: Ivan Pavlukhin
>Priority: Major
> Fix For: 2.7
>
>
> We need to support {{CacheMetrics}} API for MVCC caches. Semantics should
> conform JCache specification. Only committed stores and removes should be
> counted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-5935) MVCC TX: Tx recovery protocol
[ https://issues.apache.org/jira/browse/IGNITE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov reassigned IGNITE-5935: --- Assignee: Vladimir Ozerov (was: Igor Seliverstov) > MVCC TX: Tx recovery protocol > - > > Key: IGNITE-5935 > URL: https://issues.apache.org/jira/browse/IGNITE-5935 > Project: Ignite > Issue Type: Task > Components: cache, mvcc >Reporter: Semen Boikov >Assignee: Vladimir Ozerov >Priority: Major > Fix For: 2.7 > > > Tx recovery doesn't work properly for txs over MVCC enabled caches using > Cache API. It requires MvccSnapshot which may not be acquired at recovery > time. > Need to implement logic for checking whether snapshot was already gotten by > one of tx participants and use existing one, request and spread between > participants a new snapshot otherwise. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9402) IgnitePdsDiskErrorsRecoveringTest.testRecoveringOnWALWritingFail2 because of LogOnly mode
[ https://issues.apache.org/jira/browse/IGNITE-9402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596158#comment-16596158 ] ASF GitHub Bot commented on IGNITE-9402: GitHub user akalash opened a pull request: https://github.com/apache/ignite/pull/4640 IGNITE-9402 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9402 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4640.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4640 commit b145bb9521e94a1dd8da6775bd44d436cf41ae1b Author: Anton Kalashnikov Date: 2018-08-28T15:06:04Z IGNITE-9402 rewrited testRecoveringOnWALWritingFail2 commit f43c06a1148eb3bb3640561fb31094b10cb8a42d Author: Anton Kalashnikov Date: 2018-08-29T10:00:20Z IGNITE-9402 added message to assert commit d539ce0dc1023ee0a0b47ee7e1fbc8551f0738de Author: Anton Kalashnikov Date: 2018-08-29T10:01:34Z IGNITE-9402 remove tests from suit for test > IgnitePdsDiskErrorsRecoveringTest.testRecoveringOnWALWritingFail2 because of > LogOnly mode > - > > Key: IGNITE-9402 > URL: https://issues.apache.org/jira/browse/IGNITE-9402 > Project: Ignite > Issue Type: Test >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Labels: MakeTeamcityGreenAgain > > IgnitePdsDiskErrorsRecoveringTest.testRecoveringOnWALWritingFail2 failed > because it can lost last WAL data which have not flushed yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6532) Introduce preallocation in LFS files to avoid high fragmentation on filesystem level
[ https://issues.apache.org/jira/browse/IGNITE-6532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-6532: --- Fix Version/s: (was: 2.7) 2.8 > Introduce preallocation in LFS files to avoid high fragmentation on > filesystem level > > > Key: IGNITE-6532 > URL: https://issues.apache.org/jira/browse/IGNITE-6532 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.2 >Reporter: Ivan Rakov >Priority: Major > Fix For: 2.8 > > > Modern databases (Oracle, MySql) work with storage drive on physical level, > creating their own partition table and filesystem. > Ignite Persistent Store work with regular files. It appends new pages to > partition file once new pages are allocated and written on checkpoint. These > new pages can form one or several fragments on filesystem level. > As a result, after weeks of uptime, partition files can contain huge amount > of fragments. There were reports about 120 fragments in index.bin file on > XFS filesystem. > We can work this around by preallocating files in bigger chunks, e.g. 1000 > pages at a time. On the other hand, early allocation will increase LFS size > overhead, so we should consider reasonable heuristic for allocation. > Allocation should be performed on native level. Just writing a byte at > position (file_size + page_size * 1000) won't do it because XFS (and other > filesystems as well) has an optimization for that case. Missing range will be > just skipped. > Related article about filesystem internals: > https://blog.codecentric.de/en/2017/04/xfs-possible-memory-allocation-deadlock-kmem_alloc/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-6504) Very quick checkpoint can cause AssertionError on next start from LFS
[
https://issues.apache.org/jira/browse/IGNITE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Rakov updated IGNITE-6504:
---
Fix Version/s: (was: 2.7)
2.8
> Very quick checkpoint can cause AssertionError on next start from LFS
> -
>
> Key: IGNITE-6504
> URL: https://issues.apache.org/jira/browse/IGNITE-6504
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.1
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Major
> Labels: newbie
> Fix For: 2.8
>
>
> Checkpoint markers are compared using their timestamps. If checkpoint took
> less than 1 millisecond, two subsequent markers will have same timestamp,
> which will lead to error:
> {noformat}
> java.lang.AssertionError:
> o1=/data/teamcity/tmpfs/work/db/127_0_0_1_47503/cp/1506338145591-c4f23411-e1b1-4468-856a-4419003bba93-END.bin,
>
> o2=/data/teamcity/tmpfs/work/db/127_0_0_1_47503/cp/1506338145591-f76c023b-9982-40d7-a1eb-855a33b710f2-END.bin
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$4.compare(GridCacheDatabaseSharedManager.java:216)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$4.compare(GridCacheDatabaseSharedManager.java:195)
> at java.util.TimSort.binarySort(TimSort.java:265)
> at java.util.TimSort.sort(TimSort.java:208)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$CheckpointHistory.loadHistory(GridCacheDatabaseSharedManager.java:2704)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$CheckpointHistory.access$2600(GridCacheDatabaseSharedManager.java:2685)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1468)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:562)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:722)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:613)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2289)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-8572) StandaloneWalRecordsIterator may throw NPE if compressed WAL segment is empty
[
https://issues.apache.org/jira/browse/IGNITE-8572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Rakov updated IGNITE-8572:
---
Fix Version/s: (was: 2.7)
2.8
> StandaloneWalRecordsIterator may throw NPE if compressed WAL segment is empty
> -
>
> Key: IGNITE-8572
> URL: https://issues.apache.org/jira/browse/IGNITE-8572
> Project: Ignite
> Issue Type: New Feature
> Components: persistence
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Major
> Fix For: 2.8
>
>
> In case ZIP archive with WAL segment doesn't contain any ZIP entries, attempt
> to iterate through it with standalone WAL iterator will throw NPE:
> {noformat}
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.persistence.file.UnzipFileIO.(UnzipFileIO.java:53)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:265)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.reader.StandaloneWalRecordsIterator.advanceSegment(StandaloneWalRecordsIterator.java:262)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:155)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.reader.StandaloneWalRecordsIterator.(StandaloneWalRecordsIterator.java:111)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.reader.IgniteWalIteratorFactory.iteratorArchiveDirectory(IgniteWalIteratorFactory.java:156)
> ... 6 more
> {noformat}
> We should throw excpetion with descriptive error message instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-8299) Optimize allocations and CPU consumption in active page replacement scenario
[ https://issues.apache.org/jira/browse/IGNITE-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-8299: --- Fix Version/s: (was: 2.7) 2.8 > Optimize allocations and CPU consumption in active page replacement scenario > > > Key: IGNITE-8299 > URL: https://issues.apache.org/jira/browse/IGNITE-8299 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Ivan Rakov >Priority: Major > Fix For: 2.8 > > Attachments: loader-2018-04-17T12-12-21.jfr, > loader-2018-04-17T12-12-21.jfr, loader-2018-04-17T15-10-52.jfr, > loader-2018-04-17T15-10-52.jfr > > > Ignite performance significantly decreases when total size of local data is > much greater than size of RAM. It can be explained by change of disk access > pattern (random reads + random writes is complex even for SSDs), but after > analysis of persistence code and JFRs it's clear that there's still room for > optimization. > The following possible optimizations should be investigated: > 1) PageMemoryImpl.Segment#partGeneration performs allocation of > GroupPartitionId during HashMap.get - we can get rid of it. > 2) LoadedPagesMap#getNearestAt is invoked at least 5 times in > PageMemoryImpl.Segment#removePageForReplacement. It performs two allocations > - we can get rid of it. > 3) If one of 5 evict candidates was erroneous, we'll find 5 new ones - we can > reuse remaining 4 instead. > JFRs that highlight excessive CPU usage by page replacement code is attached. > See 1st and 3rd positions in "Hot Methods" section: > Stack Trace Sample CountPercentage(%) > PageMemoryImpl.acquirePage(int, long, boolean)4 963 19,73 > scala.Some.equals(Object) 4 932 19,606 > java.util.HashMap.getNode(int, Object)3 236 12,864 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
