[jira] [Updated] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl

2019-09-25 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12197:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
> --
>
> Key: IGNITE-12197
> URL: https://issues.apache.org/jira/browse/IGNITE-12197
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Minor
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> IGNITE-12027 introduces possible bug  due to incorrect way for getting value 
> of persistent enabled property in {{CacheGroupMetricsImpl}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12231) RollbackRecord's must be flushed after logging to WAL

2019-09-25 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12231:


 Summary: RollbackRecord's must be flushed after logging to WAL
 Key: IGNITE-12231
 URL: https://issues.apache.org/jira/browse/IGNITE-12231
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.8


Every record or records batch logged to WAL must call {{wal().flush()}} in 
order to save data on storage device.

{{TxPartitionCounterStateOnePrimaryTwoBackupsHistoryRebalanceTest.testPartialPrepare_2TX_1_1}}
 test fails periodically with disabled MMAP.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12199) WAL record with data entries doesn't flushes on backups for transactional cache

2019-09-23 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12199:
-
Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> WAL record with data entries doesn't flushes on backups for transactional 
> cache
> ---
>
> Key: IGNITE-12199
> URL: https://issues.apache.org/jira/browse/IGNITE-12199
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> WAL record with data entries doesn't flushes on backups for transactional 
> cache.
> This issue can be reproduced for example by 
> {{TxPartitionCounterStateConsistencyTest.testSingleThreadedUpdateOrder}} test 
> with disabled MMAP mode.
> Problem place in code is {{GridDistributedTxRemoteAdapter#commitIfLocked}} 
> where {{wal.log()}} doesn't assign returned file pointer to the {{ptr}} 
> variable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12145) [IEP-35] Monitoring list engine

2019-09-20 Thread Andrey Gura (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934307#comment-16934307
 ] 

Andrey Gura commented on IGNITE-12145:
--

[~NIzhikov] Nikolay, looks good to me. Thanks for contribution and thanks to 
all who was involved.

> [IEP-35] Monitoring list engine
> ---
>
> Key: IGNITE-12145
> URL: https://issues.apache.org/jira/browse/IGNITE-12145
> Project: Ignite
>  Issue Type: Sub-task
>Affects Versions: 2.7.5
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 19h 10m
>  Remaining Estimate: 0h
>
> The base engine for the monitoring list.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-09-19 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reassigned IGNITE-11927:


Assignee: Andrey Gura  (was: Nikolay Izhikov)

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Andrey Gura
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl

2019-09-19 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12197:
-
Labels: IEP-35  (was: )

> Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
> --
>
> Key: IGNITE-12197
> URL: https://issues.apache.org/jira/browse/IGNITE-12197
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> IGNITE-12027 introduces possible bug  due to incorrect way for getting value 
> of persistent enabled property in {{CacheGroupMetricsImpl}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12199) WAL record with data entries doesn't flushes on backups for transactional cache

2019-09-19 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12199:


 Summary: WAL record with data entries doesn't flushes on backups 
for transactional cache
 Key: IGNITE-12199
 URL: https://issues.apache.org/jira/browse/IGNITE-12199
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.8


WAL record with data entries doesn't flushes on backups for transactional cache.

This issue can be reproduced for example by 
{{TxPartitionCounterStateConsistencyTest.testSingleThreadedUpdateOrder}} test 
with disabled MMAP mode.

Problem place in code is {{GridDistributedTxRemoteAdapter#commitIfLocked}} 
where {{wal.log()}} doesn't assign returned file pointer to the {{ptr}} 
variable.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler

2019-09-18 Thread Andrey Gura (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932885#comment-16932885
 ] 

Andrey Gura commented on IGNITE-12198:
--

By design, accordingly to configured segmentation policy. 

> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes 
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler 
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null), 
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null), 
> stopNodeHnd);
> break; {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler

2019-09-18 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-12198.
--
Release Note: By design, accordingly to configured segmentation policy. 
  Resolution: Won't Fix

> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes 
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler 
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null), 
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null), 
> stopNodeHnd);
> break; {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler

2019-09-18 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12198:
-
Release Note:   (was: By design, accordingly to configured segmentation 
policy. )

> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes 
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler 
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null), 
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null), 
> stopNodeHnd);
> break; {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl

2019-09-18 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12197:
-
Description: IGNITE-12027 introduces possible bug  due to incorrect way for 
getting value of persistent enabled property in {{CacheGroupMetricsImpl}}  
(was: IGNITE-12027 introduce possible bug  due to incorrect way for getting 
value of persistent enabled property in {{CacheGroupMetricsImpl}})

> Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
> --
>
> Key: IGNITE-12197
> URL: https://issues.apache.org/jira/browse/IGNITE-12197
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
>
> IGNITE-12027 introduces possible bug  due to incorrect way for getting value 
> of persistent enabled property in {{CacheGroupMetricsImpl}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl

2019-09-18 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12197:


 Summary: Incorrect way for getting value of persistent enabled in 
CacheGroupMetricsImpl
 Key: IGNITE-12197
 URL: https://issues.apache.org/jira/browse/IGNITE-12197
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.8


IGNITE-12027 introduce possible bug  due to incorrect way for getting value of 
persistent enabled property in {{CacheGroupMetricsImpl}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-12172) [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite

2019-09-16 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12172:
-
Labels: IEP-35  (was: )

> [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite
> --
>
> Key: IGNITE-12172
> URL: https://issues.apache.org/jira/browse/IGNITE-12172
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> {{OpenCensusMetricExporterSpiTest}} isn't added to any test suite. As result 
> tests do not run on TC.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12172) [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite

2019-09-16 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12172:


 Summary: [IEP-35] OpenCensusMetricExporterSpiTest isn't added to 
any test suite
 Key: IGNITE-12172
 URL: https://issues.apache.org/jira/browse/IGNITE-12172
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
 Fix For: 2.8


{{OpenCensusMetricExporterSpiTest}} isn't added to any test suite. As result 
tests do not run on TC.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Commented] (IGNITE-12171) Remove the MetricRegistry.remove(String name) method.

2019-09-16 Thread Andrey Gura (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930419#comment-16930419
 ] 

Andrey Gura commented on IGNITE-12171:
--

Will done as part of IGNITE-11927 issue. Metric registries will be immutable. 
See also dev-list topic 
http://apache-ignite-developers.2346864.n4.nabble.com/IEP-35-Metrics-management-in-Ignite-td43243.html


> Remove the MetricRegistry.remove(String name) method.
> -
>
> Key: IGNITE-12171
> URL: https://issues.apache.org/jira/browse/IGNITE-12171
> Project: Ignite
>  Issue Type: Task
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I suggest removing the unused {{MetricRegistry.remove(String name)}} method 
> due to we can remove a subset of metrics using the 
> {{GridMetricManager.remove(String regName)}} method with notifying mechanism. 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Resolved] (IGNITE-2636) Server cache metrics for put-get-remove avg time are incorrect for case when request sent from client

2019-09-09 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-2636.
-
Resolution: Duplicate

> Server cache metrics for put-get-remove avg time are incorrect for case when 
> request sent from client
> -
>
> Key: IGNITE-2636
> URL: https://issues.apache.org/jira/browse/IGNITE-2636
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Vladimir Ershov
>Priority: Major
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Server cache metrics for put-get-remove avg time are incorrect for case when 
> request sent from client.
> We should add methods like CacheMetrics#addPutAndGetTimeNanos for all flows, 
> when requests for cache modifications are processed. For all type of caches.
> This will fix CacheMetricsForClusterGroupSelfTest



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Commented] (IGNITE-12127) WAL writer may close file IO with unflushed changes when MMAP is disabled

2019-09-05 Thread Andrey Gura (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923405#comment-16923405
 ] 

Andrey Gura commented on IGNITE-12127:
--

[~DmitriyGovorukhin] LGTM. Thanks for contribution! Please merge to master 
branch.

> WAL writer may close file IO with unflushed changes when MMAP is disabled
> -
>
> Key: IGNITE-12127
> URL: https://issues.apache.org/jira/browse/IGNITE-12127
> Project: Ignite
>  Issue Type: Bug
>Reporter: Dmitriy Govorukhin
>Assignee: Dmitriy Govorukhin
>Priority: Critical
> Fix For: 2.7.6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Most likely the issue manifests itself as the following critical error:
> {code}
> 2019-08-27 14:52:31.286 ERROR 26835 --- [wal-write-worker%null-#447] ROOT : 
> Critical system error detected. Will be handled accordingly to configured 
> handler [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler, 
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
> o.a.i.i.processors.cache.persistence.StorageException: Failed to write 
> buffer.]]
> org.apache.ignite.internal.processors.cache.persistence.StorageException: 
> Failed to write buffer.
> at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.writeBuffer(FileWriteAheadLogManager.java:3444)
>  [ignite-core-2.5.7.jar!/:2.5.7]
> at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.body(FileWriteAheadLogManager.java:3249)
>  [ignite-core-2.5.7.jar!/:2.5.7]
> at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) 
> [ignite-core-2.5.7.jar!/:2.5.7]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201]
> Caused by: java.nio.channels.ClosedChannelException: null
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110) 
> ~[na:1.8.0_201]
> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:253) 
> ~[na:1.8.0_201]
> at 
> org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.position(RandomAccessFileIO.java:48)
>  ~[ignite-core-2.5.7.jar!/:2.5.7]
> at 
> org.apache.ignite.internal.processors.cache.persistence.file.FileIODecorator.position(FileIODecorator.java:41)
>  ~[ignite-core-2.5.7.jar!/:2.5.7]
> at 
> org.apache.ignite.internal.processors.cache.persistence.file.AbstractFileIO.writeFully(AbstractFileIO.java:111)
>  ~[ignite-core-2.5.7.jar!/:2.5.7]
> at 
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.writeBuffer(FileWriteAheadLogManager.java:3437)
>  [ignite-core-2.5.7.jar!/:2.5.7]
> ... 3 common frames omitted
> {code}
> It appears that there following sequence is possible:
>  * Thread A attempts to log a large record which does not fit segment, 
> {{addRecord}} fails and the thread A starts segment rollover. I successfully 
> runs {{flushOrWait(null)}} and gets de-scheduled before adding switch segment 
> record
>  * Thread B attempts to log another record, which fits exactly till the end 
> of the current segment. The record is added to the buffer
>  * Thread A resumes and fails to add the switch segment record. No flush is 
> performed and the thread immediately proceeds for wal-writer close
>  * WAL writer thread wakes up, sees that there is a CLOSE request, closes the 
> file IO and immediately proceeds to write unflushed changes causing the 
> exception.
> Unconditional flush after switch segment record write should fix the issue.
> Besides the bug itself, I suggest the following changes to the 
> {{FileWriteHandleImpl}} ({{FileWriteAheadLogManager}} in earlier versions):
>  * There is an {{fsync(filePtr)}} call inside {{close()}}; however, 
> {{fsync()}} checks the {{stop}} flag (which is set inside {{close}}) and 
> returns immediately after {{flushOrWait()}} if the flag is set - this is very 
> confusing. After all, the {{close()}} itself explicitly calls {{force}} after 
> flush
>  * There is an ignored IO exception in mmap mode - this should be propagated 
> to the failure handler
>  * In WAL writer, we check for file CLOSE and then attemp to write to 
> (possibly) the same write handle - write should be always before close
>  * In WAL writer, there are racy reads of current handle - it would be better 
> if we read the current handle once and then operate on it during the whole 
> loop iteration



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Resolved] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs

2019-08-30 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-12130.
--
Resolution: Won't Fix

> Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
> 
>
> Key: IGNITE-12130
> URL: https://issues.apache.org/jira/browse/IGNITE-12130
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> Open Census metric exporter and tracing SPI's have incorrect packages 
> structure: {{org.apache.ignite.opencensus.spi}}.
> Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed 
> under org.apache.ignite.spi}} package.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Commented] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs

2019-08-30 Thread Andrey Gura (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919691#comment-16919691
 ] 

Andrey Gura commented on IGNITE-12130:
--

[~NSAmelchev] You are right! Closed as wont fix.

> Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
> 
>
> Key: IGNITE-12130
> URL: https://issues.apache.org/jira/browse/IGNITE-12130
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> Open Census metric exporter and tracing SPI's have incorrect packages 
> structure: {{org.apache.ignite.opencensus.spi}}.
> Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed 
> under org.apache.ignite.spi}} package.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Commented] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager

2019-08-30 Thread Andrey Gura (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919683#comment-16919683
 ] 

Andrey Gura commented on IGNITE-12131:
--

[~ilyak] In my opinion, mmap is just implementation detail, so it isn't good 
idea to expose it on public configuration API. There are ideas about WAL 
refactoring that can lead to creation of different implementations. 

> MMAP mode should be disabled by default for WAL manager
> ---
>
> Key: IGNITE-12131
> URL: https://issues.apache.org/jira/browse/IGNITE-12131
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
>
> MMAP mode has a bunch of problems (see for example 
> http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ )
> Users are increasingly stumbling over these issues especially in virtualized 
> environments.
> MMAP should be disabled by default because it requires additional care from 
> user stand point.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Updated] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager

2019-08-30 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12131:
-
Description: 
MMAP mode has a bunch of problems (see for example 
http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ )

Users are increasingly stumbling over these issues especially in virtualized 
environments.

MMAP should be disabled by default because it requires additional care from 
user stand point.

  was:
MMAP mode has a bunch of problems (see for example 
http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ )

Users are increasingly stumbling over these issues especially in virtualized 
environments.

MMAP should be disabled by default because it require additional care from user 
stand point.


> MMAP mode should be disabled by default for WAL manager
> ---
>
> Key: IGNITE-12131
> URL: https://issues.apache.org/jira/browse/IGNITE-12131
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
>
> MMAP mode has a bunch of problems (see for example 
> http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ )
> Users are increasingly stumbling over these issues especially in virtualized 
> environments.
> MMAP should be disabled by default because it requires additional care from 
> user stand point.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Assigned] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager

2019-08-30 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reassigned IGNITE-12131:


Assignee: Andrey Gura

> MMAP mode should be disabled by default for WAL manager
> ---
>
> Key: IGNITE-12131
> URL: https://issues.apache.org/jira/browse/IGNITE-12131
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
>
> MMAP mode has a bunch of problems (see for example 
> http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ )
> Users are increasingly stumbling over these issues especially in virtualized 
> environments.
> MMAP should be disabled by default because it require additional care from 
> user stand point.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager

2019-08-30 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12131:


 Summary: MMAP mode should be disabled by default for WAL manager
 Key: IGNITE-12131
 URL: https://issues.apache.org/jira/browse/IGNITE-12131
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
 Fix For: 2.8


MMAP mode has a bunch of problems (see for example 
http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ )

Users are increasingly stumbling over these issues especially in virtualized 
environments.

MMAP should be disabled by default because it require additional care from user 
stand point.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Updated] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs

2019-08-30 Thread Andrey Gura (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12130:
-
Description: 
Open Census metric exporter and tracing SPI's have incorrect packages 
structure: {{org.apache.ignite.opencensus.spi}}.

Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed 
under org.apache.ignite.spi}} package.

  was:
Open Census metric exporter and tracing SPI's have incorrect packages 
structure: {{org.apache.ignite.opencensus.spi}}.

Shold be {{org.apache.ignite.spi.opencensus}}.


> Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
> 
>
> Key: IGNITE-12130
> URL: https://issues.apache.org/jira/browse/IGNITE-12130
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> Open Census metric exporter and tracing SPI's have incorrect packages 
> structure: {{org.apache.ignite.opencensus.spi}}.
> Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed 
> under org.apache.ignite.spi}} package.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Created] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs

2019-08-30 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12130:


 Summary: Incorrect packages structure for OpenCensus metric 
exporter and tracing SPIs
 Key: IGNITE-12130
 URL: https://issues.apache.org/jira/browse/IGNITE-12130
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
 Fix For: 2.8


Open Census metric exporter and tracing SPI's have incorrect packages 
structure: {{org.apache.ignite.opencensus.spi}}.

Shold be {{org.apache.ignite.spi.opencensus}}.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)


[jira] [Commented] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.

2019-08-13 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906239#comment-16906239
 ] 

Andrey Gura commented on IGNITE-12046:
--

[~NIzhikov] Thanks! No we shouldn't. Only {{LongMetric}} interface had this 
methods.

> [IEP-35] LongMetric interface contains redundant methods.
> -
>
> Key: IGNITE-12046
> URL: https://issues.apache.org/jira/browse/IGNITE-12046
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Trivial
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The following methods on {{LongMetric}} interface a redundant and have the 
> same functionality:
> * {{get}}
> * {{longValue}}
> Mentioned methods should be removed.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.

2019-08-12 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905668#comment-16905668
 ] 

Andrey Gura commented on IGNITE-12046:
--

[~NIzhikov] Could you please review?

> [IEP-35] LongMetric interface contains redundant methods.
> -
>
> Key: IGNITE-12046
> URL: https://issues.apache.org/jira/browse/IGNITE-12046
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Trivial
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The following methods on {{LongMetric}} interface a redundant and have the 
> same functionality:
> * {{get}}
> * {{longValue}}
> Mentioned methods should be removed.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Issue Comment Deleted] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.

2019-08-12 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12046:
-
Comment: was deleted

(was: {panel:title=Branch: [pull/6751/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 6{color} [[tests 0 Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=4492559]]

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4492028&buildTypeId=IgniteTests24Java8_RunAll])

> [IEP-35] LongMetric interface contains redundant methods.
> -
>
> Key: IGNITE-12046
> URL: https://issues.apache.org/jira/browse/IGNITE-12046
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Trivial
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The following methods on {{LongMetric}} interface a redundant and have the 
> same functionality:
> * {{get}}
> * {{longValue}}
> Mentioned methods should be removed.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Resolved] (IGNITE-12016) Is there option to increase or decrease Ignite SQL Table backup count after its initialized?

2019-08-12 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-12016.
--
Resolution: Not A Bug

> Is there option to increase or decrease Ignite SQL Table backup count after 
> its initialized?
> 
>
> Key: IGNITE-12016
> URL: https://issues.apache.org/jira/browse/IGNITE-12016
> Project: Ignite
>  Issue Type: New Feature
>Reporter: hirik
>Priority: Major
>
> Hi, 
> We are working in a dynamic environment, based on the new nodes we should 
> update the backup count of existing SQL tables. is there any api available to 
> support this requirement? 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12011) MetricUpdater timeout can't be configured

2019-08-12 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12011:
-
Priority: Minor  (was: Major)

> MetricUpdater timeout can't be configured
> -
>
> Key: IGNITE-12011
> URL: https://issues.apache.org/jira/browse/IGNITE-12011
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Max Shonichev
>Priority: Minor
> Fix For: 3.0
>
>
> {noformat}
> /** Metrics update frequency. */
> private static final long METRICS_UPDATE_FREQ = 3000;
> ...
> /** {@inheritDoc} */
> @Override protected void onKernalStart0() throws IgniteCheckedException {
> metricsUpdateTask = ctx.timeout().schedule(new MetricsUpdater(), 
> METRICS_UPDATE_FREQ, METRICS_UPDATE_FREQ);
> }
> {noformat}
> Would be great if metric updater timeout was externally configured.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.

2019-08-06 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-12046:


 Summary: [IEP-35] LongMetric interface contains redundant methods.
 Key: IGNITE-12046
 URL: https://issues.apache.org/jira/browse/IGNITE-12046
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
 Fix For: 2.8


The following methods on {{LongMetric}} interface a redundant and have the same 
functionality:

* {{get}}
* {{longValue}}

Mentioned methods should be removed.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-11925) [IEP-35] Migrage QueryMetrics

2019-08-06 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-11925:
-
Fix Version/s: 2.8

> [IEP-35] Migrage QueryMetrics
> -
>
> Key: IGNITE-11925
> URL: https://issues.apache.org/jira/browse/IGNITE-11925
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> After merging of IGNITE-11848 we should migrate `QueryMetrics` to the new 
> metric framework.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Assigned] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read

2019-08-06 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reassigned IGNITE-11687:


Assignee: Dmitriy Govorukhin  (was: Andrey Gura)

> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Dmitriy Govorukhin
>Priority: Critical
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been 
> already updated, concurrent WAL replay will attempt to read the incompletely 
> written record and since {{end}} is not null, iterator will fail with CRC 
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-12045) [IEP-35] JmxExporterSpi has too broad name.

2019-08-05 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-12045:


 Summary: [IEP-35] JmxExporterSpi has too broad name.
 Key: IGNITE-12045
 URL: https://issues.apache.org/jira/browse/IGNITE-12045
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Gura
 Fix For: 2.8


{{JmxExporterSpi}} has too broad name. It should be renamed to 
{{JmxMetricExporterSpi}}.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12044) [IEP-35] JmxExporterSpi displays histogram values incorrectly

2019-08-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12044:
-
Description: 
{{JmxExporterSpi}} exposes histogram values incorrectly. It looks like:

{noformat}
durationHistogramlong[5]
{noformat}

I think exporter should register attribute for each histogram bucket. Something 
like this:

{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}



  was:
JmxExporterSpi exposes histogram values incorrectly. It looks like:

{noformat}
durationHistogramlong[5]
{noformat}

I think exporter should register attribute for each histogram bucket. Something 
like this:

{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}




> [IEP-35] JmxExporterSpi displays histogram values incorrectly
> -
>
> Key: IGNITE-12044
> URL: https://issues.apache.org/jira/browse/IGNITE-12044
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> {{JmxExporterSpi}} exposes histogram values incorrectly. It looks like:
> {noformat}
> durationHistogramlong[5]
> {noformat}
> I think exporter should register attribute for each histogram bucket. 
> Something like this:
> {noformat}
> durationHistogram.10 0
> durationHistogram.50 0
> durationHistogram.1000
> durationHistogram.2500
> durationHistogram.5000
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12044) [IEP-35] JmxExporterSpi displays histogram values incorrectly

2019-08-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12044:
-
Description: 
JmxExporterSpi exposes histogram values incorrectly. It looks like:

{noformat}
durationHistogramlong[5]
{noformat}

I think exporter should register attribute for each histogram bucket. Something 
like this:

{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}



  was:
JmxExporterSpi exposes histogram values incorrectly. It looks like:

{noformat}
durationHistogramlong[5]
{noformat}

I think exporter should register attribute for each histogram bucket. Something 
like this:

{noformat}
durationHistogram.10  0
durationHistogram.50  0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}




> [IEP-35] JmxExporterSpi displays histogram values incorrectly
> -
>
> Key: IGNITE-12044
> URL: https://issues.apache.org/jira/browse/IGNITE-12044
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> JmxExporterSpi exposes histogram values incorrectly. It looks like:
> {noformat}
> durationHistogramlong[5]
> {noformat}
> I think exporter should register attribute for each histogram bucket. 
> Something like this:
> {noformat}
> durationHistogram.10 0
> durationHistogram.50 0
> durationHistogram.1000
> durationHistogram.2500
> durationHistogram.5000
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-12044) [IEP-35] JmxExporterSpi displays histogram values incorrectly

2019-08-05 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-12044:


 Summary: [IEP-35] JmxExporterSpi displays histogram values 
incorrectly
 Key: IGNITE-12044
 URL: https://issues.apache.org/jira/browse/IGNITE-12044
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Gura
 Fix For: 2.8


JmxExporterSpi exposes histogram values incorrectly. It looks like:

{noformat}
durationHistogramlong[5]
{noformat}

I think exporter should register attribute for each histogram bucket. Something 
like this:

{noformat}
durationHistogram.10  0
durationHistogram.50  0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}





--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12043) Metrics: JMX exporter reports incorrect description.

2019-08-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12043:
-
Labels: IEP-35 newbie  (was: IEP-35)

> Metrics: JMX exporter reports incorrect description.
> 
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
>  Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
>  Labels: IEP-35, newbie
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human 
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see 
> {{description = .}} instead of human readable 
> description, specified by metric developer.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12043) [IEP-35] Metrics: JMX exporter reports incorrect description.

2019-08-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12043:
-
Summary: [IEP-35] Metrics: JMX exporter reports incorrect description.  
(was: Metrics: JMX exporter reports incorrect description.)

> [IEP-35] Metrics: JMX exporter reports incorrect description.
> -
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
>  Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
>  Labels: IEP-35, newbie
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human 
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see 
> {{description = .}} instead of human readable 
> description, specified by metric developer.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11987) [IEP-35] Add ability to configure metrics

2019-08-05 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900123#comment-16900123
 ] 

Andrey Gura commented on IGNITE-11987:
--

[~NIzhikov] I've reviewed PR. See my comments on GitHub. Also I followed up 
discussion on dev list.

> [IEP-35] Add ability to configure metrics
> -
>
> Key: IGNITE-11987
> URL: https://issues.apache.org/jira/browse/IGNITE-11987
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12043) Metrics: JMX exporter reports incorrect description.

2019-08-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12043:
-
Labels: IEP-35  (was: )

> Metrics: JMX exporter reports incorrect description.
> 
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
>  Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human 
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see 
> {{description = .}} instead of human readable 
> description, specified by metric developer.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12043) Metrics: JMX exporter reports incorrect description.

2019-08-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12043:
-
Fix Version/s: 2.8

> Metrics: JMX exporter reports incorrect description.
> 
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
>  Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human 
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see 
> {{description = .}} instead of human readable 
> description, specified by metric developer.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-12028) [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics exporter

2019-08-05 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900012#comment-16900012
 ] 

Andrey Gura commented on IGNITE-12028:
--

[~NSAmelchev] [~NIzhikov]

Guys, could you please avoid in the future javadoc like this:

{code:java}
/** @return Rate time interval in milliseconds. */
public long rateTimeInterval() {
return cntr.rateTimeInterval;
}
{code}

It is public method so javadoc must contain method description and additionaly 
{{return tag}}.

> [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics 
> exporter
> 
>
> Key: IGNITE-12028
> URL: https://issues.apache.org/jira/browse/IGNITE-12028
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Amelchev Nikita
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {{HitRateMetric}} allows to get only counter value while it would be useful 
> to get also {{rateTimeInterval}} in order to export this value as part of 
> metric name. 
> For example look at cache metric {{RebalancingKeysRate}}. Value of this 
> measurement could be exported as smth like this: 
> {{cache..RebalancingKeysRate. = }}.
> So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of 
> {{LongMetric}} interface.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-10654) Report in case of creating index with already existing fields collection.

2019-08-01 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898082#comment-16898082
 ] 

Andrey Gura commented on IGNITE-10654:
--

[~zstan] LGTM. Merged to master branch. Thanks for contribution!

> Report in case of creating index with already existing fields collection.
> -
>
> Key: IGNITE-10654
> URL: https://issues.apache.org/jira/browse/IGNITE-10654
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 2.7
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Report in log if new index creating with already existing fields collection.
> for example, need to log warn here:
> {code:java}
> cache.query(new SqlFieldsQuery("create index \"idx1\" on Val(keyStr, 
> keyLong)"));
> cache.query(new SqlFieldsQuery("create index \"idx3\" on Val(keyStr, 
> keyLong)"));
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-08-01 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898076#comment-16898076
 ] 

Andrey Gura edited comment on IGNITE-11927 at 8/1/19 1:13 PM:
--

[~NIzhikov] Not certainly in that way. WE don't need NO_OP_METRIC concept 
because we don't disable metric itself, we disable whole set of metrics that is 
represented by metric registry. So metric registry also should be removed. 
Moreover, holder itself can contain logic related with change if metric state. 
It should look like:

{code:java}
class MetricHolder {
  boolean enabled;

  AtomicLongMetric m;

  public MetricHolder(GirdKernalContext ctx) {
ctx.monitoring.onDisable("metrics", () -> {
  ctx.metric().remove("registryName");

  m = null;  
 }
);

ctx.monitoring.onEnable("metrics", r -> {
  MetricRegistry r = new MetricRegistry("registryName");

  m = r.longMetric("metric");

  ctx.metric().add("registryName");
 }
);
  }

  public long m() {
assert enabled;

return m;
  }

  public void changeState() {
if (enabled)
  m().increment();
  }
}

class SomeProcessor {
  public SomeProcesor() {
metricHolder = new MetricHolder(ctx);
  }
}
{code}


was (Author: agura):
[~NIzhikov] Not certainly in that way. WE don't need NO_OP_METRIC concept 
because we don't disable metric itself, we disable whole set of metrics that is 
represented by metric registry. So metric registry also should be removed. 
Moreover, holder itself can contain logic related with change if metric state. 
It should look like:

{code:java}
class MetricHolder {
  boolean enabled;

  AtomicLongMetric m;

  public MetricHolder(GirdKernalContext ctx) {
ctx.monitoring.onDisable("metrics", () -> {
  ctx.metric().remove("registryName");

  m = null;  
 }
);

ctx.monitoring.onEnable("metrics", r -> {
  MetricRegistry r = new MetricRegistry("registryName");

  m = r.longMetric("metric");

  ctx.metric().add("registryName");
 }
);
  }

  public long m() {
assert enabled;

return m;
  }

  public void changeState() {
if (enabled)
  m().increment();
  }
}

class SomeProcessor {
  public SomeProcesor() {
metricHolder = new MetricHolder(ctx);
  }
}
{java}

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-08-01 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898076#comment-16898076
 ] 

Andrey Gura commented on IGNITE-11927:
--

[~NIzhikov] Not certainly in that way. WE don't need NO_OP_METRIC concept 
because we don't disable metric itself, we disable whole set of metrics that is 
represented by metric registry. So metric registry also should be removed. 
Moreover, holder itself can contain logic related with change if metric state. 
It should look like:

{code:java}
class MetricHolder {
  boolean enabled;

  AtomicLongMetric m;

  public MetricHolder(GirdKernalContext ctx) {
ctx.monitoring.onDisable("metrics", () -> {
  ctx.metric().remove("registryName");

  m = null;  
 }
);

ctx.monitoring.onEnable("metrics", r -> {
  MetricRegistry r = new MetricRegistry("registryName");

  m = r.longMetric("metric");

  ctx.metric().add("registryName");
 }
);
  }

  public long m() {
assert enabled;

return m;
  }

  public void changeState() {
if (enabled)
  m().increment();
  }
}

class SomeProcessor {
  public SomeProcesor() {
metricHolder = new MetricHolder(ctx);
  }
}
{java}

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Resolved] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter

2019-07-31 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-12029.
--
Resolution: Duplicate

> [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
> -
>
> Key: IGNITE-12029
> URL: https://issues.apache.org/jira/browse/IGNITE-12029
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> {{HistogramMetric}} returns values as long array where each element 
> represents value for some bucket (bounded interval). At present there is no 
> way for getting label for bucket (bound value) in order to allow distinguish 
> values on user side (in some metrics backend). Should be added method which 
> returns bounds values.
> Example: 
> We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It 
> should be transformed to three metric value: 
> * {{metricName.10 = }}
> * {{metricName.50 = }}
> * {{metricName.100 = }}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter

2019-07-31 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897289#comment-16897289
 ] 

Andrey Gura commented on IGNITE-12029:
--

[~NSAmelchev] Thanks! I'll close this ticket as duplicate.

> [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
> -
>
> Key: IGNITE-12029
> URL: https://issues.apache.org/jira/browse/IGNITE-12029
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> {{HistogramMetric}} returns values as long array where each element 
> represents value for some bucket (bounded interval). At present there is no 
> way for getting label for bucket (bound value) in order to allow distinguish 
> values on user side (in some metrics backend). Should be added method which 
> returns bounds values.
> Example: 
> We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It 
> should be transformed to three metric value: 
> * {{metricName.10 = }}
> * {{metricName.50 = }}
> * {{metricName.100 = }}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12028) [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics exporter

2019-07-31 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12028:
-
Description: 
{{HitRateMetric}} allows to get only counter value while it would be useful to 
get also {{rateTimeInterval}} in order to export this value as part of metric 
name. 

For example look at cache metric {{RebalancingKeysRate}}. Value of this 
measurement could be exported as smth like this: 
{{cache..RebalancingKeysRate. = }}.

So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of 
{{LongMetric}} interface.

  was:
{{HitRateMetric}} allows to get only counter value while it would be useful to 
get also {{rateTimeInterval}} in order to export this value as part of metric 
name. 

For example look at cache metric {{RebalancingKeysRate}}. Value of this 
measurement could be exported as smth like this: 
{{cache..RebalancingKeysRate. = }}.

So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of 
{{LongMetric}} interface.


> [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics 
> exporter
> 
>
> Key: IGNITE-12028
> URL: https://issues.apache.org/jira/browse/IGNITE-12028
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> {{HitRateMetric}} allows to get only counter value while it would be useful 
> to get also {{rateTimeInterval}} in order to export this value as part of 
> metric name. 
> For example look at cache metric {{RebalancingKeysRate}}. Value of this 
> measurement could be exported as smth like this: 
> {{cache..RebalancingKeysRate. = }}.
> So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of 
> {{LongMetric}} interface.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter

2019-07-31 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-12029:
-
Environment: (was: {{HistogramMetric}} returns values as long array 
where each element represents value for some bucket (bounded interval). At 
present there is no way for getting label for bucket (bound value) in order to 
allow distinguish values on user side (in some metrics backend). Should be 
added method which returns bounds values.

Example: 

We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It 
should be transformed to three metric value: 

* {{metricName.10 = }}
* {{metricName.50 = }}
* {{metricName.100 = }}
)
Description: 
{{HistogramMetric}} returns values as long array where each element represents 
value for some bucket (bounded interval). At present there is no way for 
getting label for bucket (bound value) in order to allow distinguish values on 
user side (in some metrics backend). Should be added method which returns 
bounds values.

Example: 

We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It 
should be transformed to three metric value: 

* {{metricName.10 = }}
* {{metricName.50 = }}
* {{metricName.100 = }}


> [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
> -
>
> Key: IGNITE-12029
> URL: https://issues.apache.org/jira/browse/IGNITE-12029
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>
> {{HistogramMetric}} returns values as long array where each element 
> represents value for some bucket (bounded interval). At present there is no 
> way for getting label for bucket (bound value) in order to allow distinguish 
> values on user side (in some metrics backend). Should be added method which 
> returns bounds values.
> Example: 
> We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It 
> should be transformed to three metric value: 
> * {{metricName.10 = }}
> * {{metricName.50 = }}
> * {{metricName.100 = }}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter

2019-07-31 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-12029:


 Summary: [IEP-35] HistogramMetric should privde bounds' values to 
metrics exporter
 Key: IGNITE-12029
 URL: https://issues.apache.org/jira/browse/IGNITE-12029
 Project: Ignite
  Issue Type: Bug
 Environment: {{HistogramMetric}} returns values as long array where 
each element represents value for some bucket (bounded interval). At present 
there is no way for getting label for bucket (bound value) in order to allow 
distinguish values on user side (in some metrics backend). Should be added 
method which returns bounds values.

Example: 

We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It 
should be transformed to three metric value: 

* {{metricName.10 = }}
* {{metricName.50 = }}
* {{metricName.100 = }}

Reporter: Andrey Gura
 Fix For: 2.8






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-12028) [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics exporter

2019-07-31 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-12028:


 Summary: [IEP-35] HitRateMetric should provide rateTimeInterval 
value to metrics exporter
 Key: IGNITE-12028
 URL: https://issues.apache.org/jira/browse/IGNITE-12028
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Gura
 Fix For: 2.8


{{HitRateMetric}} allows to get only counter value while it would be useful to 
get also {{rateTimeInterval}} in order to export this value as part of metric 
name. 

For example look at cache metric {{RebalancingKeysRate}}. Value of this 
measurement could be exported as smth like this: 
{{cache..RebalancingKeysRate. = }}.

So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of 
{{LongMetric}} interface.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-7883) Cluster can have inconsistent affinity configuration

2019-07-30 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896057#comment-16896057
 ] 

Andrey Gura commented on IGNITE-7883:
-

[~a-polyakov] Thanks for your contribution! Merged to master branch.

> Cluster can have inconsistent affinity configuration 
> -
>
> Key: IGNITE-7883
> URL: https://issues.apache.org/jira/browse/IGNITE-7883
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Assignee: Alexand Polyakov
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> A cluster can have inconsistent affinity configuration if you created two 
> nodes, one with affinity key configuration and other without it(in IgniteCfg 
> or CacheCfg),  both nodes will work fine with no exceptions, but in the same 
> time they will apply different affinity rules to keys:
>  
> {code:java}
> package affinity;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheAtomicityMode;
> import org.apache.ignite.cache.CacheKeyConfiguration;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.affinity.Affinity;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import java.util.Arrays;
> public class Test {
> private static int id = 0;
> public static void main(String[] args) {
> Ignite ignite = Ignition.start(getConfiguration(true, false));
> Ignite ignite2 = Ignition.start(getConfiguration(false, false));
> Affinity affinity = ignite.affinity("TEST");
> Affinity affinity2 = ignite2.affinity("TEST");
> for (int i = 0; i < 1_000_000; i++) {
> AKey key = new AKey(i);
> if(affinity.partition(key) != affinity2.partition(key))
> System.out.println("FAILED for: " + key);
> }
> System.out.println("DONE");
> }
> private static IgniteConfiguration getConfiguration(boolean 
> withAffinityCfg, boolean client) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder(true);
> finder.setAddresses(Arrays.asList("localhost:47500..47600"));
> cfg.setClientMode(client);
> cfg.setIgniteInstanceName("test" + id++);
> CacheConfiguration cacheCfg = new CacheConfiguration("TEST");
> cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> if(withAffinityCfg) {
> cacheCfg.setKeyConfiguration(new 
> CacheKeyConfiguration("affinity.AKey", "a"));
> }
> cfg.setCacheConfiguration(cacheCfg);
> cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(finder));
> return cfg;
> }
> }
> class AKey {
> int a;
> public AKey(int a) {
> this.a = a;
> }
> @Override public String toString() {
> return "AKey{" +
> "a=" + a +
> '}';
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-10654) Report in case of creating index with already existing fields collection.

2019-07-30 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896039#comment-16896039
 ] 

Andrey Gura commented on IGNITE-10654:
--

[~zstan] [~jooger] I've added some review comments to the PR. Please fix before 
merge.

> Report in case of creating index with already existing fields collection.
> -
>
> Key: IGNITE-10654
> URL: https://issues.apache.org/jira/browse/IGNITE-10654
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 2.7
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Report in log if new index creating with already existing fields collection.
> for example, need to log warn here:
> {code:java}
> cache.query(new SqlFieldsQuery("create index \"idx1\" on Val(keyStr, 
> keyLong)"));
> cache.query(new SqlFieldsQuery("create index \"idx3\" on Val(keyStr, 
> keyLong)"));
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-18 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887990#comment-16887990
 ] 

Andrey Gura edited comment on IGNITE-11927 at 7/18/19 2:07 PM:
---

[~NIzhikov]

> Can you, please, make a simple, pseudo-code example of your idea?

Caches, for example. Not pseudo-code, but list of action items.

On cache start or metrics enabling on cache:

- create metrics holder object.
- create MetricRegistry instance.
- register metrics in MetricRegistry.
- add MetricsRegistry in GridMetricManager.

On cache stop or metrics disabling for chache:

- remove MetricRegistry from GridMetricManager.
- assign null to metrics holder object reference.

> 1. If we have 5000 caches, Ignite structures already huge. Why do you think 
> metrics bring a huge impact on GC?

Why we should ignore this impact if we can just avoid it without much effort?

> 2. All AtomicLong fields are created in previous versions of 
> CacheMetricsImpl. MetricRegistry is the only addition we made with the new 
> framework.

You are right. But this addition lead to some changes in design. It's good time 
to improve implementation.

Also, I think it would be better design if MetricRegistry will be immutable. It 
will lead to more clear code structure and behavior.

> Do we have some benchmarks or other descriptions of this issue?

No, we don't. But obviously all this objects in heap are not free.



was (Author: agura):
[~NIzhikov]

> Can you, please, make a simple, pseudo-code example of your idea?

Caches, for example. Not pseudo-code, but list of action items.

On cache start or metrics enabling on cache:

- create metrics holder object.
- create MetricRegistry instance.
- register metrics in MetricRegistry.
- add MetricsRegistry in GridMetricManager.

On cache stop or metrics disabling for chache:

- remove MetricRegistry from GridMetricManager.
- assign null to metrics holder object reference.

> 1. If we have 5000 caches, Ignite structures already huge. Why do you think 
> metrics bring a huge impact on GC?

Why we should ignore this impact if we can just avoid it without much effort?

> 2. All AtomicLong fields are created in previous versions of 
> CacheMetricsImpl. MetricRegistry is the only addition we made with the new 
> framework.

You are right. But this addition lead to some changes in design. It's good time 
to improve implementation.

> Do we have some benchmarks or other descriptions of this issue?

No, we don't. But obviously all this objects in heap are not free.

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-18 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888011#comment-16888011
 ] 

Andrey Gura commented on IGNITE-11927:
--

BTW, I found one more issue with memory consumption. When metric registers in 
metric registry we use metric name without group name as key in metric registry 
and metric name concatenated with group name as {{AbstractMetric.name}} field 
value. So we consume twice more memory just for metric name. 

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-18 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887990#comment-16887990
 ] 

Andrey Gura commented on IGNITE-11927:
--

[~NIzhikov]

> Can you, please, make a simple, pseudo-code example of your idea?

Caches, for example. Not pseudo-code, but list of action items.

On cache start or metrics enabling on cache:

- create metrics holder object.
- create MetricRegistry instance.
- register metrics in MetricRegistry.
- add MetricsRegistry in GridMetricManager.

On cache stop or metrics disabling for chache:

- remove MetricRegistry from GridMetricManager.
- assign null to metrics holder object reference.

> 1. If we have 5000 caches, Ignite structures already huge. Why do you think 
> metrics bring a huge impact on GC?

Why we should ignore this impact if we can just avoid it without much effort?

> 2. All AtomicLong fields are created in previous versions of 
> CacheMetricsImpl. MetricRegistry is the only addition we made with the new 
> framework.

You are right. But this addition lead to some changes in design. It's good time 
to improve implementation.

> Do we have some benchmarks or other descriptions of this issue?

No, we don't. But obviously all this objects in heap are not free.

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-18 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887906#comment-16887906
 ] 

Andrey Gura commented on IGNITE-11927:
--

[~NIzhikov] 

>> 3. There is no need for disabled flag in MetricRegistry. As we discused 
>> early, when metric disabled they don't consume any memory. MetricsRegistry 
>> should be collected by GC. After enabling it can be created and registered 
>> into GridMetricManager again.

>I don't understand your proposal.

>Metric instances are stored as a class field in the places where they updated.
>You can take a GridJobProcessor or CacheMetricImpl as examples.

>How and why we should clear these variables on disabling?
>Can you provide simple pseudo code for disable\enable processing.

At least metric registry can be just removed from registries. On enabling new 
instance can be created. Ideally, metrics can be moved to the special holder 
class and reference to it can be null after disabling.

Motivation: reducing memory consuming and GC pressure. There are users with big 
amount of caches (I saw cases with 5000 caches in 200 cache groups). 


> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11981) [IEP-35] Create MU shortcut instead of static import of MetricUtils

2019-07-16 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886250#comment-16886250
 ] 

Andrey Gura commented on IGNITE-11981:
--

[~NIzhikov] As I know there are no any restrictions about static imports. 

> [IEP-35] Create MU shortcut instead of static import of MetricUtils
> ---
>
> Key: IGNITE-11981
> URL: https://issues.apache.org/jira/browse/IGNITE-11981
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>
> More Ignite-way of coding is the usage of short cut classes.
> We should use MU instead of static import of {{MetricUtils}}.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11981) [IEP-35] Create MU shortcut instead of static import of MetricUtils

2019-07-15 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16885326#comment-16885326
 ] 

Andrey Gura commented on IGNITE-11981:
--

IMHO, bad idea and bad practice in the project. There is no problem in typing 
MetricUtils :)

> [IEP-35] Create MU shortcut instead of static import of MetricUtils
> ---
>
> Key: IGNITE-11981
> URL: https://issues.apache.org/jira/browse/IGNITE-11981
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>
> More Ignite-way of coding is the usage of short cut classes.
> We should use MU instead of static import of {{MetricUtils}}.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics

2019-07-14 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884683#comment-16884683
 ] 

Andrey Gura edited comment on IGNITE-11927 at 7/14/19 1:53 PM:
---

Anyway, I've took a look and have some comments:

1. There is no need to be disabled or enabled for particular metric value. So 
{{disabled}} flag in {{AbstractMetric}} is redundant. 
2. We enable or disable metrics for one source of metrics. So we can just 
remove {{MetricRegistry}} from {{GridMetricManager}} when metrics disabled.
3. There is no need for {{disabled}} flag in {{MetricRegistry}}. As we discused 
early, when metric disabled they don't consume any memory. MetricsRegistry 
should be collected by GC. After enabling it can be created and registered into 
{{GridMetricManager}} again.
4. Please, separate buckets configuration and metrics enabling/disabling into 
two different tickets.
5. Please, create review in Upsource.
6. Please, run TC and take a look to results before code review.

Could you please fix this issues? Thanks.


was (Author: agura):
Anyway, I've took a look and have some comments:

1. There is no need to be disabled or enabled for particular metric value. So 
{{disabled}} flag in {{AbstractMetric}} is redundant. 
2. We enable or disable metrics for one source of metrics. So we can just 
remove {{MetricRegistry}} from {{GridMetricManager}} when metrics disabled.
3. There is no need for {{disabled}} flag in {{MetricRegistry}}. As we discused 
early, when metric disabled they don't consume any memory. MetricsRegistry 
should be collected by GC. After enabling it can be created and registered into 
{{GridMetricManager}} again.
4. Please, separate buckets configuration and metrics enabling/disabling into 
two different tickets.
5. Please, create review in Upsource.
6. Please, run TC and take a look to results before code review.

> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics

2019-07-14 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884683#comment-16884683
 ] 

Andrey Gura commented on IGNITE-11927:
--

Anyway, I've took a look and have some comments:

1. There is no need to be disabled or enabled for particular metric value. So 
{{disabled}} flag in {{AbstractMetric}} is redundant. 
2. We enable or disable metrics for one source of metrics. So we can just 
remove {{MetricRegistry}} from {{GridMetricManager}} when metrics disabled.
3. There is no need for {{disabled}} flag in {{MetricRegistry}}. As we discused 
early, when metric disabled they don't consume any memory. MetricsRegistry 
should be collected by GC. After enabling it can be created and registered into 
{{GridMetricManager}} again.
4. Please, separate buckets configuration and metrics enabling/disabling into 
two different tickets.
5. Please, create review in Upsource.
6. Please, run TC and take a look to results before code review.

> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics

2019-07-14 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884676#comment-16884676
 ] 

Andrey Gura commented on IGNITE-11927:
--

[~NIzhikov] What about TC? 

> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics

2019-07-09 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881161#comment-16881161
 ] 

Andrey Gura commented on IGNITE-11927:
--

IMHO, histograms and hit rates configuration isn't related with metrics 
enabling/disabling and it should be done as separate units of work.

> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-7883) Cluster can have inconsistent affinity configuration

2019-07-09 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881153#comment-16881153
 ] 

Andrey Gura commented on IGNITE-7883:
-

[~a-polyakov] Could you please rebase your changes on the top of the master 
branch and rerun TC? Your changes were made too long ago.

> Cluster can have inconsistent affinity configuration 
> -
>
> Key: IGNITE-7883
> URL: https://issues.apache.org/jira/browse/IGNITE-7883
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Assignee: Alexand Polyakov
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> A cluster can have inconsistent affinity configuration if you created two 
> nodes, one with affinity key configuration and other without it(in IgniteCfg 
> or CacheCfg),  both nodes will work fine with no exceptions, but in the same 
> time they will apply different affinity rules to keys:
>  
> {code:java}
> package affinity;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheAtomicityMode;
> import org.apache.ignite.cache.CacheKeyConfiguration;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.affinity.Affinity;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import java.util.Arrays;
> public class Test {
> private static int id = 0;
> public static void main(String[] args) {
> Ignite ignite = Ignition.start(getConfiguration(true, false));
> Ignite ignite2 = Ignition.start(getConfiguration(false, false));
> Affinity affinity = ignite.affinity("TEST");
> Affinity affinity2 = ignite2.affinity("TEST");
> for (int i = 0; i < 1_000_000; i++) {
> AKey key = new AKey(i);
> if(affinity.partition(key) != affinity2.partition(key))
> System.out.println("FAILED for: " + key);
> }
> System.out.println("DONE");
> }
> private static IgniteConfiguration getConfiguration(boolean 
> withAffinityCfg, boolean client) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder(true);
> finder.setAddresses(Arrays.asList("localhost:47500..47600"));
> cfg.setClientMode(client);
> cfg.setIgniteInstanceName("test" + id++);
> CacheConfiguration cacheCfg = new CacheConfiguration("TEST");
> cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> if(withAffinityCfg) {
> cacheCfg.setKeyConfiguration(new 
> CacheKeyConfiguration("affinity.AKey", "a"));
> }
> cfg.setCacheConfiguration(cacheCfg);
> cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(finder));
> return cfg;
> }
> }
> class AKey {
> int a;
> public AKey(int a) {
> this.a = a;
> }
> @Override public String toString() {
> return "AKey{" +
> "a=" + a +
> '}';
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11958) JDBC connection validation should use it's own task instead of cache validation task

2019-07-03 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878003#comment-16878003
 ] 

Andrey Gura commented on IGNITE-11958:
--

[~Denis Chudov] What is motivation for this change? Is there any issues related 
with it?

> JDBC connection validation should use it's own task instead of cache 
> validation task
> 
>
> Key: IGNITE-11958
> URL: https://issues.apache.org/jira/browse/IGNITE-11958
> Project: Ignite
>  Issue Type: Bug
>Reporter: Denis Chudov
>Assignee: Denis Chudov
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> JDBC connection is validated using GridCacheQueryJdbcValidationTask. We 
> should create own validation task for this activity.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11945) [IEP-35] Performance drop on cache stop

2019-07-03 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16877990#comment-16877990
 ] 

Andrey Gura commented on IGNITE-11945:
--

[~NIzhikov] Looks good to me.

> [IEP-35] Performance drop on cache stop
> ---
>
> Key: IGNITE-11945
> URL: https://issues.apache.org/jira/browse/IGNITE-11945
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> `MetricRegistry` implementation drop performance on cache stops.
> Has to be fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11848) [IEP-35] Monitoring&Profiling. Phase 1

2019-06-13 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862974#comment-16862974
 ] 

Andrey Gura commented on IGNITE-11848:
--

[~NIzhikov] Thanks! LGTM. If you are ready, please, proceed with merge.

> [IEP-35] Monitoring&Profiling. Phase 1
> --
>
> Key: IGNITE-11848
> URL: https://issues.apache.org/jira/browse/IGNITE-11848
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.7
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
> Fix For: 2.8
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> Umbrella ticket for the IEP-35. Monitoring and profiling.
> Phase 1 should include:
>  * NextGen monitoring subsystem implementation to manage
>  ** metrics
>  ** -lists- (will be implemented in the following tickets)
>  ** exporters
>  * JMX, SQLView, Log exporters
>  * Migration of existing metrics to new manager
>  * -Lists for all Ignite user API-



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11899) Fix code style violation

2019-06-07 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858745#comment-16858745
 ] 

Andrey Gura commented on IGNITE-11899:
--

[~Mmuzaf] Agree with proposed steps.

About help in fixing of created issues. Isn't problem. But magic numbers is 
still magic for me :) I think we need help of [~DmitriyGovorukhin].

> Fix code style violation
> 
>
> Key: IGNITE-11899
> URL: https://issues.apache.org/jira/browse/IGNITE-11899
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> All the code style issues must be fixed, see the sample below.
> {code}
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:50:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:183:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:187:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:191:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> {code}
> TC log:
> https://ci.ignite.apache.org/viewLog.html?buildId=4062737&buildTypeId=IgniteTests24Java8_CheckCodeStyle&tab=buildLog&branch_IgniteTests24Java8=%3Cdefault%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (IGNITE-11899) Fix code style violation

2019-06-07 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858627#comment-16858627
 ] 

Andrey Gura edited comment on IGNITE-11899 at 6/7/19 1:18 PM:
--

[~Mmuzaf] Just took a quick look at your change. The first, thanks a lot for 
care about code style. 
I want to comment some things that should be fixed also. Notice that my 
comments addressed to the code author not to you. 

1. Using one-line Javadoc for class fields.

{code:java}
 /**
  *
 */
private static final int PAGE_ID_OFFSET = 0;
{code}

Can be replaced by

{code:java}
 /** */
private static final int PAGE_ID_OFFSET = 0;
{code}

Moreover, empty Javadoc comment it's very bad practice, so I believe that all 
comments like this should be replaced by meaningful comment. 

2. Magic numbers in some file should be replaced by constants with 
self-descriptive names.

Just an example:

{code:java}
MemoryCalculator(){
onHeapAllocated(16 + (8 + 16) * 2);
}
{code}

I believe that this arithmetic expression can be rewritten in more clear manner.


was (Author: agura):
[~Mmuzaf] Just took a quick look at your change. The first, thanks a lot for 
care about code style. 
I want to comment some things that should be fixed also. Notice that my 
comments addressed to the code author not to you. 

1. Using one-line Javadoc for class fields.

{code:java}
 /**
 *
 */
private static final int PAGE_ID_OFFSET = 0;
{code}

Can be replaced by

{code:java}
 /** */
private static final int PAGE_ID_OFFSET = 0;
{code}

Moreover, empty Javadoc comment it's very bad practice, so I believe that all 
comments like this should be replaced by meaningful comment. 

2. Magic numbers in some file should be replaced by constants with 
self-descriptive names.

Just an example:

{code:java}
MemoryCalculator(){
onHeapAllocated(16 + (8 + 16) * 2);
}
{code}

I believe that this arithmetic expression can be rewritten in more clear manner.

> Fix code style violation
> 
>
> Key: IGNITE-11899
> URL: https://issues.apache.org/jira/browse/IGNITE-11899
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> All the code style issues must be fixed, see the sample below.
> {code}
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:50:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:183:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:187:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:191:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> {code}
> TC log:
> https://ci.ignite.apache.org/viewLog.html?buildId=4062737&buildTypeId=IgniteTests24Java8_CheckCodeStyle&tab=buildLog&branch_IgniteTests24Java8=%3Cdefault%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11899) Fix code style violation

2019-06-07 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16858627#comment-16858627
 ] 

Andrey Gura commented on IGNITE-11899:
--

[~Mmuzaf] Just took a quick look at your change. The first, thanks a lot for 
care about code style. 
I want to comment some things that should be fixed also. Notice that my 
comments addressed to the code author not to you. 

1. Using one-line Javadoc for class fields.

{code:java}
 /**
 *
 */
private static final int PAGE_ID_OFFSET = 0;
{code}

Can be replaced by

{code:java}
 /** */
private static final int PAGE_ID_OFFSET = 0;
{code}

Moreover, empty Javadoc comment it's very bad practice, so I believe that all 
comments like this should be replaced by meaningful comment. 

2. Magic numbers in some file should be replaced by constants with 
self-descriptive names.

Just an example:

{code:java}
MemoryCalculator(){
onHeapAllocated(16 + (8 + 16) * 2);
}
{code}

I believe that this arithmetic expression can be rewritten in more clear manner.

> Fix code style violation
> 
>
> Key: IGNITE-11899
> URL: https://issues.apache.org/jira/browse/IGNITE-11899
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> All the code style issues must be fixed, see the sample below.
> {code}
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:50:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:183:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:187:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR] 
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:191:
>  'VARIABLE_DEF' should be separated from previous statement. 
> [EmptyLineSeparator]
> {code}
> TC log:
> https://ci.ignite.apache.org/viewLog.html?buildId=4062737&buildTypeId=IgniteTests24Java8_CheckCodeStyle&tab=buildLog&branch_IgniteTests24Java8=%3Cdefault%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (IGNITE-10687) Document new JMX bean wchih expose IO statistics.

2019-06-04 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-10687.
--
Resolution: Won't Fix

> Document new JMX bean  wchih expose IO statistics.
> --
>
> Key: IGNITE-10687
> URL: https://issues.apache.org/jira/browse/IGNITE-10687
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, sql
>Reporter: Yury Gerzhedovich
>Priority: Major
> Fix For: 2.8
>
>
> We need to document a new JMX bean which expose Input/Output statistics.
> Start point is 
> [https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/mxbean/IoStatisticsMetricsMXBean.java]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Reopened] (IGNITE-10687) Document new JMX bean wchih expose IO statistics.

2019-06-04 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reopened IGNITE-10687:
--

> Document new JMX bean  wchih expose IO statistics.
> --
>
> Key: IGNITE-10687
> URL: https://issues.apache.org/jira/browse/IGNITE-10687
> Project: Ignite
>  Issue Type: Task
>  Components: documentation, sql
>Reporter: Yury Gerzhedovich
>Priority: Major
> Fix For: 2.8
>
>
> We need to document a new JMX bean which expose Input/Output statistics.
> Start point is 
> [https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/mxbean/IoStatisticsMetricsMXBean.java]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11848) [IEP-35] Monitoring&Profiling. Phase 1

2019-06-03 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854447#comment-16854447
 ] 

Andrey Gura commented on IGNITE-11848:
--

[~NIzhikov] I'll take a look. [~Jokser] could you please also join to the code 
review?

> [IEP-35] Monitoring&Profiling. Phase 1
> --
>
> Key: IGNITE-11848
> URL: https://issues.apache.org/jira/browse/IGNITE-11848
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.7
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Umbrella ticket for the IEP-35. Monitoring and profiling.
> Phase 1 should include:
>  * NextGen monitoring subsystem implementation to manage
>  ** metrics
>  ** -lists- (will be implemented in the following tickets)
>  ** exporters
>  * JMX, SQLView, Log exporters
>  * Migration of existing metrics to new manager
>  * -Lists for all Ignite user API-



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11804) Assertion error on affinity initialization

2019-04-25 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826004#comment-16826004
 ] 

Andrey Gura commented on IGNITE-11804:
--

[~ascherbakov] Just good practice. In many cases you can understand the problem 
just looking to stack trace. Thanks a lot!

> Assertion error on affinity initialization
> --
>
> Key: IGNITE-11804
> URL: https://issues.apache.org/jira/browse/IGNITE-11804
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexei Scherbakov
>Priority: Major
>
> Reproducer (needs some cleanup)
> {noformat}
> /*
>  * Licensed to the Apache Software Foundation (ASF) under one or more
>  * contributor license agreements.  See the NOTICE file distributed with
>  * this work for additional information regarding copyright ownership.
>  * The ASF licenses this file to You under the Apache License, Version 2.0
>  * (the "License"); you may not use this file except in compliance with
>  * the License.  You may obtain a copy of the License at
>  *
>  *  http://www.apache.org/licenses/LICENSE-2.0
>  *
>  * Unless required by applicable law or agreed to in writing, software
>  * distributed under the License is distributed on an "AS IS" BASIS,
>  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>  * See the License for the specific language governing permissions and
>  * limitations under the License.
>  */
> package org.apache.ignite.internal.processors.cache.transactions;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.concurrent.atomic.AtomicReference;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.DataRegionConfiguration;
> import org.apache.ignite.configuration.DataStorageConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.failure.StopNodeFailureHandler;
> import org.apache.ignite.internal.IgniteEx;
> import org.apache.ignite.internal.processors.cache.GridCacheContext;
> import org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl;
> import 
> org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
> import org.apache.ignite.transactions.Transaction;
> import org.junit.Test;
> import static java.util.concurrent.TimeUnit.DAYS;
> import static java.util.concurrent.TimeUnit.MILLISECONDS;
> import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL;
> import static org.apache.ignite.cache.CacheWriteSynchronizationMode.FULL_SYNC;
> import static org.apache.ignite.configuration.WALMode.LOG_ONLY;
> import static 
> org.apache.ignite.transactions.TransactionConcurrency.PESSIMISTIC;
> import static 
> org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_READ;
> /**
>  * Test framework for ordering transaction's prepares and commits by 
> intercepting messages and releasing then
>  * in user defined order.
>  */
> public class TxPartitionCounterStateAbstractTest extends 
> GridCommonAbstractTest {
> /** IP finder. */
> private static final TcpDiscoveryVmIpFinder IP_FINDER = new 
> TcpDiscoveryVmIpFinder(true);
> /** */
> private static final int MB = 1024 * 1024;
> /** */
> protected int backups;
> /** */
> public static final int TEST_TIMEOUT = 30_000;
> public static final String DEFAULT_CACHE_NAME_2 = DEFAULT_CACHE_NAME + 
> "2";
> /** */
> private AtomicReference testFailed = new AtomicReference<>();
> /** Number of keys to preload before txs to enable historical rebalance. 
> */
> protected static final int PRELOAD_KEYS_CNT = 1;
> /** */
> protected static final String CLIENT_GRID_NAME = "client";
> /** */
> protected static final int PARTS_CNT = 32;
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String 
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> cfg.setConsistentId("node" + igniteInstanceName);
> cfg.setFailureHandler(new StopNodeFailureHandler());
> cfg.setRebalanceThreadPoolSize(4); // Necessary to reproduce some 
> issues.
> ((TcpDiscoverySpi)cfg.getDiscoverySpi()).setIpFinder(IP_FINDER);
> // TODO set this only for historical rebalance tests.
> cfg.setCommunicationSpi(new 
> IgniteWalRebalanceTest.WalRebalanceCheckingCommunicationSpi());
>   

[jira] [Commented] (IGNITE-11804) Assertion error

2019-04-24 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825354#comment-16825354
 ] 

Andrey Gura commented on IGNITE-11804:
--

[~ascherbakov] Could you please add assertion stack trace to the issue 
description?

> Assertion error
> ---
>
> Key: IGNITE-11804
> URL: https://issues.apache.org/jira/browse/IGNITE-11804
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexei Scherbakov
>Priority: Major
>
> Reproducer (needs some cleanup)
> {noformat}
> /*
>  * Licensed to the Apache Software Foundation (ASF) under one or more
>  * contributor license agreements.  See the NOTICE file distributed with
>  * this work for additional information regarding copyright ownership.
>  * The ASF licenses this file to You under the Apache License, Version 2.0
>  * (the "License"); you may not use this file except in compliance with
>  * the License.  You may obtain a copy of the License at
>  *
>  *  http://www.apache.org/licenses/LICENSE-2.0
>  *
>  * Unless required by applicable law or agreed to in writing, software
>  * distributed under the License is distributed on an "AS IS" BASIS,
>  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>  * See the License for the specific language governing permissions and
>  * limitations under the License.
>  */
> package org.apache.ignite.internal.processors.cache.transactions;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.concurrent.atomic.AtomicReference;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.DataRegionConfiguration;
> import org.apache.ignite.configuration.DataStorageConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.failure.StopNodeFailureHandler;
> import org.apache.ignite.internal.IgniteEx;
> import org.apache.ignite.internal.processors.cache.GridCacheContext;
> import org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl;
> import 
> org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
> import org.apache.ignite.transactions.Transaction;
> import org.junit.Test;
> import static java.util.concurrent.TimeUnit.DAYS;
> import static java.util.concurrent.TimeUnit.MILLISECONDS;
> import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL;
> import static org.apache.ignite.cache.CacheWriteSynchronizationMode.FULL_SYNC;
> import static org.apache.ignite.configuration.WALMode.LOG_ONLY;
> import static 
> org.apache.ignite.transactions.TransactionConcurrency.PESSIMISTIC;
> import static 
> org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_READ;
> /**
>  * Test framework for ordering transaction's prepares and commits by 
> intercepting messages and releasing then
>  * in user defined order.
>  */
> public class TxPartitionCounterStateAbstractTest extends 
> GridCommonAbstractTest {
> /** IP finder. */
> private static final TcpDiscoveryVmIpFinder IP_FINDER = new 
> TcpDiscoveryVmIpFinder(true);
> /** */
> private static final int MB = 1024 * 1024;
> /** */
> protected int backups;
> /** */
> public static final int TEST_TIMEOUT = 30_000;
> public static final String DEFAULT_CACHE_NAME_2 = DEFAULT_CACHE_NAME + 
> "2";
> /** */
> private AtomicReference testFailed = new AtomicReference<>();
> /** Number of keys to preload before txs to enable historical rebalance. 
> */
> protected static final int PRELOAD_KEYS_CNT = 1;
> /** */
> protected static final String CLIENT_GRID_NAME = "client";
> /** */
> protected static final int PARTS_CNT = 32;
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String 
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> cfg.setConsistentId("node" + igniteInstanceName);
> cfg.setFailureHandler(new StopNodeFailureHandler());
> cfg.setRebalanceThreadPoolSize(4); // Necessary to reproduce some 
> issues.
> ((TcpDiscoverySpi)cfg.getDiscoverySpi()).setIpFinder(IP_FINDER);
> // TODO set this only for historical rebalance tests.
> cfg.setCommunicationSpi(new 
> IgniteWalRebalanceTest.WalRebalanceCheckingCommunicationSpi());
> boolean client = igniteInstanceName.startsWith(CLIENT_GRID_NAME);
> cfg.setClie

[jira] [Assigned] (IGNITE-11267) Print warning when keystore password arguments are used in control.sh (bat)

2019-04-24 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reassigned IGNITE-11267:


Assignee: Andrey Gura  (was: Andrey Kuznetsov)

> Print warning when keystore password arguments are used in control.sh (bat)
> ---
>
> Key: IGNITE-11267
> URL: https://issues.apache.org/jira/browse/IGNITE-11267
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.7
>Reporter: Andrey Kuznetsov
>Assignee: Andrey Gura
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Control utility gets keystore/truststore password either as command line 
> argument or as console input. Former way is insecure, and user should be 
> warned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (IGNITE-11267) Print warning when keystore password arguments are used in control.sh (bat)

2019-04-24 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reassigned IGNITE-11267:


Assignee: Andrey Kuznetsov  (was: Andrey Gura)

> Print warning when keystore password arguments are used in control.sh (bat)
> ---
>
> Key: IGNITE-11267
> URL: https://issues.apache.org/jira/browse/IGNITE-11267
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.7
>Reporter: Andrey Kuznetsov
>Assignee: Andrey Kuznetsov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Control utility gets keystore/truststore password either as command line 
> argument or as console input. Former way is insecure, and user should be 
> warned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11774) NPE TxDeadlockDetection in case of concurrent cache stop and tx.

2019-04-19 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16821957#comment-16821957
 ] 

Andrey Gura commented on IGNITE-11774:
--

[~zstan] Does it prevent successful node stopping?

> NPE TxDeadlockDetection in case of concurrent cache stop and tx.
> 
>
> Key: IGNITE-11774
> URL: https://issues.apache.org/jira/browse/IGNITE-11774
> Project: Ignite
>  Issue Type: Bug
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>
> In process of working under : 
> [IGNITE-11592|https://issues.apache.org/jira/browse/IGNITE-11592] reproduced 
> NPE, need further research, reproducer commented in upper ticket code.
> {code:java}
> java.lang.NullPointerException
>   at 
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.primary(TxDeadlockDetection.java:400)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.mapTxKeys(TxDeadlockDetection.java:376)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.map(TxDeadlockDetection.java:275)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.init(TxDeadlockDetection.java:267)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.access$100(TxDeadlockDetection.java:158)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection.detectDeadlock(TxDeadlockDetection.java:91)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.detectDeadlock(IgniteTxManager.java:2155)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onTimeout(GridNearOptimisticTxPrepareFuture.java:776)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read

2019-04-15 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818481#comment-16818481
 ] 

Andrey Gura commented on IGNITE-11687:
--

[~agoncharuk] I've investigated the problem deeper. While code snippet pointed 
by you is incorrect and must be fixed it never executes by test because MMAP 
mode is switched on by default. I think that 
{{FileWriteHandleImpl#addRecord()}} method is root of the problem. See the 
following code snippet:

{code:java}
fillBuffer(buf, rec);

if (mmap) {
// written field must grow only, but segment with 
greater position can be serialized
// earlier than segment with smaller position.
while (true) {
long written0 = written;

if (seg.position() > written0) {
if (WRITTEN_UPD.compareAndSet(this, written0, 
seg.position()))
break;
}
else
break;
}
}

return ptr;
{code}

WAL iterator on {{wal.replay()}} call gets {{hnd.written}} field value while 
some previous WAL record before this position is still not fully serialized. 
What do you think?

> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been 
> already updated, concurrent WAL replay will attempt to read the incompletely 
> written record and since {{end}} is not null, iterator will fail with CRC 
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read

2019-04-05 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811031#comment-16811031
 ] 

Andrey Gura commented on IGNITE-11687:
--

[~agoncharuk] You are right. It could be fixed easy. 
{{FileHandleManagerImpl.WALWriter#writeBuffer}} method has only one usage so it 
can just return delta that must be added to the {{hnd.written}.}

> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been 
> already updated, concurrent WAL replay will attempt to read the incompletely 
> written record and since {{end}} is not null, iterator will fail with CRC 
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read

2019-04-05 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811031#comment-16811031
 ] 

Andrey Gura edited comment on IGNITE-11687 at 4/5/19 4:38 PM:
--

[~agoncharuk] You are right. It could be fixed easy. 
{{FileHandleManagerImpl.WALWriter#writeBuffer}} method has only one usage so it 
can just return delta that must be added to the {{hnd.written}}.


was (Author: agura):
[~agoncharuk] You are right. It could be fixed easy. 
{{FileHandleManagerImpl.WALWriter#writeBuffer}} method has only one usage so it 
can just return delta that must be added to the {{hnd.written}.}

> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been 
> already updated, concurrent WAL replay will attempt to read the incompletely 
> written record and since {{end}} is not null, iterator will fail with CRC 
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read

2019-04-05 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura reassigned IGNITE-11687:


Assignee: Andrey Gura

> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been 
> already updated, concurrent WAL replay will attempt to read the incompletely 
> written record and since {{end}} is not null, iterator will fail with CRC 
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-7883) Cluster can have inconsistent affinity configuration

2019-03-27 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802903#comment-16802903
 ] 

Andrey Gura commented on IGNITE-7883:
-

[~a-polyakov] I've took a look ti the change and have some comments:

1. {{GridCacheUtils#validateAffinityKeyConfiguration(CacheKeyConfiguration[])}}

I think better name is {{validateKeyConfigiration}}.

Please fix message of {{IgniteCheckedException}}. At the moment it is too 
verbose and unclear. I think it should be something like "Cache key 
configuration contains conflicting definitions: [cacheGroup=<>, cache=<>, 
typeName=<>, affKeyFieldName1=<>, affKeyFieldName2=<>]".

Javadoc should have more accurate formulation. E..g it's unclear words "items" 
and "full". Proper comment will be "all fields are initialized and not empty". 
Please rewrite javadoc.

2. {{GridCacheUtils#validateAffinityKeyConfiguration(String, UUID, 
CacheKeyConfiguration[], CacheKeyConfiguration[])}}

I think better name is {{validateKeyConfigiration}}. 

{{oldCacheKeyCfgs}} and {{newCacheKeyCfgs}} parameters should be renamed to 
{{rmtCacheKeyCfgs}} and {{locCacheKeyCfgs}} respectively.

Please fix message of {{IgniteCheckedException}}. At the moment it is too 
verbose and unclear. For example see message that generates 
{{GridCacheUtils#checkAttributeMismatch}} method. Also note that it could be 
exception or just log message at warning level (it depends on 
{{IGNITE_SKIP_CONFIGURATION_CONSISTENCY_CHECK}} system property). Also see p. 1 
above.

Format method's code properly.

Fix javadoc in order to properly describe method parameters after renaming.

3. {{CacheKeyConfiguration#equals}}

Use {{Objects.equals}} instead of too long expressions at the method end.

4. {{CacheAffinityKeyConfigurationMismatchTest}}

Please, rewrite javadoc o the class. 

Reformat code. Comma should always be on the same line where previous method 
parameter placed.

Rewrite test code. Framework has  set of methods for getting ignite and cache 
configuration. So methods like {getIgnite}} are redundant. See 
{{GridAbstractTest#getConfiguration()}} and other ignite tests for example.

Add additional tests for teh following cases:
- testKeyConfigurationLengthMismatch  - you test only case when we have key 
configuration on one node and don't have on another. What about different key 
configurations on nodes?
- testKeyConfigurationDuplicateTypeName - you test only case for one node. What 
about conflicting defintions from different nodes?

It makes sense to add test cases for configurations that were constructed due 
to using {{@AffinityKeyMapped}} annotation.



> Cluster can have inconsistent affinity configuration 
> -
>
> Key: IGNITE-7883
> URL: https://issues.apache.org/jira/browse/IGNITE-7883
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Assignee: Alexand Polyakov
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> A cluster can have inconsistent affinity configuration if you created two 
> nodes, one with affinity key configuration and other without it(in IgniteCfg 
> or CacheCfg),  both nodes will work fine with no exceptions, but in the same 
> time they will apply different affinity rules to keys:
>  
> {code:java}
> package affinity;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheAtomicityMode;
> import org.apache.ignite.cache.CacheKeyConfiguration;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.affinity.Affinity;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import java.util.Arrays;
> public class Test {
> private static int id = 0;
> public static void main(String[] args) {
> Ignite ignite = Ignition.start(getConfiguration(true, false));
> Ignite ignite2 = Ignition.start(getConfiguration(false, false));
> Affinity affinity = ignite.affinity("TEST");
> Affinity affinity2 = ignite2.affinity("TEST");
> for (int i = 0; i < 1_000_000; i++) {
> AKey key = new AKey(i);
> if(affinity.partition(key) != affinity2.partition(key))
> System.out.println("FAILED for: " + key);
> }
> System.out.println("DONE");
> }
> private static IgniteConfiguration getConfiguration(boolean 
> withAffinityCfg, boolean client) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDis

[jira] [Commented] (IGNITE-9812) Explicit tests with expired SSL certificates

2019-03-27 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802772#comment-16802772
 ] 

Andrey Gura commented on IGNITE-9812:
-

[~ilyak] LGTM! Please proceed with merge.

> Explicit tests with expired SSL certificates
> 
>
> Key: IGNITE-9812
> URL: https://issues.apache.org/jira/browse/IGNITE-9812
> Project: Ignite
>  Issue Type: Test
>  Components: clients, security
>Reporter: Ilya Kasnacheev
>Assignee: Ilya Kasnacheev
>Priority: Minor
>  Labels: test
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It came to our attention that we have no test which checks that expired SSL 
> certificates do not work indeed.
> Should get such old certificate from git, check if it will fail to join 
> cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11127) GridDhtInvalidPartitionException not handled by GridCacheTtlManager

2019-03-27 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802765#comment-16802765
 ] 

Andrey Gura commented on IGNITE-11127:
--

[~roman_s] Looks good to me too.

> GridDhtInvalidPartitionException not handled by GridCacheTtlManager
> ---
>
> Key: IGNITE-11127
> URL: https://issues.apache.org/jira/browse/IGNITE-11127
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.4, 2.7
>Reporter: Ilya Kasnacheev
>Assignee: Roman Shtykh
>Priority: Critical
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Leading to either message processing problems:
> {code}
> [2019-01-27 16:57:45,474][ERROR][sys-stripe-2-#3][GridCacheIoManager] Failed 
> to process message [senderId=4839b5a2-a295-44cf-8a44-f0cb932b689e, 
> messageType=class 
> o.a.i.i.processors.cache.distributed.dht.atomic.GridNearAtomicFullUpdateRequest]
> class 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException
>  [part=381, msg=Adding entry to partition that is concurrently evicted 
> [grp=, part=381, shouldBeMoving=, belongs=false, 
> topVer=AffinityTopologyVersion [topVer=818, minorTopVer=0], 
> curTopVer=AffinityTopologyVersion [topVer=818, minorTopVer=0]]]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition0(GridDhtPartitionTopologyImpl.java:917)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition(GridDhtPartitionTopologyImpl.java:794)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.localPartition(GridCachePartitionedConcurrentMap.java:69)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.putEntryIfObsoleteOrAbsent(GridCachePartitionedConcurrentMap.java:88)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.entryEx(GridCacheAdapter.java:952)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.entryEx(GridDhtCacheAdapter.java:525)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.entryEx(GridCacheAdapter.java:943)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:1047)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:197)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:835)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessageProcessed(GridCacheIoManager.java:1093)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1066)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
> at 
> org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> or unhandled unspecified exceptions in user code (possibly violating JCache):
> {code}
> [2019-01-27 10:23:35,451][ERROR][pub-#840058][ComputeJobProcess] class 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException
>  [part=485, msg=Adding entry to partition that is concurrently evicted 
> [grp=, part=485, shouldBeMoving=, belongs=false, 
> topVer=AffinityTopologyVersion [topVer=815, minorTopVer=0], 
> curTopVer=AffinityTopologyVersion [topVer=815, minorTopVer=0]]]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridD

[jira] [Commented] (IGNITE-10925) Failure to submit affinity task from client node

2019-02-27 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779470#comment-16779470
 ] 

Andrey Gura commented on IGNITE-10925:
--

[~ilyak] LGTM. Please merge. Thanks!

> Failure to submit affinity task from client node
> 
>
> Key: IGNITE-10925
> URL: https://issues.apache.org/jira/browse/IGNITE-10925
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.7
>Reporter: Prasad
>Assignee: Ilya Kasnacheev
>Priority: Blocker
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Getting following exception while submitting the affinity task from client 
> node to server node.
> Before submitting the affinity task ignite first gets the affinity cached 
> function (AffinityInfo) by submitting the cluster wide task "AffinityJob". 
> But while in the process of retrieving the output of this AffinityJob, ignite 
> deserializes this output. I am getting exception while deserailizing this 
> output.
> Code fails while un-marshalling cachesnapshotmetrics on client node.
>  
> [Userlist 
> Discussion|http://apache-ignite-users.70518.x6.nabble.com/After-upgrading-2-7-getting-Unexpected-error-occurred-during-unmarshalling-td26262.html]
> [Reproducer 
> Project|https://github.com/prasadbhalerao1983/IgniteIssueReproducer.git]
>  
> Step to Reproduce:
> 1) First Run com.example.demo.Server class as a java program
> 2) Then run com.example.demo.Client as java program.
>  
> {noformat}
> 2019-01-14 15:37:02.723 ERROR 10712 --- [springDataNode%] 
> o.a.i.i.processors.task.GridTaskWorker   : Error deserializing job response: 
> GridJobExecuteResponse [nodeId=e9a24c20-0d00-4808-b2f5-13e1ce35496a, 
> sesId=76324db4861-1d85ad49-5b25-454a-b69c-d8685cfc73b0, 
> jobId=86324db4861-1d85ad49-5b25-454a-b69c-d8685cfc73b0, gridEx=null, 
> isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with 
> optimized marshaller
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146) 
> ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
>  [ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
>  [ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>  [ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>  [ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>  [ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>  [ignite-core-2.7.0.jar:2.7.0]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  [na:1.8.0_144]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  [na:1.8.0_144]
>  at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to 
> unmarshal object with optimized marshaller
>  at 
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  ~[ignite-core-2.7.0.jar:2.7.0]
>  at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140) 
> ~[ignite-core-2.7.0.jar:2.7.0]
>  ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize 
> object with given class loader: 
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to deserialize 
> object [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
>  at 
> org.apache.ignite.internal.ma

[jira] [Commented] (IGNITE-11253) When a node that is not part of the baseline topology joins the cluster, it may lead to a node failure.

2019-02-11 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16764906#comment-16764906
 ] 

Andrey Gura commented on IGNITE-11253:
--

[~slava.koptilin] Could you please add at least {{InterruptedException}} to 
{{X.hasCause}} method's parameters. Usually any worker stops via {{cancel}} 
call that interrupt worker's runner.

> When a node that is not part of the baseline topology joins the cluster, it 
> may lead to a node failure.
> ---
>
> Key: IGNITE-11253
> URL: https://issues.apache.org/jira/browse/IGNITE-11253
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> * In case of eager TTL is configured, a starting node creates and starts 
> {{cleanupWorker}} (see {{GridCacheTtlManager.start0()}})
>  * {{GridCacheSharedTtlCleanupManager.CleanupWorker}}, in its turn, has to 
> wait for {{discovery().localJoin()}} future that is completed by discovery 
> thread.
>  * On the other hand, the exchange thread stops cache contexts and, 
> therefore, it stops the {{cleanupWorker}} as well.
>  
> {code:java}
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager.stopCleanupWorker(GridCacheSharedTtlCleanupManager.java:109)
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager.unregister(GridCacheSharedTtlCleanupManager.java:82)
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.onKernalStop0(GridCacheTtlManager.java:110)
> org.apache.ignite.internal.processors.cache.GridCacheManagerAdapter.onKernalStop(GridCacheManagerAdapter.java:111)
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:1495)
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStopCaches(GridCacheProcessor.java:1182)
> org.apache.ignite.internal.processors.cache.GridCacheProcessor$CacheRecoveryLifecycle.onBaselineChange(GridCacheProcessor.java:5637)
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:910)
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:792)
> {code}
> So, exchange thread may try to stop the {{cleanupWorker}} before the 
> {{localJoin}} future is completed by discovery thread. Unfortunately, 
> `cleanupWorker` incorrectly handles this situation, and this fact can lead to 
> a node failure:
> {code:java}
> Critical system error detected. Will be handled accordingly to configured 
> handler [hnd=StopNodeFailureHandler [super=AbstractFailureHandler 
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, 
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext 
> [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Got 
> interrupted while waiting for future to complete.]]
> class org.apache.ignite.IgniteException: Got interrupted while waiting for 
> future to complete.
> at 
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.localJoin(GridDiscoveryManager.java:2217)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:136)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class 
> org.apache.ignite.internal.IgniteInterruptedCheckedException: Got interrupted 
> while waiting for future to complete.
> at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:186)
> at 
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
> at 
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.localJoin(GridDiscoveryManager.java:2214)
> ... 3 more
> {code}
>  The obvious fix is changing the catch block
> {code:java}
> catch (Throwable t) {
> if (!(t instanceof IgniteInterruptedCheckedException))
> err = t;
> throw t;
> }
> {code}
> to the following:
> {code:java}
> catch (Throwable t) {
> if (!(X.hasCause(t, IgniteInterruptedCheckedException.class)))
> err = t;
> throw t;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11130) BackupPostProcessingClosure should not save initial value on a backup node when cache store is not configured.

2019-02-01 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758266#comment-16758266
 ] 

Andrey Gura commented on IGNITE-11130:
--

[~slava.koptilin] Thanks. Merged to master branch.

> BackupPostProcessingClosure should not save initial value on a backup node 
> when cache store is not configured.
> --
>
> Key: IGNITE-11130
> URL: https://issues.apache.org/jira/browse/IGNITE-11130
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The change introduced by IGNITE-7086 lead to the fact that 
> {{BackupPostProcessingClosure}} could attempt to store the initial value even 
> if the corresponding cache store is not configured.
> For instance, transactional read operation where atomicity mode is 
> {{OPTIMISTIC}} and isolation level is {{SERIALIZABLE}} (native persistent 
> enabled) tries to update the local backup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11130) BackupPostProcessingClosure should not save initial value on a backup node when cache store is not configured.

2019-01-31 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757850#comment-16757850
 ] 

Andrey Gura commented on IGNITE-11130:
--

[~slava.koptilin] LGTM. But is seems that some test are failed. Could you 
please check.

> BackupPostProcessingClosure should not save initial value on a backup node 
> when cache store is not configured.
> --
>
> Key: IGNITE-11130
> URL: https://issues.apache.org/jira/browse/IGNITE-11130
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The change introduced by IGNITE-7086 lead to the fact that 
> {{BackupPostProcessingClosure}} could attempt to store the initial value even 
> if the corresponding cache store is not configured.
> For instance, transactional read operation where atomicity mode is 
> {{OPTIMISTIC}} and isolation level is {{SERIALIZABLE}} (native persistent 
> enabled) tries to update the local backup.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE

2019-01-31 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura closed IGNITE-11129.


> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
> 
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Assignee: Stepachev Maksim
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
> switched on. Size for this record type must be constant.
> See 
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException 
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-9903) Copy only actual amount of data during archiving of WAL segment

2019-01-31 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757843#comment-16757843
 ] 

Andrey Gura commented on IGNITE-9903:
-

[~astelmak] LGTM. Merged to master branch. Thanks for contribution!

> Copy only actual amount of data during archiving of WAL segment
> ---
>
> Key: IGNITE-9903
> URL: https://issues.apache.org/jira/browse/IGNITE-9903
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Alexey Stelmak
>Priority: Major
> Fix For: 2.8
>
>
> Current WAL archiver implementation copies full WAL segment to the archive 
> while actual size of the segment can be significantly less then 
> {{maxWalSegmentSize}} (segment files are preallocated for max possible 
> segment size). 
> In order to reduce disk space consuming we need copy only actual data of 
> segment using {{SWITCH_SEGMENT_RECORD}} as marker. It will require some kind 
> of WAL iterator implementation that will just copy WAL records using 
> information about record size without records deserialization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE

2019-01-31 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757846#comment-16757846
 ] 

Andrey Gura commented on IGNITE-11129:
--

[~mstepachev] Agree. Thanks.

> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
> 
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Assignee: Stepachev Maksim
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
> switched on. Size for this record type must be constant.
> See 
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException 
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (IGNITE-9903) Copy only actual amount of data during archiving of WAL segment

2019-01-29 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755142#comment-16755142
 ] 

Andrey Gura edited comment on IGNITE-9903 at 1/29/19 6:40 PM:
--

[~astelmak] I've looked at your change and have some comments:

- Size of {{SWITCH_SEGMENT_RECORD}} should be calculated using 
{{RecordSerializer.size}} (see example here: 
{{org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl#close}}).
- Unused import in all changed files.
- {{switchSegmentRecordOffset}} array never updates (except of {{0}} value). 
Please add test for this situation.

Could you please fix this comments? Thanks.



was (Author: agura):
[~astelmak] I've looked at your change and have some comments:

- Size of {{SWITCH_SEGMENT_RECORD}} should be calculated using 
{{RecordSerializer.size}} (see example here: 
{{org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl#close}}).
- Unused import in all changed files.
- {{switchSegmentRecordOffset}} array never updates (except of {{0}} value).

Could you please fix this comments? Thanks.


> Copy only actual amount of data during archiving of WAL segment
> ---
>
> Key: IGNITE-9903
> URL: https://issues.apache.org/jira/browse/IGNITE-9903
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Alexey Stelmak
>Priority: Major
> Fix For: 2.8
>
>
> Current WAL archiver implementation copies full WAL segment to the archive 
> while actual size of the segment can be significantly less then 
> {{maxWalSegmentSize}} (segment files are preallocated for max possible 
> segment size). 
> In order to reduce disk space consuming we need copy only actual data of 
> segment using {{SWITCH_SEGMENT_RECORD}} as marker. It will require some kind 
> of WAL iterator implementation that will just copy WAL records using 
> information about record size without records deserialization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (IGNITE-9903) Copy only actual amount of data during archiving of WAL segment

2019-01-29 Thread Andrey Gura (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755142#comment-16755142
 ] 

Andrey Gura commented on IGNITE-9903:
-

[~astelmak] I've looked at your change and have some comments:

- Size of {{SWITCH_SEGMENT_RECORD}} should be calculated using 
{{RecordSerializer.size}} (see example here: 
{{org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl#close}}).
- Unused import in all changed files.
- {{switchSegmentRecordOffset}} array never updates (except of {{0}} value).

Could you please fix this comments? Thanks.


> Copy only actual amount of data during archiving of WAL segment
> ---
>
> Key: IGNITE-9903
> URL: https://issues.apache.org/jira/browse/IGNITE-9903
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Alexey Stelmak
>Priority: Major
> Fix For: 2.8
>
>
> Current WAL archiver implementation copies full WAL segment to the archive 
> while actual size of the segment can be significantly less then 
> {{maxWalSegmentSize}} (segment files are preallocated for max possible 
> segment size). 
> In order to reduce disk space consuming we need copy only actual data of 
> segment using {{SWITCH_SEGMENT_RECORD}} as marker. It will require some kind 
> of WAL iterator implementation that will just copy WAL records using 
> information about record size without records deserialization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE

2019-01-29 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-11129:
-
Description: 
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
switched on. Size for this record type must be constant.

See 
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:

{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);

if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
size */ + REC_TYPE_SIZE;

return clSz;
}
{code}

  was:
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
switched on. Size for this record type should be constant.

See 
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:

{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);

if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
size */ + REC_TYPE_SIZE;

return clSz;
}
{code}


> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
> 
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
> switched on. Size for this record type must be constant.
> See 
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException 
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE

2019-01-29 Thread Andrey Gura (JIRA)
Andrey Gura created IGNITE-11129:


 Summary: Incorrect size calculation for SWITCH_SEGMENT_RECORD for 
TDE
 Key: IGNITE-11129
 URL: https://issues.apache.org/jira/browse/IGNITE-11129
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Gura


Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
switched on. Size for this record type should be constant.
See 
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:

{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);

if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
size */ + REC_TYPE_SIZE;

return clSz;
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE

2019-01-29 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-11129:
-
Description: 
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
switched on. Size for this record type should be constant.

See 
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:

{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);

if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
size */ + REC_TYPE_SIZE;

return clSz;
}
{code}

  was:
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
switched on. Size for this record type should be constant.
See 
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:

{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);

if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
size */ + REC_TYPE_SIZE;

return clSz;
}
{code}


> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
> 
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
>  Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption 
> switched on. Size for this record type should be constant.
> See 
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException 
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data 
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (IGNITE-10938) After restart cluster with non-blt nodes - they left by handler

2019-01-21 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-10938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura resolved IGNITE-10938.
--
Resolution: Duplicate

> After restart cluster with non-blt nodes - they left by handler
> ---
>
> Key: IGNITE-10938
> URL: https://issues.apache.org/jira/browse/IGNITE-10938
> Project: Ignite
>  Issue Type: Bug
>  Components: data structures
>Affects Versions: 2.8
>Reporter: ARomantsov
>Priority: Critical
> Fix For: 2.8
>
>
> I have cluster wherein topology contain blt and non-blt nodes, but after 
> restart - nodes left by handler
> java.lang.IllegalStateException: Unable to find consistentId by UUID



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (IGNITE-9739) Critical exception in transaction processing in case we have nodes out of baseline and non-persisted cache

2019-01-16 Thread Andrey Gura (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Gura updated IGNITE-9739:

Fix Version/s: 2.8

> Critical exception in transaction processing in case we have nodes out of 
> baseline and non-persisted cache
> --
>
> Key: IGNITE-9739
> URL: https://issues.apache.org/jira/browse/IGNITE-9739
> Project: Ignite
>  Issue Type: Bug
>  Components: persistence
>Reporter: Sergey Kosarev
>Assignee: Sergey Kosarev
>Priority: Major
> Fix For: 2.8
>
>
> Activation finished
> {code:java}
> 2018-09-20 20:47:05.169 [INFO 
> ][sys-#307%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
>  Successfully performed final activation steps 
> [nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557, client=false, 
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=1]]
> {code}
> but we have nodes not in base line
> {code:java}
> 2018-09-20 20:45:36.116 [INFO 
> ][sys-#305%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
>  Local node is not included in Baseline Topology and will not be used for 
> persistent data storage. Use control.(sh|bat) script or IgniteCluster 
> interface to include the node to Baseline Topology.
> {code}
> And we have cache (869481129) in the data region with persistanceEnabled=false
> {code:java}
> 2018-09-20 20:49:01.825 [INFO 
> ][exchange-worker-#154%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheProcessor]
>  Started cache [name=DPL_PUBLISHED_CACHES_REGISTRY$, *id=869481129*, group=SY
> STEM_CACHEGROUP_PUBLISHED_REGISTRY, memoryPolicyName=not-persisted, 
> mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=3]
> {code}
> Transaction on this cache(869481129)
> {code:java}
> 869481129{code}
> leads to critical error causing nodes by faulure handler:
> {code:java}
> 2018-09-20 20:50:24.275 
> [ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
>  Failed processing message [senderId=62e986f0-62b5-4ec8-8cc7-27b74d345235, 
> msg=GridDhtTxPrepareRequest [nearNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774, 
> futId=520e308f561-255fdea5-a996-4102-a120-afa380c54570, miniId=1, 
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2], 
> invalidateNearEntries={}, nearWrites=null, owned=null, 
> nearXidVer=GridCacheVersion [topVer=148944365, order=1537511036821, 
> nodeOrder=132], subjId=814af7c4-2de5-4511-b1ea-065b91eaa774, taskNameHash=0, 
> preloadKeys=null, skipCompletedVers=false, 
> super=GridDistributedTxPrepareRequest [threadId=58, concurrency=PESSIMISTIC, 
> isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=148944365, 
> order=1537511036824, nodeOrder=7], timeout=299970, reads=null, 
> writes=ArrayList [
> IgniteTxEntry [key=KeyCacheObjectImpl [part=27254, 
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true], 
> *cacheId=869481129*,
> txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=27254, 
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true], 
> *cacheId=869481129*], val=[op=CREATE, 
> val=com.sbt.dpl.gridgain.PublishedRegistry$PublishedCacheTuple 
> [idHash=811765531, hash=1522508040, 
> cacheName=com.sbt.gbk.entities.DocType_DPL_union-module,indexes=ArrayList 
> {com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=1583970836, hash=363194492, isSoftReference=false, 
> unselectiveBuckets=4096, fieldNames=ArrayList 
> \{isDeleted},moduleName=union-module
> , cachedUnselectives=1, selectors=ArrayList {isDeleted}, 
> exceptUnselectives=false, primitiveCollection=false, isVersioned=false, 
> isComposite=false, isSystemTypeBelongs=false,
> name=com.sbt.gbk.entities.DocType_DPL_isDeleted, isIndexedCollection=false, 
> isGlobal=false, maxSelective=1000], 
> com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=2060926101, hash=1983794578, isSoftReference=false, 
> unselectiveBuckets=4096, fieldNames=ArrayList ,moduleName=union-module, 
> cachedUnselectives=1, selectors=ArrayList, exceptUnselectives=false, 
> primitiveCollection=false, isVersioned=false, isComposite=false, 
> isSystemTypeBelongs=false, name=com.sbt.gbk.entities.DocType_DPL_code, 
> isIndexedCollection=false, isGlobal=true, maxSelective=1000]
> , com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=1821682714, hash=-1245813786, isSoftReference=false, 
> unselectiveBuckets=4096, fieldNames=ArrayList {globalId},
> moduleName=union-module, cachedUnselectives=1, selectors=ArrayList 
> {globalId}, exceptUnselectives=false, primitiveCollection=false, 
> isVersioned=false, isComposite=false, isSystemTypeBelongs=false,
> name=com.sbt.gbk.entities.DocType_DPL_globalId, isIndexedCollection=false, 
> isGlobal=fal

  1   2   3   4   5   6   7   8   9   10   >