[jira] [Updated] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
[
https://issues.apache.org/jira/browse/IGNITE-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12197:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
> --
>
> Key: IGNITE-12197
> URL: https://issues.apache.org/jira/browse/IGNITE-12197
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Minor
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> IGNITE-12027 introduces possible bug due to incorrect way for getting value
> of persistent enabled property in {{CacheGroupMetricsImpl}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12231) RollbackRecord's must be flushed after logging to WAL
Andrey Gura created IGNITE-12231:
Summary: RollbackRecord's must be flushed after logging to WAL
Key: IGNITE-12231
URL: https://issues.apache.org/jira/browse/IGNITE-12231
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
Fix For: 2.8
Every record or records batch logged to WAL must call {{wal().flush()}} in
order to save data on storage device.
{{TxPartitionCounterStateOnePrimaryTwoBackupsHistoryRebalanceTest.testPartialPrepare_2TX_1_1}}
test fails periodically with disabled MMAP.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12199) WAL record with data entries doesn't flushes on backups for transactional cache
[
https://issues.apache.org/jira/browse/IGNITE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12199:
-
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
> WAL record with data entries doesn't flushes on backups for transactional
> cache
> ---
>
> Key: IGNITE-12199
> URL: https://issues.apache.org/jira/browse/IGNITE-12199
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> WAL record with data entries doesn't flushes on backups for transactional
> cache.
> This issue can be reproduced for example by
> {{TxPartitionCounterStateConsistencyTest.testSingleThreadedUpdateOrder}} test
> with disabled MMAP mode.
> Problem place in code is {{GridDistributedTxRemoteAdapter#commitIfLocked}}
> where {{wal.log()}} doesn't assign returned file pointer to the {{ptr}}
> variable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12145) [IEP-35] Monitoring list engine
[ https://issues.apache.org/jira/browse/IGNITE-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934307#comment-16934307 ] Andrey Gura commented on IGNITE-12145: -- [~NIzhikov] Nikolay, looks good to me. Thanks for contribution and thanks to all who was involved. > [IEP-35] Monitoring list engine > --- > > Key: IGNITE-12145 > URL: https://issues.apache.org/jira/browse/IGNITE-12145 > Project: Ignite > Issue Type: Sub-task >Affects Versions: 2.7.5 >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Fix For: 2.8 > > Time Spent: 19h 10m > Remaining Estimate: 0h > > The base engine for the monitoring list. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[ https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura reassigned IGNITE-11927: Assignee: Andrey Gura (was: Nikolay Izhikov) > [IEP-35] Add ability to enable\disable subset of metrics > > > Key: IGNITE-11927 > URL: https://issues.apache.org/jira/browse/IGNITE-11927 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Andrey Gura >Priority: Major > Labels: IEP-35 > Time Spent: 20m > Remaining Estimate: 0h > > Ignite should be able to: > * Enable or disable an arbitrary subset of the metrics. User should be able > to do it in runtime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
[
https://issues.apache.org/jira/browse/IGNITE-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12197:
-
Labels: IEP-35 (was: )
> Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
> --
>
> Key: IGNITE-12197
> URL: https://issues.apache.org/jira/browse/IGNITE-12197
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> IGNITE-12027 introduces possible bug due to incorrect way for getting value
> of persistent enabled property in {{CacheGroupMetricsImpl}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12199) WAL record with data entries doesn't flushes on backups for transactional cache
Andrey Gura created IGNITE-12199:
Summary: WAL record with data entries doesn't flushes on backups
for transactional cache
Key: IGNITE-12199
URL: https://issues.apache.org/jira/browse/IGNITE-12199
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
Fix For: 2.8
WAL record with data entries doesn't flushes on backups for transactional cache.
This issue can be reproduced for example by
{{TxPartitionCounterStateConsistencyTest.testSingleThreadedUpdateOrder}} test
with disabled MMAP mode.
Problem place in code is {{GridDistributedTxRemoteAdapter#commitIfLocked}}
where {{wal.log()}} doesn't assign returned file pointer to the {{ptr}}
variable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler
[
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura resolved IGNITE-12198.
--
Release Note: By design, accordingly to configured segmentation policy.
Resolution: Won't Fix
> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> stopNodeHnd);
> break; {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler
[
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932885#comment-16932885
]
Andrey Gura commented on IGNITE-12198:
--
By design, accordingly to configured segmentation policy.
> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> stopNodeHnd);
> break; {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler
[
https://issues.apache.org/jira/browse/IGNITE-12198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12198:
-
Release Note: (was: By design, accordingly to configured segmentation
policy. )
> GridDiscoveryManager uses hardcoded failure handler
> ---
>
> Key: IGNITE-12198
> URL: https://issues.apache.org/jira/browse/IGNITE-12198
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Rohit Joshi
>Priority: Major
>
> GridDiscoveryManager.onSegmentation() explicitly passes
> StopNodeFailureHandler to FailureProcessor overriding the failureHandler
> provided in IgniteConfiguration.
> {code:java}
> case RESTART_JVM:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> restartProcHnd);
> break;
> case STOP:
> ctx.failure().process(new FailureContext(FailureType.SEGMENTATION, null),
> stopNodeHnd);
> break; {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
[
https://issues.apache.org/jira/browse/IGNITE-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12197:
-
Description: IGNITE-12027 introduces possible bug due to incorrect way for
getting value of persistent enabled property in {{CacheGroupMetricsImpl}}
(was: IGNITE-12027 introduce possible bug due to incorrect way for getting
value of persistent enabled property in {{CacheGroupMetricsImpl}})
> Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
> --
>
> Key: IGNITE-12197
> URL: https://issues.apache.org/jira/browse/IGNITE-12197
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Major
> Fix For: 2.8
>
>
> IGNITE-12027 introduces possible bug due to incorrect way for getting value
> of persistent enabled property in {{CacheGroupMetricsImpl}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl
Andrey Gura created IGNITE-12197:
Summary: Incorrect way for getting value of persistent enabled in
CacheGroupMetricsImpl
Key: IGNITE-12197
URL: https://issues.apache.org/jira/browse/IGNITE-12197
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
Fix For: 2.8
IGNITE-12027 introduce possible bug due to incorrect way for getting value of
persistent enabled property in {{CacheGroupMetricsImpl}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12172) [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite
[
https://issues.apache.org/jira/browse/IGNITE-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12172:
-
Labels: IEP-35 (was: )
> [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite
> --
>
> Key: IGNITE-12172
> URL: https://issues.apache.org/jira/browse/IGNITE-12172
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> {{OpenCensusMetricExporterSpiTest}} isn't added to any test suite. As result
> tests do not run on TC.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (IGNITE-12172) [IEP-35] OpenCensusMetricExporterSpiTest isn't added to any test suite
Andrey Gura created IGNITE-12172:
Summary: [IEP-35] OpenCensusMetricExporterSpiTest isn't added to
any test suite
Key: IGNITE-12172
URL: https://issues.apache.org/jira/browse/IGNITE-12172
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Fix For: 2.8
{{OpenCensusMetricExporterSpiTest}} isn't added to any test suite. As result
tests do not run on TC.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-12171) Remove the MetricRegistry.remove(String name) method.
[
https://issues.apache.org/jira/browse/IGNITE-12171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930419#comment-16930419
]
Andrey Gura commented on IGNITE-12171:
--
Will done as part of IGNITE-11927 issue. Metric registries will be immutable.
See also dev-list topic
http://apache-ignite-developers.2346864.n4.nabble.com/IEP-35-Metrics-management-in-Ignite-td43243.html
> Remove the MetricRegistry.remove(String name) method.
> -
>
> Key: IGNITE-12171
> URL: https://issues.apache.org/jira/browse/IGNITE-12171
> Project: Ignite
> Issue Type: Task
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I suggest removing the unused {{MetricRegistry.remove(String name)}} method
> due to we can remove a subset of metrics using the
> {{GridMetricManager.remove(String regName)}} method with notifying mechanism.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (IGNITE-2636) Server cache metrics for put-get-remove avg time are incorrect for case when request sent from client
[ https://issues.apache.org/jira/browse/IGNITE-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura resolved IGNITE-2636. - Resolution: Duplicate > Server cache metrics for put-get-remove avg time are incorrect for case when > request sent from client > - > > Key: IGNITE-2636 > URL: https://issues.apache.org/jira/browse/IGNITE-2636 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 1.5.0.final >Reporter: Vladimir Ershov >Priority: Major > Original Estimate: 120h > Remaining Estimate: 120h > > Server cache metrics for put-get-remove avg time are incorrect for case when > request sent from client. > We should add methods like CacheMetrics#addPutAndGetTimeNanos for all flows, > when requests for cache modifications are processed. For all type of caches. > This will fix CacheMetricsForClusterGroupSelfTest -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (IGNITE-12127) WAL writer may close file IO with unflushed changes when MMAP is disabled
[
https://issues.apache.org/jira/browse/IGNITE-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923405#comment-16923405
]
Andrey Gura commented on IGNITE-12127:
--
[~DmitriyGovorukhin] LGTM. Thanks for contribution! Please merge to master
branch.
> WAL writer may close file IO with unflushed changes when MMAP is disabled
> -
>
> Key: IGNITE-12127
> URL: https://issues.apache.org/jira/browse/IGNITE-12127
> Project: Ignite
> Issue Type: Bug
>Reporter: Dmitriy Govorukhin
>Assignee: Dmitriy Govorukhin
>Priority: Critical
> Fix For: 2.7.6
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Most likely the issue manifests itself as the following critical error:
> {code}
> 2019-08-27 14:52:31.286 ERROR 26835 --- [wal-write-worker%null-#447] ROOT :
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler,
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.StorageException: Failed to write
> buffer.]]
> org.apache.ignite.internal.processors.cache.persistence.StorageException:
> Failed to write buffer.
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.writeBuffer(FileWriteAheadLogManager.java:3444)
> [ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.body(FileWriteAheadLogManager.java:3249)
> [ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> [ignite-core-2.5.7.jar!/:2.5.7]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201]
> Caused by: java.nio.channels.ClosedChannelException: null
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
> ~[na:1.8.0_201]
> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:253)
> ~[na:1.8.0_201]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.position(RandomAccessFileIO.java:48)
> ~[ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FileIODecorator.position(FileIODecorator.java:41)
> ~[ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.AbstractFileIO.writeFully(AbstractFileIO.java:111)
> ~[ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.writeBuffer(FileWriteAheadLogManager.java:3437)
> [ignite-core-2.5.7.jar!/:2.5.7]
> ... 3 common frames omitted
> {code}
> It appears that there following sequence is possible:
> * Thread A attempts to log a large record which does not fit segment,
> {{addRecord}} fails and the thread A starts segment rollover. I successfully
> runs {{flushOrWait(null)}} and gets de-scheduled before adding switch segment
> record
> * Thread B attempts to log another record, which fits exactly till the end
> of the current segment. The record is added to the buffer
> * Thread A resumes and fails to add the switch segment record. No flush is
> performed and the thread immediately proceeds for wal-writer close
> * WAL writer thread wakes up, sees that there is a CLOSE request, closes the
> file IO and immediately proceeds to write unflushed changes causing the
> exception.
> Unconditional flush after switch segment record write should fix the issue.
> Besides the bug itself, I suggest the following changes to the
> {{FileWriteHandleImpl}} ({{FileWriteAheadLogManager}} in earlier versions):
> * There is an {{fsync(filePtr)}} call inside {{close()}}; however,
> {{fsync()}} checks the {{stop}} flag (which is set inside {{close}}) and
> returns immediately after {{flushOrWait()}} if the flag is set - this is very
> confusing. After all, the {{close()}} itself explicitly calls {{force}} after
> flush
> * There is an ignored IO exception in mmap mode - this should be propagated
> to the failure handler
> * In WAL writer, we check for file CLOSE and then attemp to write to
> (possibly) the same write handle - write should be always before close
> * In WAL writer, there are racy reads of current handle - it would be better
> if we read the current handle once and then operate on it during the whole
> loop iteration
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
[
https://issues.apache.org/jira/browse/IGNITE-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura resolved IGNITE-12130.
--
Resolution: Won't Fix
> Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
>
>
> Key: IGNITE-12130
> URL: https://issues.apache.org/jira/browse/IGNITE-12130
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> Open Census metric exporter and tracing SPI's have incorrect packages
> structure: {{org.apache.ignite.opencensus.spi}}.
> Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed
> under org.apache.ignite.spi}} package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
[
https://issues.apache.org/jira/browse/IGNITE-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919691#comment-16919691
]
Andrey Gura commented on IGNITE-12130:
--
[~NSAmelchev] You are right! Closed as wont fix.
> Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
>
>
> Key: IGNITE-12130
> URL: https://issues.apache.org/jira/browse/IGNITE-12130
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> Open Census metric exporter and tracing SPI's have incorrect packages
> structure: {{org.apache.ignite.opencensus.spi}}.
> Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed
> under org.apache.ignite.spi}} package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager
[ https://issues.apache.org/jira/browse/IGNITE-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919683#comment-16919683 ] Andrey Gura commented on IGNITE-12131: -- [~ilyak] In my opinion, mmap is just implementation detail, so it isn't good idea to expose it on public configuration API. There are ideas about WAL refactoring that can lead to creation of different implementations. > MMAP mode should be disabled by default for WAL manager > --- > > Key: IGNITE-12131 > URL: https://issues.apache.org/jira/browse/IGNITE-12131 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Gura >Assignee: Andrey Gura >Priority: Major > Fix For: 2.8 > > > MMAP mode has a bunch of problems (see for example > http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ ) > Users are increasingly stumbling over these issues especially in virtualized > environments. > MMAP should be disabled by default because it requires additional care from > user stand point. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager
[ https://issues.apache.org/jira/browse/IGNITE-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura updated IGNITE-12131: - Description: MMAP mode has a bunch of problems (see for example http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ ) Users are increasingly stumbling over these issues especially in virtualized environments. MMAP should be disabled by default because it requires additional care from user stand point. was: MMAP mode has a bunch of problems (see for example http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ ) Users are increasingly stumbling over these issues especially in virtualized environments. MMAP should be disabled by default because it require additional care from user stand point. > MMAP mode should be disabled by default for WAL manager > --- > > Key: IGNITE-12131 > URL: https://issues.apache.org/jira/browse/IGNITE-12131 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Gura >Assignee: Andrey Gura >Priority: Major > Fix For: 2.8 > > > MMAP mode has a bunch of problems (see for example > http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ ) > Users are increasingly stumbling over these issues especially in virtualized > environments. > MMAP should be disabled by default because it requires additional care from > user stand point. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager
[ https://issues.apache.org/jira/browse/IGNITE-12131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura reassigned IGNITE-12131: Assignee: Andrey Gura > MMAP mode should be disabled by default for WAL manager > --- > > Key: IGNITE-12131 > URL: https://issues.apache.org/jira/browse/IGNITE-12131 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Gura >Assignee: Andrey Gura >Priority: Major > Fix For: 2.8 > > > MMAP mode has a bunch of problems (see for example > http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ ) > Users are increasingly stumbling over these issues especially in virtualized > environments. > MMAP should be disabled by default because it require additional care from > user stand point. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-12131) MMAP mode should be disabled by default for WAL manager
Andrey Gura created IGNITE-12131: Summary: MMAP mode should be disabled by default for WAL manager Key: IGNITE-12131 URL: https://issues.apache.org/jira/browse/IGNITE-12131 Project: Ignite Issue Type: Improvement Reporter: Andrey Gura Fix For: 2.8 MMAP mode has a bunch of problems (see for example http://www.mapdb.org/blog/mmap_files_alloc_and_jvm_crash/ ) Users are increasingly stumbling over these issues especially in virtualized environments. MMAP should be disabled by default because it require additional care from user stand point. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
[
https://issues.apache.org/jira/browse/IGNITE-12130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12130:
-
Description:
Open Census metric exporter and tracing SPI's have incorrect packages
structure: {{org.apache.ignite.opencensus.spi}}.
Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed
under org.apache.ignite.spi}} package.
was:
Open Census metric exporter and tracing SPI's have incorrect packages
structure: {{org.apache.ignite.opencensus.spi}}.
Shold be {{org.apache.ignite.spi.opencensus}}.
> Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
>
>
> Key: IGNITE-12130
> URL: https://issues.apache.org/jira/browse/IGNITE-12130
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> Open Census metric exporter and tracing SPI's have incorrect packages
> structure: {{org.apache.ignite.opencensus.spi}}.
> Should be {{org.apache.ignite.spi.opencensus}}. All other SPI's are placed
> under org.apache.ignite.spi}} package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (IGNITE-12130) Incorrect packages structure for OpenCensus metric exporter and tracing SPIs
Andrey Gura created IGNITE-12130:
Summary: Incorrect packages structure for OpenCensus metric
exporter and tracing SPIs
Key: IGNITE-12130
URL: https://issues.apache.org/jira/browse/IGNITE-12130
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Fix For: 2.8
Open Census metric exporter and tracing SPI's have incorrect packages
structure: {{org.apache.ignite.opencensus.spi}}.
Shold be {{org.apache.ignite.spi.opencensus}}.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.
[
https://issues.apache.org/jira/browse/IGNITE-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906239#comment-16906239
]
Andrey Gura commented on IGNITE-12046:
--
[~NIzhikov] Thanks! No we shouldn't. Only {{LongMetric}} interface had this
methods.
> [IEP-35] LongMetric interface contains redundant methods.
> -
>
> Key: IGNITE-12046
> URL: https://issues.apache.org/jira/browse/IGNITE-12046
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Trivial
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following methods on {{LongMetric}} interface a redundant and have the
> same functionality:
> * {{get}}
> * {{longValue}}
> Mentioned methods should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.
[
https://issues.apache.org/jira/browse/IGNITE-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905668#comment-16905668
]
Andrey Gura commented on IGNITE-12046:
--
[~NIzhikov] Could you please review?
> [IEP-35] LongMetric interface contains redundant methods.
> -
>
> Key: IGNITE-12046
> URL: https://issues.apache.org/jira/browse/IGNITE-12046
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Trivial
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following methods on {{LongMetric}} interface a redundant and have the
> same functionality:
> * {{get}}
> * {{longValue}}
> Mentioned methods should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Issue Comment Deleted] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.
[
https://issues.apache.org/jira/browse/IGNITE-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12046:
-
Comment: was deleted
(was: {panel:title=Branch: [pull/6751/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 6{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4492559]]
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4492028buildTypeId=IgniteTests24Java8_RunAll])
> [IEP-35] LongMetric interface contains redundant methods.
> -
>
> Key: IGNITE-12046
> URL: https://issues.apache.org/jira/browse/IGNITE-12046
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Andrey Gura
>Priority: Trivial
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following methods on {{LongMetric}} interface a redundant and have the
> same functionality:
> * {{get}}
> * {{longValue}}
> Mentioned methods should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (IGNITE-12016) Is there option to increase or decrease Ignite SQL Table backup count after its initialized?
[ https://issues.apache.org/jira/browse/IGNITE-12016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura resolved IGNITE-12016. -- Resolution: Not A Bug > Is there option to increase or decrease Ignite SQL Table backup count after > its initialized? > > > Key: IGNITE-12016 > URL: https://issues.apache.org/jira/browse/IGNITE-12016 > Project: Ignite > Issue Type: New Feature >Reporter: hirik >Priority: Major > > Hi, > We are working in a dynamic environment, based on the new nodes we should > update the backup count of existing SQL tables. is there any api available to > support this requirement? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (IGNITE-12011) MetricUpdater timeout can't be configured
[
https://issues.apache.org/jira/browse/IGNITE-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12011:
-
Priority: Minor (was: Major)
> MetricUpdater timeout can't be configured
> -
>
> Key: IGNITE-12011
> URL: https://issues.apache.org/jira/browse/IGNITE-12011
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Max Shonichev
>Priority: Minor
> Fix For: 3.0
>
>
> {noformat}
> /** Metrics update frequency. */
> private static final long METRICS_UPDATE_FREQ = 3000;
> ...
> /** {@inheritDoc} */
> @Override protected void onKernalStart0() throws IgniteCheckedException {
> metricsUpdateTask = ctx.timeout().schedule(new MetricsUpdater(),
> METRICS_UPDATE_FREQ, METRICS_UPDATE_FREQ);
> }
> {noformat}
> Would be great if metric updater timeout was externally configured.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12046) [IEP-35] LongMetric interface contains redundant methods.
Andrey Gura created IGNITE-12046:
Summary: [IEP-35] LongMetric interface contains redundant methods.
Key: IGNITE-12046
URL: https://issues.apache.org/jira/browse/IGNITE-12046
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Assignee: Andrey Gura
Fix For: 2.8
The following methods on {{LongMetric}} interface a redundant and have the same
functionality:
* {{get}}
* {{longValue}}
Mentioned methods should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-11925) [IEP-35] Migrage QueryMetrics
[ https://issues.apache.org/jira/browse/IGNITE-11925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura updated IGNITE-11925: - Fix Version/s: 2.8 > [IEP-35] Migrage QueryMetrics > - > > Key: IGNITE-11925 > URL: https://issues.apache.org/jira/browse/IGNITE-11925 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Fix For: 2.8 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > After merging of IGNITE-11848 we should migrate `QueryMetrics` to the new > metric framework. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read
[
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura reassigned IGNITE-11687:
Assignee: Dmitriy Govorukhin (was: Andrey Gura)
> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Dmitriy Govorukhin
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been
> already updated, concurrent WAL replay will attempt to read the incompletely
> written record and since {{end}} is not null, iterator will fail with CRC
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12045) [IEP-35] JmxExporterSpi has too broad name.
Andrey Gura created IGNITE-12045:
Summary: [IEP-35] JmxExporterSpi has too broad name.
Key: IGNITE-12045
URL: https://issues.apache.org/jira/browse/IGNITE-12045
Project: Ignite
Issue Type: Bug
Reporter: Andrey Gura
Fix For: 2.8
{{JmxExporterSpi}} has too broad name. It should be renamed to
{{JmxMetricExporterSpi}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12044) [IEP-35] JmxExporterSpi displays histogram values incorrectly
[
https://issues.apache.org/jira/browse/IGNITE-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12044:
-
Description:
{{JmxExporterSpi}} exposes histogram values incorrectly. It looks like:
{noformat}
durationHistogramlong[5]
{noformat}
I think exporter should register attribute for each histogram bucket. Something
like this:
{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}
was:
JmxExporterSpi exposes histogram values incorrectly. It looks like:
{noformat}
durationHistogramlong[5]
{noformat}
I think exporter should register attribute for each histogram bucket. Something
like this:
{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}
> [IEP-35] JmxExporterSpi displays histogram values incorrectly
> -
>
> Key: IGNITE-12044
> URL: https://issues.apache.org/jira/browse/IGNITE-12044
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> {{JmxExporterSpi}} exposes histogram values incorrectly. It looks like:
> {noformat}
> durationHistogramlong[5]
> {noformat}
> I think exporter should register attribute for each histogram bucket.
> Something like this:
> {noformat}
> durationHistogram.10 0
> durationHistogram.50 0
> durationHistogram.1000
> durationHistogram.2500
> durationHistogram.5000
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12044) [IEP-35] JmxExporterSpi displays histogram values incorrectly
[
https://issues.apache.org/jira/browse/IGNITE-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12044:
-
Description:
JmxExporterSpi exposes histogram values incorrectly. It looks like:
{noformat}
durationHistogramlong[5]
{noformat}
I think exporter should register attribute for each histogram bucket. Something
like this:
{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}
was:
JmxExporterSpi exposes histogram values incorrectly. It looks like:
{noformat}
durationHistogramlong[5]
{noformat}
I think exporter should register attribute for each histogram bucket. Something
like this:
{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}
> [IEP-35] JmxExporterSpi displays histogram values incorrectly
> -
>
> Key: IGNITE-12044
> URL: https://issues.apache.org/jira/browse/IGNITE-12044
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> JmxExporterSpi exposes histogram values incorrectly. It looks like:
> {noformat}
> durationHistogramlong[5]
> {noformat}
> I think exporter should register attribute for each histogram bucket.
> Something like this:
> {noformat}
> durationHistogram.10 0
> durationHistogram.50 0
> durationHistogram.1000
> durationHistogram.2500
> durationHistogram.5000
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12044) [IEP-35] JmxExporterSpi displays histogram values incorrectly
Andrey Gura created IGNITE-12044:
Summary: [IEP-35] JmxExporterSpi displays histogram values
incorrectly
Key: IGNITE-12044
URL: https://issues.apache.org/jira/browse/IGNITE-12044
Project: Ignite
Issue Type: Bug
Reporter: Andrey Gura
Fix For: 2.8
JmxExporterSpi exposes histogram values incorrectly. It looks like:
{noformat}
durationHistogramlong[5]
{noformat}
I think exporter should register attribute for each histogram bucket. Something
like this:
{noformat}
durationHistogram.10 0
durationHistogram.50 0
durationHistogram.1000
durationHistogram.2500
durationHistogram.5000
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12043) Metrics: JMX exporter reports incorrect description.
[
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12043:
-
Labels: IEP-35 newbie (was: IEP-35)
> Metrics: JMX exporter reports incorrect description.
>
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
> Labels: IEP-35, newbie
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see
> {{description = .}} instead of human readable
> description, specified by metric developer.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12043) [IEP-35] Metrics: JMX exporter reports incorrect description.
[
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12043:
-
Summary: [IEP-35] Metrics: JMX exporter reports incorrect description.
(was: Metrics: JMX exporter reports incorrect description.)
> [IEP-35] Metrics: JMX exporter reports incorrect description.
> -
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
> Labels: IEP-35, newbie
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see
> {{description = .}} instead of human readable
> description, specified by metric developer.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-11987) [IEP-35] Add ability to configure metrics
[ https://issues.apache.org/jira/browse/IGNITE-11987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900123#comment-16900123 ] Andrey Gura commented on IGNITE-11987: -- [~NIzhikov] I've reviewed PR. See my comments on GitHub. Also I followed up discussion on dev list. > [IEP-35] Add ability to configure metrics > - > > Key: IGNITE-11987 > URL: https://issues.apache.org/jira/browse/IGNITE-11987 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 2h 50m > Remaining Estimate: 0h > > Ignite should be able to: > * Configure Histogram metrics > * Configure HitRate metrics. > We should provide 2 ways to configure metric: > 1. -Configuration file.- Discussed on dev-list. Agreed to go with the > simplest solution - JMX method. > 2. JMX method. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (IGNITE-12043) Metrics: JMX exporter reports incorrect description.
[
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12043:
-
Labels: IEP-35 (was: )
> Metrics: JMX exporter reports incorrect description.
>
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see
> {{description = .}} instead of human readable
> description, specified by metric developer.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12043) Metrics: JMX exporter reports incorrect description.
[
https://issues.apache.org/jira/browse/IGNITE-12043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12043:
-
Fix Version/s: 2.8
> Metrics: JMX exporter reports incorrect description.
>
>
> Key: IGNITE-12043
> URL: https://issues.apache.org/jira/browse/IGNITE-12043
> Project: Ignite
> Issue Type: Bug
>Reporter: Pavel Kuznetsov
>Priority: Major
> Fix For: 2.8
>
> Attachments: screenshoot.png
>
>
> JMX exporter creates bean for each metric. Metric's registration takes human
> readable description field.
> If we open any MBean explorer and get Metadata of any metric, we see
> {{description = .}} instead of human readable
> description, specified by metric developer.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-12028) [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics exporter
[
https://issues.apache.org/jira/browse/IGNITE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900012#comment-16900012
]
Andrey Gura commented on IGNITE-12028:
--
[~NSAmelchev] [~NIzhikov]
Guys, could you please avoid in the future javadoc like this:
{code:java}
/** @return Rate time interval in milliseconds. */
public long rateTimeInterval() {
return cntr.rateTimeInterval;
}
{code}
It is public method so javadoc must contain method description and additionaly
{{return tag}}.
> [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics
> exporter
>
>
> Key: IGNITE-12028
> URL: https://issues.apache.org/jira/browse/IGNITE-12028
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{HitRateMetric}} allows to get only counter value while it would be useful
> to get also {{rateTimeInterval}} in order to export this value as part of
> metric name.
> For example look at cache metric {{RebalancingKeysRate}}. Value of this
> measurement could be exported as smth like this:
> {{cache..RebalancingKeysRate. = }}.
> So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of
> {{LongMetric}} interface.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-10654) Report in case of creating index with already existing fields collection.
[
https://issues.apache.org/jira/browse/IGNITE-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898082#comment-16898082
]
Andrey Gura commented on IGNITE-10654:
--
[~zstan] LGTM. Merged to master branch. Thanks for contribution!
> Report in case of creating index with already existing fields collection.
> -
>
> Key: IGNITE-10654
> URL: https://issues.apache.org/jira/browse/IGNITE-10654
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.7
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Report in log if new index creating with already existing fields collection.
> for example, need to log warn here:
> {code:java}
> cache.query(new SqlFieldsQuery("create index \"idx1\" on Val(keyStr,
> keyLong)"));
> cache.query(new SqlFieldsQuery("create index \"idx3\" on Val(keyStr,
> keyLong)"));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898076#comment-16898076
]
Andrey Gura edited comment on IGNITE-11927 at 8/1/19 1:13 PM:
--
[~NIzhikov] Not certainly in that way. WE don't need NO_OP_METRIC concept
because we don't disable metric itself, we disable whole set of metrics that is
represented by metric registry. So metric registry also should be removed.
Moreover, holder itself can contain logic related with change if metric state.
It should look like:
{code:java}
class MetricHolder {
boolean enabled;
AtomicLongMetric m;
public MetricHolder(GirdKernalContext ctx) {
ctx.monitoring.onDisable("metrics", () -> {
ctx.metric().remove("registryName");
m = null;
}
);
ctx.monitoring.onEnable("metrics", r -> {
MetricRegistry r = new MetricRegistry("registryName");
m = r.longMetric("metric");
ctx.metric().add("registryName");
}
);
}
public long m() {
assert enabled;
return m;
}
public void changeState() {
if (enabled)
m().increment();
}
}
class SomeProcessor {
public SomeProcesor() {
metricHolder = new MetricHolder(ctx);
}
}
{code}
was (Author: agura):
[~NIzhikov] Not certainly in that way. WE don't need NO_OP_METRIC concept
because we don't disable metric itself, we disable whole set of metrics that is
represented by metric registry. So metric registry also should be removed.
Moreover, holder itself can contain logic related with change if metric state.
It should look like:
{code:java}
class MetricHolder {
boolean enabled;
AtomicLongMetric m;
public MetricHolder(GirdKernalContext ctx) {
ctx.monitoring.onDisable("metrics", () -> {
ctx.metric().remove("registryName");
m = null;
}
);
ctx.monitoring.onEnable("metrics", r -> {
MetricRegistry r = new MetricRegistry("registryName");
m = r.longMetric("metric");
ctx.metric().add("registryName");
}
);
}
public long m() {
assert enabled;
return m;
}
public void changeState() {
if (enabled)
m().increment();
}
}
class SomeProcessor {
public SomeProcesor() {
metricHolder = new MetricHolder(ctx);
}
}
{java}
> [IEP-35] Add ability to enable\disable subset of metrics
>
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able
> to do it in runtime.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898076#comment-16898076
]
Andrey Gura commented on IGNITE-11927:
--
[~NIzhikov] Not certainly in that way. WE don't need NO_OP_METRIC concept
because we don't disable metric itself, we disable whole set of metrics that is
represented by metric registry. So metric registry also should be removed.
Moreover, holder itself can contain logic related with change if metric state.
It should look like:
{code:java}
class MetricHolder {
boolean enabled;
AtomicLongMetric m;
public MetricHolder(GirdKernalContext ctx) {
ctx.monitoring.onDisable("metrics", () -> {
ctx.metric().remove("registryName");
m = null;
}
);
ctx.monitoring.onEnable("metrics", r -> {
MetricRegistry r = new MetricRegistry("registryName");
m = r.longMetric("metric");
ctx.metric().add("registryName");
}
);
}
public long m() {
assert enabled;
return m;
}
public void changeState() {
if (enabled)
m().increment();
}
}
class SomeProcessor {
public SomeProcesor() {
metricHolder = new MetricHolder(ctx);
}
}
{java}
> [IEP-35] Add ability to enable\disable subset of metrics
>
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able
> to do it in runtime.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
[
https://issues.apache.org/jira/browse/IGNITE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura resolved IGNITE-12029.
--
Resolution: Duplicate
> [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
> -
>
> Key: IGNITE-12029
> URL: https://issues.apache.org/jira/browse/IGNITE-12029
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> {{HistogramMetric}} returns values as long array where each element
> represents value for some bucket (bounded interval). At present there is no
> way for getting label for bucket (bound value) in order to allow distinguish
> values on user side (in some metrics backend). Should be added method which
> returns bounds values.
> Example:
> We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It
> should be transformed to three metric value:
> * {{metricName.10 = }}
> * {{metricName.50 = }}
> * {{metricName.100 = }}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
[
https://issues.apache.org/jira/browse/IGNITE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897289#comment-16897289
]
Andrey Gura commented on IGNITE-12029:
--
[~NSAmelchev] Thanks! I'll close this ticket as duplicate.
> [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
> -
>
> Key: IGNITE-12029
> URL: https://issues.apache.org/jira/browse/IGNITE-12029
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> {{HistogramMetric}} returns values as long array where each element
> represents value for some bucket (bounded interval). At present there is no
> way for getting label for bucket (bound value) in order to allow distinguish
> values on user side (in some metrics backend). Should be added method which
> returns bounds values.
> Example:
> We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It
> should be transformed to three metric value:
> * {{metricName.10 = }}
> * {{metricName.50 = }}
> * {{metricName.100 = }}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12028) [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics exporter
[
https://issues.apache.org/jira/browse/IGNITE-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12028:
-
Description:
{{HitRateMetric}} allows to get only counter value while it would be useful to
get also {{rateTimeInterval}} in order to export this value as part of metric
name.
For example look at cache metric {{RebalancingKeysRate}}. Value of this
measurement could be exported as smth like this:
{{cache..RebalancingKeysRate. = }}.
So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of
{{LongMetric}} interface.
was:
{{HitRateMetric}} allows to get only counter value while it would be useful to
get also {{rateTimeInterval}} in order to export this value as part of metric
name.
For example look at cache metric {{RebalancingKeysRate}}. Value of this
measurement could be exported as smth like this:
{{cache..RebalancingKeysRate. = }}.
So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of
{{LongMetric}} interface.
> [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics
> exporter
>
>
> Key: IGNITE-12028
> URL: https://issues.apache.org/jira/browse/IGNITE-12028
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> {{HitRateMetric}} allows to get only counter value while it would be useful
> to get also {{rateTimeInterval}} in order to export this value as part of
> metric name.
> For example look at cache metric {{RebalancingKeysRate}}. Value of this
> measurement could be exported as smth like this:
> {{cache..RebalancingKeysRate. = }}.
> So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of
> {{LongMetric}} interface.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
[
https://issues.apache.org/jira/browse/IGNITE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-12029:
-
Environment: (was: {{HistogramMetric}} returns values as long array
where each element represents value for some bucket (bounded interval). At
present there is no way for getting label for bucket (bound value) in order to
allow distinguish values on user side (in some metrics backend). Should be
added method which returns bounds values.
Example:
We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It
should be transformed to three metric value:
* {{metricName.10 = }}
* {{metricName.50 = }}
* {{metricName.100 = }}
)
Description:
{{HistogramMetric}} returns values as long array where each element represents
value for some bucket (bounded interval). At present there is no way for
getting label for bucket (bound value) in order to allow distinguish values on
user side (in some metrics backend). Should be added method which returns
bounds values.
Example:
We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It
should be transformed to three metric value:
* {{metricName.10 = }}
* {{metricName.50 = }}
* {{metricName.100 = }}
> [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
> -
>
> Key: IGNITE-12029
> URL: https://issues.apache.org/jira/browse/IGNITE-12029
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
>
> {{HistogramMetric}} returns values as long array where each element
> represents value for some bucket (bounded interval). At present there is no
> way for getting label for bucket (bound value) in order to allow distinguish
> values on user side (in some metrics backend). Should be added method which
> returns bounds values.
> Example:
> We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It
> should be transformed to three metric value:
> * {{metricName.10 = }}
> * {{metricName.50 = }}
> * {{metricName.100 = }}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12029) [IEP-35] HistogramMetric should privde bounds' values to metrics exporter
Andrey Gura created IGNITE-12029:
Summary: [IEP-35] HistogramMetric should privde bounds' values to
metrics exporter
Key: IGNITE-12029
URL: https://issues.apache.org/jira/browse/IGNITE-12029
Project: Ignite
Issue Type: Bug
Environment: {{HistogramMetric}} returns values as long array where
each element represents value for some bucket (bounded interval). At present
there is no way for getting label for bucket (bound value) in order to allow
distinguish values on user side (in some metrics backend). Should be added
method which returns bounds values.
Example:
We have {{HistogramMetric}} with the following buckets set {10, 50, 100}. It
should be transformed to three metric value:
* {{metricName.10 = }}
* {{metricName.50 = }}
* {{metricName.100 = }}
Reporter: Andrey Gura
Fix For: 2.8
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IGNITE-12028) [IEP-35] HitRateMetric should provide rateTimeInterval value to metrics exporter
Andrey Gura created IGNITE-12028:
Summary: [IEP-35] HitRateMetric should provide rateTimeInterval
value to metrics exporter
Key: IGNITE-12028
URL: https://issues.apache.org/jira/browse/IGNITE-12028
Project: Ignite
Issue Type: Improvement
Reporter: Andrey Gura
Fix For: 2.8
{{HitRateMetric}} allows to get only counter value while it would be useful to
get also {{rateTimeInterval}} in order to export this value as part of metric
name.
For example look at cache metric {{RebalancingKeysRate}}. Value of this
measurement could be exported as smth like this:
{{cache..RebalancingKeysRate. = }}.
So {{HitRateMetric}} should implement {{ObjectMetric}} interface instead of
{{LongMetric}} interface.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-7883) Cluster can have inconsistent affinity configuration
[
https://issues.apache.org/jira/browse/IGNITE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896057#comment-16896057
]
Andrey Gura commented on IGNITE-7883:
-
[~a-polyakov] Thanks for your contribution! Merged to master branch.
> Cluster can have inconsistent affinity configuration
> -
>
> Key: IGNITE-7883
> URL: https://issues.apache.org/jira/browse/IGNITE-7883
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Assignee: Alexand Polyakov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A cluster can have inconsistent affinity configuration if you created two
> nodes, one with affinity key configuration and other without it(in IgniteCfg
> or CacheCfg), both nodes will work fine with no exceptions, but in the same
> time they will apply different affinity rules to keys:
>
> {code:java}
> package affinity;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheAtomicityMode;
> import org.apache.ignite.cache.CacheKeyConfiguration;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.affinity.Affinity;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import java.util.Arrays;
> public class Test {
> private static int id = 0;
> public static void main(String[] args) {
> Ignite ignite = Ignition.start(getConfiguration(true, false));
> Ignite ignite2 = Ignition.start(getConfiguration(false, false));
> Affinity affinity = ignite.affinity("TEST");
> Affinity affinity2 = ignite2.affinity("TEST");
> for (int i = 0; i < 1_000_000; i++) {
> AKey key = new AKey(i);
> if(affinity.partition(key) != affinity2.partition(key))
> System.out.println("FAILED for: " + key);
> }
> System.out.println("DONE");
> }
> private static IgniteConfiguration getConfiguration(boolean
> withAffinityCfg, boolean client) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder(true);
> finder.setAddresses(Arrays.asList("localhost:47500..47600"));
> cfg.setClientMode(client);
> cfg.setIgniteInstanceName("test" + id++);
> CacheConfiguration cacheCfg = new CacheConfiguration("TEST");
> cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> if(withAffinityCfg) {
> cacheCfg.setKeyConfiguration(new
> CacheKeyConfiguration("affinity.AKey", "a"));
> }
> cfg.setCacheConfiguration(cacheCfg);
> cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(finder));
> return cfg;
> }
> }
> class AKey {
> int a;
> public AKey(int a) {
> this.a = a;
> }
> @Override public String toString() {
> return "AKey{" +
> "a=" + a +
> '}';
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-10654) Report in case of creating index with already existing fields collection.
[
https://issues.apache.org/jira/browse/IGNITE-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896039#comment-16896039
]
Andrey Gura commented on IGNITE-10654:
--
[~zstan] [~jooger] I've added some review comments to the PR. Please fix before
merge.
> Report in case of creating index with already existing fields collection.
> -
>
> Key: IGNITE-10654
> URL: https://issues.apache.org/jira/browse/IGNITE-10654
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.7
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Report in log if new index creating with already existing fields collection.
> for example, need to log warn here:
> {code:java}
> cache.query(new SqlFieldsQuery("create index \"idx1\" on Val(keyStr,
> keyLong)"));
> cache.query(new SqlFieldsQuery("create index \"idx3\" on Val(keyStr,
> keyLong)"));
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[ https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887990#comment-16887990 ] Andrey Gura edited comment on IGNITE-11927 at 7/18/19 2:07 PM: --- [~NIzhikov] > Can you, please, make a simple, pseudo-code example of your idea? Caches, for example. Not pseudo-code, but list of action items. On cache start or metrics enabling on cache: - create metrics holder object. - create MetricRegistry instance. - register metrics in MetricRegistry. - add MetricsRegistry in GridMetricManager. On cache stop or metrics disabling for chache: - remove MetricRegistry from GridMetricManager. - assign null to metrics holder object reference. > 1. If we have 5000 caches, Ignite structures already huge. Why do you think > metrics bring a huge impact on GC? Why we should ignore this impact if we can just avoid it without much effort? > 2. All AtomicLong fields are created in previous versions of > CacheMetricsImpl. MetricRegistry is the only addition we made with the new > framework. You are right. But this addition lead to some changes in design. It's good time to improve implementation. Also, I think it would be better design if MetricRegistry will be immutable. It will lead to more clear code structure and behavior. > Do we have some benchmarks or other descriptions of this issue? No, we don't. But obviously all this objects in heap are not free. was (Author: agura): [~NIzhikov] > Can you, please, make a simple, pseudo-code example of your idea? Caches, for example. Not pseudo-code, but list of action items. On cache start or metrics enabling on cache: - create metrics holder object. - create MetricRegistry instance. - register metrics in MetricRegistry. - add MetricsRegistry in GridMetricManager. On cache stop or metrics disabling for chache: - remove MetricRegistry from GridMetricManager. - assign null to metrics holder object reference. > 1. If we have 5000 caches, Ignite structures already huge. Why do you think > metrics bring a huge impact on GC? Why we should ignore this impact if we can just avoid it without much effort? > 2. All AtomicLong fields are created in previous versions of > CacheMetricsImpl. MetricRegistry is the only addition we made with the new > framework. You are right. But this addition lead to some changes in design. It's good time to improve implementation. > Do we have some benchmarks or other descriptions of this issue? No, we don't. But obviously all this objects in heap are not free. > [IEP-35] Add ability to enable\disable subset of metrics > > > Key: IGNITE-11927 > URL: https://issues.apache.org/jira/browse/IGNITE-11927 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > Ignite should be able to: > * Enable or disable an arbitrary subset of the metrics. User should be able > to do it in runtime. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16888011#comment-16888011
]
Andrey Gura commented on IGNITE-11927:
--
BTW, I found one more issue with memory consumption. When metric registers in
metric registry we use metric name without group name as key in metric registry
and metric name concatenated with group name as {{AbstractMetric.name}} field
value. So we consume twice more memory just for metric name.
> [IEP-35] Add ability to enable\disable subset of metrics
>
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able
> to do it in runtime.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[ https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887990#comment-16887990 ] Andrey Gura commented on IGNITE-11927: -- [~NIzhikov] > Can you, please, make a simple, pseudo-code example of your idea? Caches, for example. Not pseudo-code, but list of action items. On cache start or metrics enabling on cache: - create metrics holder object. - create MetricRegistry instance. - register metrics in MetricRegistry. - add MetricsRegistry in GridMetricManager. On cache stop or metrics disabling for chache: - remove MetricRegistry from GridMetricManager. - assign null to metrics holder object reference. > 1. If we have 5000 caches, Ignite structures already huge. Why do you think > metrics bring a huge impact on GC? Why we should ignore this impact if we can just avoid it without much effort? > 2. All AtomicLong fields are created in previous versions of > CacheMetricsImpl. MetricRegistry is the only addition we made with the new > framework. You are right. But this addition lead to some changes in design. It's good time to improve implementation. > Do we have some benchmarks or other descriptions of this issue? No, we don't. But obviously all this objects in heap are not free. > [IEP-35] Add ability to enable\disable subset of metrics > > > Key: IGNITE-11927 > URL: https://issues.apache.org/jira/browse/IGNITE-11927 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > Ignite should be able to: > * Enable or disable an arbitrary subset of the metrics. User should be able > to do it in runtime. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics
[ https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887906#comment-16887906 ] Andrey Gura commented on IGNITE-11927: -- [~NIzhikov] >> 3. There is no need for disabled flag in MetricRegistry. As we discused >> early, when metric disabled they don't consume any memory. MetricsRegistry >> should be collected by GC. After enabling it can be created and registered >> into GridMetricManager again. >I don't understand your proposal. >Metric instances are stored as a class field in the places where they updated. >You can take a GridJobProcessor or CacheMetricImpl as examples. >How and why we should clear these variables on disabling? >Can you provide simple pseudo code for disable\enable processing. At least metric registry can be just removed from registries. On enabling new instance can be created. Ideally, metrics can be moved to the special holder class and reference to it can be null after disabling. Motivation: reducing memory consuming and GC pressure. There are users with big amount of caches (I saw cases with 5000 caches in 200 cache groups). > [IEP-35] Add ability to enable\disable subset of metrics > > > Key: IGNITE-11927 > URL: https://issues.apache.org/jira/browse/IGNITE-11927 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > Ignite should be able to: > * Enable or disable an arbitrary subset of the metrics. User should be able > to do it in runtime. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (IGNITE-11981) [IEP-35] Create MU shortcut instead of static import of MetricUtils
[
https://issues.apache.org/jira/browse/IGNITE-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886250#comment-16886250
]
Andrey Gura commented on IGNITE-11981:
--
[~NIzhikov] As I know there are no any restrictions about static imports.
> [IEP-35] Create MU shortcut instead of static import of MetricUtils
> ---
>
> Key: IGNITE-11981
> URL: https://issues.apache.org/jira/browse/IGNITE-11981
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
>
> More Ignite-way of coding is the usage of short cut classes.
> We should use MU instead of static import of {{MetricUtils}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-11981) [IEP-35] Create MU shortcut instead of static import of MetricUtils
[
https://issues.apache.org/jira/browse/IGNITE-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885326#comment-16885326
]
Andrey Gura commented on IGNITE-11981:
--
IMHO, bad idea and bad practice in the project. There is no problem in typing
MetricUtils :)
> [IEP-35] Create MU shortcut instead of static import of MetricUtils
> ---
>
> Key: IGNITE-11981
> URL: https://issues.apache.org/jira/browse/IGNITE-11981
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
>
> More Ignite-way of coding is the usage of short cut classes.
> We should use MU instead of static import of {{MetricUtils}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics
[
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884683#comment-16884683
]
Andrey Gura edited comment on IGNITE-11927 at 7/14/19 1:53 PM:
---
Anyway, I've took a look and have some comments:
1. There is no need to be disabled or enabled for particular metric value. So
{{disabled}} flag in {{AbstractMetric}} is redundant.
2. We enable or disable metrics for one source of metrics. So we can just
remove {{MetricRegistry}} from {{GridMetricManager}} when metrics disabled.
3. There is no need for {{disabled}} flag in {{MetricRegistry}}. As we discused
early, when metric disabled they don't consume any memory. MetricsRegistry
should be collected by GC. After enabling it can be created and registered into
{{GridMetricManager}} again.
4. Please, separate buckets configuration and metrics enabling/disabling into
two different tickets.
5. Please, create review in Upsource.
6. Please, run TC and take a look to results before code review.
Could you please fix this issues? Thanks.
was (Author: agura):
Anyway, I've took a look and have some comments:
1. There is no need to be disabled or enabled for particular metric value. So
{{disabled}} flag in {{AbstractMetric}} is redundant.
2. We enable or disable metrics for one source of metrics. So we can just
remove {{MetricRegistry}} from {{GridMetricManager}} when metrics disabled.
3. There is no need for {{disabled}} flag in {{MetricRegistry}}. As we discused
early, when metric disabled they don't consume any memory. MetricsRegistry
should be collected by GC. After enabling it can be created and registered into
{{GridMetricManager}} again.
4. Please, separate buckets configuration and metrics enabling/disabling into
two different tickets.
5. Please, create review in Upsource.
6. Please, run TC and take a look to results before code review.
> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the
> simplest solution - JMX method.
> 2. JMX method.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics
[
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884683#comment-16884683
]
Andrey Gura commented on IGNITE-11927:
--
Anyway, I've took a look and have some comments:
1. There is no need to be disabled or enabled for particular metric value. So
{{disabled}} flag in {{AbstractMetric}} is redundant.
2. We enable or disable metrics for one source of metrics. So we can just
remove {{MetricRegistry}} from {{GridMetricManager}} when metrics disabled.
3. There is no need for {{disabled}} flag in {{MetricRegistry}}. As we discused
early, when metric disabled they don't consume any memory. MetricsRegistry
should be collected by GC. After enabling it can be created and registered into
{{GridMetricManager}} again.
4. Please, separate buckets configuration and metrics enabling/disabling into
two different tickets.
5. Please, create review in Upsource.
6. Please, run TC and take a look to results before code review.
> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the
> simplest solution - JMX method.
> 2. JMX method.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics
[ https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884676#comment-16884676 ] Andrey Gura commented on IGNITE-11927: -- [~NIzhikov] What about TC? > [IEP-35] Add ability to configure subset of metrics > --- > > Key: IGNITE-11927 > URL: https://issues.apache.org/jira/browse/IGNITE-11927 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > Ignite should be able to: > * Enable or disable an arbitrary subset of the metrics. User should be able > to do it in runtime. > * Configure Histogram metrics > * Configure HitRate metrics. > We should provide 2 ways to configure metric: > 1. -Configuration file.- Discussed on dev-list. Agreed to go with the > simplest solution - JMX method. > 2. JMX method. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics
[ https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881161#comment-16881161 ] Andrey Gura commented on IGNITE-11927: -- IMHO, histograms and hit rates configuration isn't related with metrics enabling/disabling and it should be done as separate units of work. > [IEP-35] Add ability to configure subset of metrics > --- > > Key: IGNITE-11927 > URL: https://issues.apache.org/jira/browse/IGNITE-11927 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > Ignite should be able to: > * Enable or disable an arbitrary subset of the metrics. User should be able > to do it in runtime. > * Configure Histogram metrics > * Configure HitRate metrics. > We should provide 2 ways to configure metric: > 1. -Configuration file.- Discussed on dev-list. Agreed to go with the > simplest solution - JMX method. > 2. JMX method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-7883) Cluster can have inconsistent affinity configuration
[
https://issues.apache.org/jira/browse/IGNITE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881153#comment-16881153
]
Andrey Gura commented on IGNITE-7883:
-
[~a-polyakov] Could you please rebase your changes on the top of the master
branch and rerun TC? Your changes were made too long ago.
> Cluster can have inconsistent affinity configuration
> -
>
> Key: IGNITE-7883
> URL: https://issues.apache.org/jira/browse/IGNITE-7883
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Assignee: Alexand Polyakov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A cluster can have inconsistent affinity configuration if you created two
> nodes, one with affinity key configuration and other without it(in IgniteCfg
> or CacheCfg), both nodes will work fine with no exceptions, but in the same
> time they will apply different affinity rules to keys:
>
> {code:java}
> package affinity;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheAtomicityMode;
> import org.apache.ignite.cache.CacheKeyConfiguration;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.affinity.Affinity;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import java.util.Arrays;
> public class Test {
> private static int id = 0;
> public static void main(String[] args) {
> Ignite ignite = Ignition.start(getConfiguration(true, false));
> Ignite ignite2 = Ignition.start(getConfiguration(false, false));
> Affinity affinity = ignite.affinity("TEST");
> Affinity affinity2 = ignite2.affinity("TEST");
> for (int i = 0; i < 1_000_000; i++) {
> AKey key = new AKey(i);
> if(affinity.partition(key) != affinity2.partition(key))
> System.out.println("FAILED for: " + key);
> }
> System.out.println("DONE");
> }
> private static IgniteConfiguration getConfiguration(boolean
> withAffinityCfg, boolean client) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder finder = new TcpDiscoveryVmIpFinder(true);
> finder.setAddresses(Arrays.asList("localhost:47500..47600"));
> cfg.setClientMode(client);
> cfg.setIgniteInstanceName("test" + id++);
> CacheConfiguration cacheCfg = new CacheConfiguration("TEST");
> cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> if(withAffinityCfg) {
> cacheCfg.setKeyConfiguration(new
> CacheKeyConfiguration("affinity.AKey", "a"));
> }
> cfg.setCacheConfiguration(cacheCfg);
> cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(finder));
> return cfg;
> }
> }
> class AKey {
> int a;
> public AKey(int a) {
> this.a = a;
> }
> @Override public String toString() {
> return "AKey{" +
> "a=" + a +
> '}';
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11958) JDBC connection validation should use it's own task instead of cache validation task
[ https://issues.apache.org/jira/browse/IGNITE-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878003#comment-16878003 ] Andrey Gura commented on IGNITE-11958: -- [~Denis Chudov] What is motivation for this change? Is there any issues related with it? > JDBC connection validation should use it's own task instead of cache > validation task > > > Key: IGNITE-11958 > URL: https://issues.apache.org/jira/browse/IGNITE-11958 > Project: Ignite > Issue Type: Bug >Reporter: Denis Chudov >Assignee: Denis Chudov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > JDBC connection is validated using GridCacheQueryJdbcValidationTask. We > should create own validation task for this activity. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11945) [IEP-35] Performance drop on cache stop
[ https://issues.apache.org/jira/browse/IGNITE-11945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877990#comment-16877990 ] Andrey Gura commented on IGNITE-11945: -- [~NIzhikov] Looks good to me. > [IEP-35] Performance drop on cache stop > --- > > Key: IGNITE-11945 > URL: https://issues.apache.org/jira/browse/IGNITE-11945 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Time Spent: 10m > Remaining Estimate: 0h > > `MetricRegistry` implementation drop performance on cache stops. > Has to be fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11848) [IEP-35] Monitoring Phase 1
[ https://issues.apache.org/jira/browse/IGNITE-11848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862974#comment-16862974 ] Andrey Gura commented on IGNITE-11848: -- [~NIzhikov] Thanks! LGTM. If you are ready, please, proceed with merge. > [IEP-35] Monitoring Phase 1 > -- > > Key: IGNITE-11848 > URL: https://issues.apache.org/jira/browse/IGNITE-11848 > Project: Ignite > Issue Type: Task >Affects Versions: 2.7 >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > Fix For: 2.8 > > Time Spent: 6h 50m > Remaining Estimate: 0h > > Umbrella ticket for the IEP-35. Monitoring and profiling. > Phase 1 should include: > * NextGen monitoring subsystem implementation to manage > ** metrics > ** -lists- (will be implemented in the following tickets) > ** exporters > * JMX, SQLView, Log exporters > * Migration of existing metrics to new manager > * -Lists for all Ignite user API- -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11899) Fix code style violation
[
https://issues.apache.org/jira/browse/IGNITE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858745#comment-16858745
]
Andrey Gura commented on IGNITE-11899:
--
[~Mmuzaf] Agree with proposed steps.
About help in fixing of created issues. Isn't problem. But magic numbers is
still magic for me :) I think we need help of [~DmitriyGovorukhin].
> Fix code style violation
>
>
> Key: IGNITE-11899
> URL: https://issues.apache.org/jira/browse/IGNITE-11899
> Project: Ignite
> Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All the code style issues must be fixed, see the sample below.
> {code}
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:50:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:183:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:187:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:191:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> {code}
> TC log:
> https://ci.ignite.apache.org/viewLog.html?buildId=4062737=IgniteTests24Java8_CheckCodeStyle=buildLog_IgniteTests24Java8=%3Cdefault%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-11899) Fix code style violation
[
https://issues.apache.org/jira/browse/IGNITE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858627#comment-16858627
]
Andrey Gura edited comment on IGNITE-11899 at 6/7/19 1:18 PM:
--
[~Mmuzaf] Just took a quick look at your change. The first, thanks a lot for
care about code style.
I want to comment some things that should be fixed also. Notice that my
comments addressed to the code author not to you.
1. Using one-line Javadoc for class fields.
{code:java}
/**
*
*/
private static final int PAGE_ID_OFFSET = 0;
{code}
Can be replaced by
{code:java}
/** */
private static final int PAGE_ID_OFFSET = 0;
{code}
Moreover, empty Javadoc comment it's very bad practice, so I believe that all
comments like this should be replaced by meaningful comment.
2. Magic numbers in some file should be replaced by constants with
self-descriptive names.
Just an example:
{code:java}
MemoryCalculator(){
onHeapAllocated(16 + (8 + 16) * 2);
}
{code}
I believe that this arithmetic expression can be rewritten in more clear manner.
was (Author: agura):
[~Mmuzaf] Just took a quick look at your change. The first, thanks a lot for
care about code style.
I want to comment some things that should be fixed also. Notice that my
comments addressed to the code author not to you.
1. Using one-line Javadoc for class fields.
{code:java}
/**
*
*/
private static final int PAGE_ID_OFFSET = 0;
{code}
Can be replaced by
{code:java}
/** */
private static final int PAGE_ID_OFFSET = 0;
{code}
Moreover, empty Javadoc comment it's very bad practice, so I believe that all
comments like this should be replaced by meaningful comment.
2. Magic numbers in some file should be replaced by constants with
self-descriptive names.
Just an example:
{code:java}
MemoryCalculator(){
onHeapAllocated(16 + (8 + 16) * 2);
}
{code}
I believe that this arithmetic expression can be rewritten in more clear manner.
> Fix code style violation
>
>
> Key: IGNITE-11899
> URL: https://issues.apache.org/jira/browse/IGNITE-11899
> Project: Ignite
> Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All the code style issues must be fixed, see the sample below.
> {code}
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:50:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:183:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:187:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:191:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> {code}
> TC log:
> https://ci.ignite.apache.org/viewLog.html?buildId=4062737=IgniteTests24Java8_CheckCodeStyle=buildLog_IgniteTests24Java8=%3Cdefault%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11899) Fix code style violation
[
https://issues.apache.org/jira/browse/IGNITE-11899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858627#comment-16858627
]
Andrey Gura commented on IGNITE-11899:
--
[~Mmuzaf] Just took a quick look at your change. The first, thanks a lot for
care about code style.
I want to comment some things that should be fixed also. Notice that my
comments addressed to the code author not to you.
1. Using one-line Javadoc for class fields.
{code:java}
/**
*
*/
private static final int PAGE_ID_OFFSET = 0;
{code}
Can be replaced by
{code:java}
/** */
private static final int PAGE_ID_OFFSET = 0;
{code}
Moreover, empty Javadoc comment it's very bad practice, so I believe that all
comments like this should be replaced by meaningful comment.
2. Magic numbers in some file should be replaced by constants with
self-descriptive names.
Just an example:
{code:java}
MemoryCalculator(){
onHeapAllocated(16 + (8 + 16) * 2);
}
{code}
I believe that this arithmetic expression can be rewritten in more clear manner.
> Fix code style violation
>
>
> Key: IGNITE-11899
> URL: https://issues.apache.org/jira/browse/IGNITE-11899
> Project: Ignite
> Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> All the code style issues must be fixed, see the sample below.
> {code}
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:50:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:183:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:187:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> [09:38:03][Step 2/2] [ERROR]
> /opt/buildagent/work/69588afcb2ab3382/modules/core/src/main/java/org/apache/ignite/internal/commandline/diagnostic/PageLocksCommand.java:191:
> 'VARIABLE_DEF' should be separated from previous statement.
> [EmptyLineSeparator]
> {code}
> TC log:
> https://ci.ignite.apache.org/viewLog.html?buildId=4062737=IgniteTests24Java8_CheckCodeStyle=buildLog_IgniteTests24Java8=%3Cdefault%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-10687) Document new JMX bean wchih expose IO statistics.
[ https://issues.apache.org/jira/browse/IGNITE-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura resolved IGNITE-10687. -- Resolution: Won't Fix > Document new JMX bean wchih expose IO statistics. > -- > > Key: IGNITE-10687 > URL: https://issues.apache.org/jira/browse/IGNITE-10687 > Project: Ignite > Issue Type: Task > Components: documentation, sql >Reporter: Yury Gerzhedovich >Priority: Major > Fix For: 2.8 > > > We need to document a new JMX bean which expose Input/Output statistics. > Start point is > [https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/mxbean/IoStatisticsMetricsMXBean.java] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (IGNITE-10687) Document new JMX bean wchih expose IO statistics.
[ https://issues.apache.org/jira/browse/IGNITE-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura reopened IGNITE-10687: -- > Document new JMX bean wchih expose IO statistics. > -- > > Key: IGNITE-10687 > URL: https://issues.apache.org/jira/browse/IGNITE-10687 > Project: Ignite > Issue Type: Task > Components: documentation, sql >Reporter: Yury Gerzhedovich >Priority: Major > Fix For: 2.8 > > > We need to document a new JMX bean which expose Input/Output statistics. > Start point is > [https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/mxbean/IoStatisticsMetricsMXBean.java] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11848) [IEP-35] Monitoring Phase 1
[ https://issues.apache.org/jira/browse/IGNITE-11848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854447#comment-16854447 ] Andrey Gura commented on IGNITE-11848: -- [~NIzhikov] I'll take a look. [~Jokser] could you please also join to the code review? > [IEP-35] Monitoring Phase 1 > -- > > Key: IGNITE-11848 > URL: https://issues.apache.org/jira/browse/IGNITE-11848 > Project: Ignite > Issue Type: Task >Affects Versions: 2.7 >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Umbrella ticket for the IEP-35. Monitoring and profiling. > Phase 1 should include: > * NextGen monitoring subsystem implementation to manage > ** metrics > ** -lists- (will be implemented in the following tickets) > ** exporters > * JMX, SQLView, Log exporters > * Migration of existing metrics to new manager > * -Lists for all Ignite user API- -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11804) Assertion error on affinity initialization
[
https://issues.apache.org/jira/browse/IGNITE-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826004#comment-16826004
]
Andrey Gura commented on IGNITE-11804:
--
[~ascherbakov] Just good practice. In many cases you can understand the problem
just looking to stack trace. Thanks a lot!
> Assertion error on affinity initialization
> --
>
> Key: IGNITE-11804
> URL: https://issues.apache.org/jira/browse/IGNITE-11804
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexei Scherbakov
>Priority: Major
>
> Reproducer (needs some cleanup)
> {noformat}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version 2.0
> * (the "License"); you may not use this file except in compliance with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.ignite.internal.processors.cache.transactions;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.concurrent.atomic.AtomicReference;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.DataRegionConfiguration;
> import org.apache.ignite.configuration.DataStorageConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.failure.StopNodeFailureHandler;
> import org.apache.ignite.internal.IgniteEx;
> import org.apache.ignite.internal.processors.cache.GridCacheContext;
> import org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl;
> import
> org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
> import org.apache.ignite.transactions.Transaction;
> import org.junit.Test;
> import static java.util.concurrent.TimeUnit.DAYS;
> import static java.util.concurrent.TimeUnit.MILLISECONDS;
> import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL;
> import static org.apache.ignite.cache.CacheWriteSynchronizationMode.FULL_SYNC;
> import static org.apache.ignite.configuration.WALMode.LOG_ONLY;
> import static
> org.apache.ignite.transactions.TransactionConcurrency.PESSIMISTIC;
> import static
> org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_READ;
> /**
> * Test framework for ordering transaction's prepares and commits by
> intercepting messages and releasing then
> * in user defined order.
> */
> public class TxPartitionCounterStateAbstractTest extends
> GridCommonAbstractTest {
> /** IP finder. */
> private static final TcpDiscoveryVmIpFinder IP_FINDER = new
> TcpDiscoveryVmIpFinder(true);
> /** */
> private static final int MB = 1024 * 1024;
> /** */
> protected int backups;
> /** */
> public static final int TEST_TIMEOUT = 30_000;
> public static final String DEFAULT_CACHE_NAME_2 = DEFAULT_CACHE_NAME +
> "2";
> /** */
> private AtomicReference testFailed = new AtomicReference<>();
> /** Number of keys to preload before txs to enable historical rebalance.
> */
> protected static final int PRELOAD_KEYS_CNT = 1;
> /** */
> protected static final String CLIENT_GRID_NAME = "client";
> /** */
> protected static final int PARTS_CNT = 32;
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> cfg.setConsistentId("node" + igniteInstanceName);
> cfg.setFailureHandler(new StopNodeFailureHandler());
> cfg.setRebalanceThreadPoolSize(4); // Necessary to reproduce some
> issues.
> ((TcpDiscoverySpi)cfg.getDiscoverySpi()).setIpFinder(IP_FINDER);
> // TODO set this only for historical rebalance tests.
> cfg.setCommunicationSpi(new
> IgniteWalRebalanceTest.WalRebalanceCheckingCommunicationSpi());
> boolean
[jira] [Commented] (IGNITE-11804) Assertion error
[
https://issues.apache.org/jira/browse/IGNITE-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16825354#comment-16825354
]
Andrey Gura commented on IGNITE-11804:
--
[~ascherbakov] Could you please add assertion stack trace to the issue
description?
> Assertion error
> ---
>
> Key: IGNITE-11804
> URL: https://issues.apache.org/jira/browse/IGNITE-11804
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexei Scherbakov
>Priority: Major
>
> Reproducer (needs some cleanup)
> {noformat}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version 2.0
> * (the "License"); you may not use this file except in compliance with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.ignite.internal.processors.cache.transactions;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.concurrent.atomic.AtomicReference;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.DataRegionConfiguration;
> import org.apache.ignite.configuration.DataStorageConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.failure.StopNodeFailureHandler;
> import org.apache.ignite.internal.IgniteEx;
> import org.apache.ignite.internal.processors.cache.GridCacheContext;
> import org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl;
> import
> org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
> import org.apache.ignite.transactions.Transaction;
> import org.junit.Test;
> import static java.util.concurrent.TimeUnit.DAYS;
> import static java.util.concurrent.TimeUnit.MILLISECONDS;
> import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL;
> import static org.apache.ignite.cache.CacheWriteSynchronizationMode.FULL_SYNC;
> import static org.apache.ignite.configuration.WALMode.LOG_ONLY;
> import static
> org.apache.ignite.transactions.TransactionConcurrency.PESSIMISTIC;
> import static
> org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_READ;
> /**
> * Test framework for ordering transaction's prepares and commits by
> intercepting messages and releasing then
> * in user defined order.
> */
> public class TxPartitionCounterStateAbstractTest extends
> GridCommonAbstractTest {
> /** IP finder. */
> private static final TcpDiscoveryVmIpFinder IP_FINDER = new
> TcpDiscoveryVmIpFinder(true);
> /** */
> private static final int MB = 1024 * 1024;
> /** */
> protected int backups;
> /** */
> public static final int TEST_TIMEOUT = 30_000;
> public static final String DEFAULT_CACHE_NAME_2 = DEFAULT_CACHE_NAME +
> "2";
> /** */
> private AtomicReference testFailed = new AtomicReference<>();
> /** Number of keys to preload before txs to enable historical rebalance.
> */
> protected static final int PRELOAD_KEYS_CNT = 1;
> /** */
> protected static final String CLIENT_GRID_NAME = "client";
> /** */
> protected static final int PARTS_CNT = 32;
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> cfg.setConsistentId("node" + igniteInstanceName);
> cfg.setFailureHandler(new StopNodeFailureHandler());
> cfg.setRebalanceThreadPoolSize(4); // Necessary to reproduce some
> issues.
> ((TcpDiscoverySpi)cfg.getDiscoverySpi()).setIpFinder(IP_FINDER);
> // TODO set this only for historical rebalance tests.
> cfg.setCommunicationSpi(new
> IgniteWalRebalanceTest.WalRebalanceCheckingCommunicationSpi());
> boolean client = igniteInstanceName.startsWith(CLIENT_GRID_NAME);
> cfg.setClientMode(client);
[jira] [Assigned] (IGNITE-11267) Print warning when keystore password arguments are used in control.sh (bat)
[ https://issues.apache.org/jira/browse/IGNITE-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura reassigned IGNITE-11267: Assignee: Andrey Gura (was: Andrey Kuznetsov) > Print warning when keystore password arguments are used in control.sh (bat) > --- > > Key: IGNITE-11267 > URL: https://issues.apache.org/jira/browse/IGNITE-11267 > Project: Ignite > Issue Type: Task >Affects Versions: 2.7 >Reporter: Andrey Kuznetsov >Assignee: Andrey Gura >Priority: Minor > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Control utility gets keystore/truststore password either as command line > argument or as console input. Former way is insecure, and user should be > warned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-11267) Print warning when keystore password arguments are used in control.sh (bat)
[ https://issues.apache.org/jira/browse/IGNITE-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura reassigned IGNITE-11267: Assignee: Andrey Kuznetsov (was: Andrey Gura) > Print warning when keystore password arguments are used in control.sh (bat) > --- > > Key: IGNITE-11267 > URL: https://issues.apache.org/jira/browse/IGNITE-11267 > Project: Ignite > Issue Type: Task >Affects Versions: 2.7 >Reporter: Andrey Kuznetsov >Assignee: Andrey Kuznetsov >Priority: Minor > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Control utility gets keystore/truststore password either as command line > argument or as console input. Former way is insecure, and user should be > warned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11774) NPE TxDeadlockDetection in case of concurrent cache stop and tx.
[
https://issues.apache.org/jira/browse/IGNITE-11774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821957#comment-16821957
]
Andrey Gura commented on IGNITE-11774:
--
[~zstan] Does it prevent successful node stopping?
> NPE TxDeadlockDetection in case of concurrent cache stop and tx.
>
>
> Key: IGNITE-11774
> URL: https://issues.apache.org/jira/browse/IGNITE-11774
> Project: Ignite
> Issue Type: Bug
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>
> In process of working under :
> [IGNITE-11592|https://issues.apache.org/jira/browse/IGNITE-11592] reproduced
> NPE, need further research, reproducer commented in upper ticket code.
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.primary(TxDeadlockDetection.java:400)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.mapTxKeys(TxDeadlockDetection.java:376)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.map(TxDeadlockDetection.java:275)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.init(TxDeadlockDetection.java:267)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection$TxDeadlockFuture.access$100(TxDeadlockDetection.java:158)
> at
> org.apache.ignite.internal.processors.cache.transactions.TxDeadlockDetection.detectDeadlock(TxDeadlockDetection.java:91)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.detectDeadlock(IgniteTxManager.java:2155)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearOptimisticTxPrepareFuture.onTimeout(GridNearOptimisticTxPrepareFuture.java:776)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read
[
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818481#comment-16818481
]
Andrey Gura commented on IGNITE-11687:
--
[~agoncharuk] I've investigated the problem deeper. While code snippet pointed
by you is incorrect and must be fixed it never executes by test because MMAP
mode is switched on by default. I think that
{{FileWriteHandleImpl#addRecord()}} method is root of the problem. See the
following code snippet:
{code:java}
fillBuffer(buf, rec);
if (mmap) {
// written field must grow only, but segment with
greater position can be serialized
// earlier than segment with smaller position.
while (true) {
long written0 = written;
if (seg.position() > written0) {
if (WRITTEN_UPD.compareAndSet(this, written0,
seg.position()))
break;
}
else
break;
}
}
return ptr;
{code}
WAL iterator on {{wal.replay()}} call gets {{hnd.written}} field value while
some previous WAL record before this position is still not fully serialized.
What do you think?
> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been
> already updated, concurrent WAL replay will attempt to read the incompletely
> written record and since {{end}} is not null, iterator will fail with CRC
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read
[
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811031#comment-16811031
]
Andrey Gura commented on IGNITE-11687:
--
[~agoncharuk] You are right. It could be fixed easy.
{{FileHandleManagerImpl.WALWriter#writeBuffer}} method has only one usage so it
can just return delta that must be added to the {{hnd.written}.}
> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been
> already updated, concurrent WAL replay will attempt to read the incompletely
> written record and since {{end}} is not null, iterator will fail with CRC
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read
[
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811031#comment-16811031
]
Andrey Gura edited comment on IGNITE-11687 at 4/5/19 4:38 PM:
--
[~agoncharuk] You are right. It could be fixed easy.
{{FileHandleManagerImpl.WALWriter#writeBuffer}} method has only one usage so it
can just return delta that must be added to the {{hnd.written}}.
was (Author: agura):
[~agoncharuk] You are right. It could be fixed easy.
{{FileHandleManagerImpl.WALWriter#writeBuffer}} method has only one usage so it
can just return delta that must be added to the {{hnd.written}.}
> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been
> already updated, concurrent WAL replay will attempt to read the incompletely
> written record and since {{end}} is not null, iterator will fail with CRC
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-11687) Concurrent WAL replay & log may fail with CRC error on read
[
https://issues.apache.org/jira/browse/IGNITE-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura reassigned IGNITE-11687:
Assignee: Andrey Gura
> Concurrent WAL replay & log may fail with CRC error on read
> ---
>
> Key: IGNITE-11687
> URL: https://issues.apache.org/jira/browse/IGNITE-11687
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexey Goncharuk
>Assignee: Andrey Gura
>Priority: Critical
> Fix For: 2.8
>
>
> The cause is the way {{end}} is calculated for WAL iterator:
> {code}
> if (hnd != null)
> end = hnd.position();
> {code}
> {code}
> @Override public FileWALPointer position() {
> lock.lock();
> try {
> return new FileWALPointer(getSegmentId(), (int)written, 0);
> }
> finally {
> lock.unlock();
> }
> }
> {code}
> Consider a partially written entry. In this case, {{written}} has been
> already updated, concurrent WAL replay will attempt to read the incompletely
> written record and since {{end}} is not null, iterator will fail with CRC
> error.
> The issue may be rarely reproduced by {{IgniteWalSerializerVersionTest}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7883) Cluster can have inconsistent affinity configuration
[
https://issues.apache.org/jira/browse/IGNITE-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802903#comment-16802903
]
Andrey Gura commented on IGNITE-7883:
-
[~a-polyakov] I've took a look ti the change and have some comments:
1. {{GridCacheUtils#validateAffinityKeyConfiguration(CacheKeyConfiguration[])}}
I think better name is {{validateKeyConfigiration}}.
Please fix message of {{IgniteCheckedException}}. At the moment it is too
verbose and unclear. I think it should be something like "Cache key
configuration contains conflicting definitions: [cacheGroup=<>, cache=<>,
typeName=<>, affKeyFieldName1=<>, affKeyFieldName2=<>]".
Javadoc should have more accurate formulation. E..g it's unclear words "items"
and "full". Proper comment will be "all fields are initialized and not empty".
Please rewrite javadoc.
2. {{GridCacheUtils#validateAffinityKeyConfiguration(String, UUID,
CacheKeyConfiguration[], CacheKeyConfiguration[])}}
I think better name is {{validateKeyConfigiration}}.
{{oldCacheKeyCfgs}} and {{newCacheKeyCfgs}} parameters should be renamed to
{{rmtCacheKeyCfgs}} and {{locCacheKeyCfgs}} respectively.
Please fix message of {{IgniteCheckedException}}. At the moment it is too
verbose and unclear. For example see message that generates
{{GridCacheUtils#checkAttributeMismatch}} method. Also note that it could be
exception or just log message at warning level (it depends on
{{IGNITE_SKIP_CONFIGURATION_CONSISTENCY_CHECK}} system property). Also see p. 1
above.
Format method's code properly.
Fix javadoc in order to properly describe method parameters after renaming.
3. {{CacheKeyConfiguration#equals}}
Use {{Objects.equals}} instead of too long expressions at the method end.
4. {{CacheAffinityKeyConfigurationMismatchTest}}
Please, rewrite javadoc o the class.
Reformat code. Comma should always be on the same line where previous method
parameter placed.
Rewrite test code. Framework has set of methods for getting ignite and cache
configuration. So methods like {getIgnite}} are redundant. See
{{GridAbstractTest#getConfiguration()}} and other ignite tests for example.
Add additional tests for teh following cases:
- testKeyConfigurationLengthMismatch - you test only case when we have key
configuration on one node and don't have on another. What about different key
configurations on nodes?
- testKeyConfigurationDuplicateTypeName - you test only case for one node. What
about conflicting defintions from different nodes?
It makes sense to add test cases for configurations that were constructed due
to using {{@AffinityKeyMapped}} annotation.
> Cluster can have inconsistent affinity configuration
> -
>
> Key: IGNITE-7883
> URL: https://issues.apache.org/jira/browse/IGNITE-7883
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.3
>Reporter: Mikhail Cherkasov
>Assignee: Alexand Polyakov
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> A cluster can have inconsistent affinity configuration if you created two
> nodes, one with affinity key configuration and other without it(in IgniteCfg
> or CacheCfg), both nodes will work fine with no exceptions, but in the same
> time they will apply different affinity rules to keys:
>
> {code:java}
> package affinity;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheAtomicityMode;
> import org.apache.ignite.cache.CacheKeyConfiguration;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.affinity.Affinity;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import java.util.Arrays;
> public class Test {
> private static int id = 0;
> public static void main(String[] args) {
> Ignite ignite = Ignition.start(getConfiguration(true, false));
> Ignite ignite2 = Ignition.start(getConfiguration(false, false));
> Affinity affinity = ignite.affinity("TEST");
> Affinity affinity2 = ignite2.affinity("TEST");
> for (int i = 0; i < 1_000_000; i++) {
> AKey key = new AKey(i);
> if(affinity.partition(key) != affinity2.partition(key))
> System.out.println("FAILED for: " + key);
> }
> System.out.println("DONE");
> }
> private static IgniteConfiguration getConfiguration(boolean
> withAffinityCfg, boolean client) {
> IgniteConfiguration cfg = new IgniteConfiguration();
> TcpDiscoveryVmIpFinder finder = new
[jira] [Commented] (IGNITE-9812) Explicit tests with expired SSL certificates
[ https://issues.apache.org/jira/browse/IGNITE-9812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802772#comment-16802772 ] Andrey Gura commented on IGNITE-9812: - [~ilyak] LGTM! Please proceed with merge. > Explicit tests with expired SSL certificates > > > Key: IGNITE-9812 > URL: https://issues.apache.org/jira/browse/IGNITE-9812 > Project: Ignite > Issue Type: Test > Components: clients, security >Reporter: Ilya Kasnacheev >Assignee: Ilya Kasnacheev >Priority: Minor > Labels: test > Time Spent: 10m > Remaining Estimate: 0h > > It came to our attention that we have no test which checks that expired SSL > certificates do not work indeed. > Should get such old certificate from git, check if it will fail to join > cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-11127) GridDhtInvalidPartitionException not handled by GridCacheTtlManager
[
https://issues.apache.org/jira/browse/IGNITE-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802765#comment-16802765
]
Andrey Gura commented on IGNITE-11127:
--
[~roman_s] Looks good to me too.
> GridDhtInvalidPartitionException not handled by GridCacheTtlManager
> ---
>
> Key: IGNITE-11127
> URL: https://issues.apache.org/jira/browse/IGNITE-11127
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.4, 2.7
>Reporter: Ilya Kasnacheev
>Assignee: Roman Shtykh
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Leading to either message processing problems:
> {code}
> [2019-01-27 16:57:45,474][ERROR][sys-stripe-2-#3][GridCacheIoManager] Failed
> to process message [senderId=4839b5a2-a295-44cf-8a44-f0cb932b689e,
> messageType=class
> o.a.i.i.processors.cache.distributed.dht.atomic.GridNearAtomicFullUpdateRequest]
> class
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException
> [part=381, msg=Adding entry to partition that is concurrently evicted
> [grp=, part=381, shouldBeMoving=, belongs=false,
> topVer=AffinityTopologyVersion [topVer=818, minorTopVer=0],
> curTopVer=AffinityTopologyVersion [topVer=818, minorTopVer=0]]]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition0(GridDhtPartitionTopologyImpl.java:917)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartition(GridDhtPartitionTopologyImpl.java:794)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.localPartition(GridCachePartitionedConcurrentMap.java:69)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridCachePartitionedConcurrentMap.putEntryIfObsoleteOrAbsent(GridCachePartitionedConcurrentMap.java:88)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.entryEx(GridCacheAdapter.java:952)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.entryEx(GridDhtCacheAdapter.java:525)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.entryEx(GridCacheAdapter.java:943)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:1047)
> at
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:197)
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:835)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessageProcessed(GridCacheIoManager.java:1093)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1066)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
> at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> or unhandled unspecified exceptions in user code (possibly violating JCache):
> {code}
> [2019-01-27 10:23:35,451][ERROR][pub-#840058][ComputeJobProcess] class
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtInvalidPartitionException
> [part=485, msg=Adding entry to partition that is concurrently evicted
> [grp=, part=485, shouldBeMoving=, belongs=false,
> topVer=AffinityTopologyVersion [topVer=815, minorTopVer=0],
> curTopVer=AffinityTopologyVersion [topVer=815, minorTopVer=0]]]
> at
>
[jira] [Commented] (IGNITE-10925) Failure to submit affinity task from client node
[
https://issues.apache.org/jira/browse/IGNITE-10925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779470#comment-16779470
]
Andrey Gura commented on IGNITE-10925:
--
[~ilyak] LGTM. Please merge. Thanks!
> Failure to submit affinity task from client node
>
>
> Key: IGNITE-10925
> URL: https://issues.apache.org/jira/browse/IGNITE-10925
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 2.7
>Reporter: Prasad
>Assignee: Ilya Kasnacheev
>Priority: Blocker
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Getting following exception while submitting the affinity task from client
> node to server node.
> Before submitting the affinity task ignite first gets the affinity cached
> function (AffinityInfo) by submitting the cluster wide task "AffinityJob".
> But while in the process of retrieving the output of this AffinityJob, ignite
> deserializes this output. I am getting exception while deserailizing this
> output.
> Code fails while un-marshalling cachesnapshotmetrics on client node.
>
> [Userlist
> Discussion|http://apache-ignite-users.70518.x6.nabble.com/After-upgrading-2-7-getting-Unexpected-error-occurred-during-unmarshalling-td26262.html]
> [Reproducer
> Project|https://github.com/prasadbhalerao1983/IgniteIssueReproducer.git]
>
> Step to Reproduce:
> 1) First Run com.example.demo.Server class as a java program
> 2) Then run com.example.demo.Client as java program.
>
> {noformat}
> 2019-01-14 15:37:02.723 ERROR 10712 --- [springDataNode%]
> o.a.i.i.processors.task.GridTaskWorker : Error deserializing job response:
> GridJobExecuteResponse [nodeId=e9a24c20-0d00-4808-b2f5-13e1ce35496a,
> sesId=76324db4861-1d85ad49-5b25-454a-b69c-d8685cfc73b0,
> jobId=86324db4861-1d85ad49-5b25-454a-b69c-d8685cfc73b0, gridEx=null,
> isCancelled=false, retry=null]
> org.apache.ignite.IgniteCheckedException: Failed to unmarshal object with
> optimized marshaller
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10146)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:831)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
> [ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
> [ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
> [ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
> [ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> [ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
> [ignite-core-2.7.0.jar:2.7.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_144]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_144]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to
> unmarshal object with optimized marshaller
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1765)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:313)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:102)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> ~[ignite-core-2.7.0.jar:2.7.0]
> at
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10140)
> ~[ignite-core-2.7.0.jar:2.7.0]
> ... 10 common frames omitted
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to deserialize
> object with given class loader:
> [clsLdr=sun.misc.Launcher$AppClassLoader@18b4aac2, err=Failed to deserialize
> object [typeName=org.apache.ignite.internal.util.lang.GridTuple3]]
> at
>
[jira] [Commented] (IGNITE-11253) When a node that is not part of the baseline topology joins the cluster, it may lead to a node failure.
[
https://issues.apache.org/jira/browse/IGNITE-11253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764906#comment-16764906
]
Andrey Gura commented on IGNITE-11253:
--
[~slava.koptilin] Could you please add at least {{InterruptedException}} to
{{X.hasCause}} method's parameters. Usually any worker stops via {{cancel}}
call that interrupt worker's runner.
> When a node that is not part of the baseline topology joins the cluster, it
> may lead to a node failure.
> ---
>
> Key: IGNITE-11253
> URL: https://issues.apache.org/jira/browse/IGNITE-11253
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> * In case of eager TTL is configured, a starting node creates and starts
> {{cleanupWorker}} (see {{GridCacheTtlManager.start0()}})
> * {{GridCacheSharedTtlCleanupManager.CleanupWorker}}, in its turn, has to
> wait for {{discovery().localJoin()}} future that is completed by discovery
> thread.
> * On the other hand, the exchange thread stops cache contexts and,
> therefore, it stops the {{cleanupWorker}} as well.
>
> {code:java}
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager.stopCleanupWorker(GridCacheSharedTtlCleanupManager.java:109)
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager.unregister(GridCacheSharedTtlCleanupManager.java:82)
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.onKernalStop0(GridCacheTtlManager.java:110)
> org.apache.ignite.internal.processors.cache.GridCacheManagerAdapter.onKernalStop(GridCacheManagerAdapter.java:111)
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:1495)
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStopCaches(GridCacheProcessor.java:1182)
> org.apache.ignite.internal.processors.cache.GridCacheProcessor$CacheRecoveryLifecycle.onBaselineChange(GridCacheProcessor.java:5637)
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:910)
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:792)
> {code}
> So, exchange thread may try to stop the {{cleanupWorker}} before the
> {{localJoin}} future is completed by discovery thread. Unfortunately,
> `cleanupWorker` incorrectly handles this situation, and this fact can lead to
> a node failure:
> {code:java}
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Got
> interrupted while waiting for future to complete.]]
> class org.apache.ignite.IgniteException: Got interrupted while waiting for
> future to complete.
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.localJoin(GridDiscoveryManager.java:2217)
> at
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:136)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.internal.IgniteInterruptedCheckedException: Got interrupted
> while waiting for future to complete.
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:186)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.localJoin(GridDiscoveryManager.java:2214)
> ... 3 more
> {code}
> The obvious fix is changing the catch block
> {code:java}
> catch (Throwable t) {
> if (!(t instanceof IgniteInterruptedCheckedException))
> err = t;
> throw t;
> }
> {code}
> to the following:
> {code:java}
> catch (Throwable t) {
> if (!(X.hasCause(t, IgniteInterruptedCheckedException.class)))
> err = t;
> throw t;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11130) BackupPostProcessingClosure should not save initial value on a backup node when cache store is not configured.
[
https://issues.apache.org/jira/browse/IGNITE-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758266#comment-16758266
]
Andrey Gura commented on IGNITE-11130:
--
[~slava.koptilin] Thanks. Merged to master branch.
> BackupPostProcessingClosure should not save initial value on a backup node
> when cache store is not configured.
> --
>
> Key: IGNITE-11130
> URL: https://issues.apache.org/jira/browse/IGNITE-11130
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The change introduced by IGNITE-7086 lead to the fact that
> {{BackupPostProcessingClosure}} could attempt to store the initial value even
> if the corresponding cache store is not configured.
> For instance, transactional read operation where atomicity mode is
> {{OPTIMISTIC}} and isolation level is {{SERIALIZABLE}} (native persistent
> enabled) tries to update the local backup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11130) BackupPostProcessingClosure should not save initial value on a backup node when cache store is not configured.
[
https://issues.apache.org/jira/browse/IGNITE-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757850#comment-16757850
]
Andrey Gura commented on IGNITE-11130:
--
[~slava.koptilin] LGTM. But is seems that some test are failed. Could you
please check.
> BackupPostProcessingClosure should not save initial value on a backup node
> when cache store is not configured.
> --
>
> Key: IGNITE-11130
> URL: https://issues.apache.org/jira/browse/IGNITE-11130
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The change introduced by IGNITE-7086 lead to the fact that
> {{BackupPostProcessingClosure}} could attempt to store the initial value even
> if the corresponding cache store is not configured.
> For instance, transactional read operation where atomicity mode is
> {{OPTIMISTIC}} and isolation level is {{SERIALIZABLE}} (native persistent
> enabled) tries to update the local backup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Closed] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
[
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura closed IGNITE-11129.
> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
>
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Assignee: Stepachev Maksim
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
> switched on. Size for this record type must be constant.
> See
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9903) Copy only actual amount of data during archiving of WAL segment
[
https://issues.apache.org/jira/browse/IGNITE-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757843#comment-16757843
]
Andrey Gura commented on IGNITE-9903:
-
[~astelmak] LGTM. Merged to master branch. Thanks for contribution!
> Copy only actual amount of data during archiving of WAL segment
> ---
>
> Key: IGNITE-9903
> URL: https://issues.apache.org/jira/browse/IGNITE-9903
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Alexey Stelmak
>Priority: Major
> Fix For: 2.8
>
>
> Current WAL archiver implementation copies full WAL segment to the archive
> while actual size of the segment can be significantly less then
> {{maxWalSegmentSize}} (segment files are preallocated for max possible
> segment size).
> In order to reduce disk space consuming we need copy only actual data of
> segment using {{SWITCH_SEGMENT_RECORD}} as marker. It will require some kind
> of WAL iterator implementation that will just copy WAL records using
> information about record size without records deserialization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
[
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757846#comment-16757846
]
Andrey Gura commented on IGNITE-11129:
--
[~mstepachev] Agree. Thanks.
> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
>
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Assignee: Stepachev Maksim
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
> switched on. Size for this record type must be constant.
> See
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9903) Copy only actual amount of data during archiving of WAL segment
[
https://issues.apache.org/jira/browse/IGNITE-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755142#comment-16755142
]
Andrey Gura edited comment on IGNITE-9903 at 1/29/19 6:40 PM:
--
[~astelmak] I've looked at your change and have some comments:
- Size of {{SWITCH_SEGMENT_RECORD}} should be calculated using
{{RecordSerializer.size}} (see example here:
{{org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl#close}}).
- Unused import in all changed files.
- {{switchSegmentRecordOffset}} array never updates (except of {{0}} value).
Please add test for this situation.
Could you please fix this comments? Thanks.
was (Author: agura):
[~astelmak] I've looked at your change and have some comments:
- Size of {{SWITCH_SEGMENT_RECORD}} should be calculated using
{{RecordSerializer.size}} (see example here:
{{org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl#close}}).
- Unused import in all changed files.
- {{switchSegmentRecordOffset}} array never updates (except of {{0}} value).
Could you please fix this comments? Thanks.
> Copy only actual amount of data during archiving of WAL segment
> ---
>
> Key: IGNITE-9903
> URL: https://issues.apache.org/jira/browse/IGNITE-9903
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Alexey Stelmak
>Priority: Major
> Fix For: 2.8
>
>
> Current WAL archiver implementation copies full WAL segment to the archive
> while actual size of the segment can be significantly less then
> {{maxWalSegmentSize}} (segment files are preallocated for max possible
> segment size).
> In order to reduce disk space consuming we need copy only actual data of
> segment using {{SWITCH_SEGMENT_RECORD}} as marker. It will require some kind
> of WAL iterator implementation that will just copy WAL records using
> information about record size without records deserialization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9903) Copy only actual amount of data during archiving of WAL segment
[
https://issues.apache.org/jira/browse/IGNITE-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755142#comment-16755142
]
Andrey Gura commented on IGNITE-9903:
-
[~astelmak] I've looked at your change and have some comments:
- Size of {{SWITCH_SEGMENT_RECORD}} should be calculated using
{{RecordSerializer.size}} (see example here:
{{org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl#close}}).
- Unused import in all changed files.
- {{switchSegmentRecordOffset}} array never updates (except of {{0}} value).
Could you please fix this comments? Thanks.
> Copy only actual amount of data during archiving of WAL segment
> ---
>
> Key: IGNITE-9903
> URL: https://issues.apache.org/jira/browse/IGNITE-9903
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Gura
>Assignee: Alexey Stelmak
>Priority: Major
> Fix For: 2.8
>
>
> Current WAL archiver implementation copies full WAL segment to the archive
> while actual size of the segment can be significantly less then
> {{maxWalSegmentSize}} (segment files are preallocated for max possible
> segment size).
> In order to reduce disk space consuming we need copy only actual data of
> segment using {{SWITCH_SEGMENT_RECORD}} as marker. It will require some kind
> of WAL iterator implementation that will just copy WAL records using
> information about record size without records deserialization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
[
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-11129:
-
Description:
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
switched on. Size for this record type must be constant.
See
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);
if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
size */ + REC_TYPE_SIZE;
return clSz;
}
{code}
was:
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
switched on. Size for this record type should be constant.
See
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);
if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
size */ + REC_TYPE_SIZE;
return clSz;
}
{code}
> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
>
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
> switched on. Size for this record type must be constant.
> See
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
Andrey Gura created IGNITE-11129:
Summary: Incorrect size calculation for SWITCH_SEGMENT_RECORD for
TDE
Key: IGNITE-11129
URL: https://issues.apache.org/jira/browse/IGNITE-11129
Project: Ignite
Issue Type: Bug
Reporter: Andrey Gura
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
switched on. Size for this record type should be constant.
See
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);
if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
size */ + REC_TYPE_SIZE;
return clSz;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-11129) Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
[
https://issues.apache.org/jira/browse/IGNITE-11129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-11129:
-
Description:
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
switched on. Size for this record type should be constant.
See
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);
if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
size */ + REC_TYPE_SIZE;
return clSz;
}
{code}
was:
Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
switched on. Size for this record type should be constant.
See
{{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
{code:java}
@Override public int size(WALRecord record) throws IgniteCheckedException {
int clSz = plainSize(record);
if (needEncryption(record))
return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
size */ + REC_TYPE_SIZE;
return clSz;
}
{code}
> Incorrect size calculation for SWITCH_SEGMENT_RECORD for TDE
>
>
> Key: IGNITE-11129
> URL: https://issues.apache.org/jira/browse/IGNITE-11129
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Gura
>Priority: Major
>
> Size of {{SWITCH_SEGMENT_RECORD}} will be invalid in case of encryption
> switched on. Size for this record type should be constant.
> See
> {{org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer#size}}:
> {code:java}
> @Override public int size(WALRecord record) throws IgniteCheckedException
> {
> int clSz = plainSize(record);
> if (needEncryption(record))
> return encSpi.encryptedSize(clSz) + 4 /* groupId */ + 4 /* data
> size */ + REC_TYPE_SIZE;
> return clSz;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-10938) After restart cluster with non-blt nodes - they left by handler
[ https://issues.apache.org/jira/browse/IGNITE-10938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Gura resolved IGNITE-10938. -- Resolution: Duplicate > After restart cluster with non-blt nodes - they left by handler > --- > > Key: IGNITE-10938 > URL: https://issues.apache.org/jira/browse/IGNITE-10938 > Project: Ignite > Issue Type: Bug > Components: data structures >Affects Versions: 2.8 >Reporter: ARomantsov >Priority: Critical > Fix For: 2.8 > > > I have cluster wherein topology contain blt and non-blt nodes, but after > restart - nodes left by handler > java.lang.IllegalStateException: Unable to find consistentId by UUID -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9739) Critical exception in transaction processing in case we have nodes out of baseline and non-persisted cache
[
https://issues.apache.org/jira/browse/IGNITE-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Gura updated IGNITE-9739:
Fix Version/s: 2.8
> Critical exception in transaction processing in case we have nodes out of
> baseline and non-persisted cache
> --
>
> Key: IGNITE-9739
> URL: https://issues.apache.org/jira/browse/IGNITE-9739
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Reporter: Sergey Kosarev
>Assignee: Sergey Kosarev
>Priority: Major
> Fix For: 2.8
>
>
> Activation finished
> {code:java}
> 2018-09-20 20:47:05.169 [INFO
> ][sys-#307%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
> Successfully performed final activation steps
> [nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557, client=false,
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=1]]
> {code}
> but we have nodes not in base line
> {code:java}
> 2018-09-20 20:45:36.116 [INFO
> ][sys-#305%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
> Local node is not included in Baseline Topology and will not be used for
> persistent data storage. Use control.(sh|bat) script or IgniteCluster
> interface to include the node to Baseline Topology.
> {code}
> And we have cache (869481129) in the data region with persistanceEnabled=false
> {code:java}
> 2018-09-20 20:49:01.825 [INFO
> ][exchange-worker-#154%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheProcessor]
> Started cache [name=DPL_PUBLISHED_CACHES_REGISTRY$, *id=869481129*, group=SY
> STEM_CACHEGROUP_PUBLISHED_REGISTRY, memoryPolicyName=not-persisted,
> mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=3]
> {code}
> Transaction on this cache(869481129)
> {code:java}
> 869481129{code}
> leads to critical error causing nodes by faulure handler:
> {code:java}
> 2018-09-20 20:50:24.275
> [ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed processing message [senderId=62e986f0-62b5-4ec8-8cc7-27b74d345235,
> msg=GridDhtTxPrepareRequest [nearNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774,
> futId=520e308f561-255fdea5-a996-4102-a120-afa380c54570, miniId=1,
> topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2],
> invalidateNearEntries={}, nearWrites=null, owned=null,
> nearXidVer=GridCacheVersion [topVer=148944365, order=1537511036821,
> nodeOrder=132], subjId=814af7c4-2de5-4511-b1ea-065b91eaa774, taskNameHash=0,
> preloadKeys=null, skipCompletedVers=false,
> super=GridDistributedTxPrepareRequest [threadId=58, concurrency=PESSIMISTIC,
> isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=148944365,
> order=1537511036824, nodeOrder=7], timeout=299970, reads=null,
> writes=ArrayList [
> IgniteTxEntry [key=KeyCacheObjectImpl [part=27254,
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
> *cacheId=869481129*,
> txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=27254,
> val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
> *cacheId=869481129*], val=[op=CREATE,
> val=com.sbt.dpl.gridgain.PublishedRegistry$PublishedCacheTuple
> [idHash=811765531, hash=1522508040,
> cacheName=com.sbt.gbk.entities.DocType_DPL_union-module,indexes=ArrayList
> {com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=1583970836, hash=363194492, isSoftReference=false,
> unselectiveBuckets=4096, fieldNames=ArrayList
> \{isDeleted},moduleName=union-module
> , cachedUnselectives=1, selectors=ArrayList {isDeleted},
> exceptUnselectives=false, primitiveCollection=false, isVersioned=false,
> isComposite=false, isSystemTypeBelongs=false,
> name=com.sbt.gbk.entities.DocType_DPL_isDeleted, isIndexedCollection=false,
> isGlobal=false, maxSelective=1000],
> com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=2060926101, hash=1983794578, isSoftReference=false,
> unselectiveBuckets=4096, fieldNames=ArrayList ,moduleName=union-module,
> cachedUnselectives=1, selectors=ArrayList, exceptUnselectives=false,
> primitiveCollection=false, isVersioned=false, isComposite=false,
> isSystemTypeBelongs=false, name=com.sbt.gbk.entities.DocType_DPL_code,
> isIndexedCollection=false, isGlobal=true, maxSelective=1000]
> , com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
> [idHash=1821682714, hash=-1245813786, isSoftReference=false,
> unselectiveBuckets=4096, fieldNames=ArrayList {globalId},
> moduleName=union-module, cachedUnselectives=1, selectors=ArrayList
> {globalId}, exceptUnselectives=false, primitiveCollection=false,
> isVersioned=false, isComposite=false, isSystemTypeBelongs=false,
> name=com.sbt.gbk.entities.DocType_DPL_globalId, isIndexedCollection=false,
>
