[MTCGA]: new failures in builds [5276709] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *Test with high flaky rate in master-nightly IgfsLocalSecondaryFileSystemDualAsyncSelfTest.testAccessAndModificationTimeUpwardsPropagation https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4311898764848411041=%3Cdefault%3E=testDetails Changes may lead to failure were done by - samaitra https://ci.ignite.apache.org/viewModification.html?modId=901424 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 05:46:34 05-05-2020
Re: [REVIEW] IGNITE-12980 Use Apache Ignite logo with registered trade mark symbol in README.md
Denis, Thank your for the review. Merged to master. Best regards, Ivan Pavlukhin пн, 4 мая 2020 г. в 20:11, Denis Magda : > > Ivan, thanks for the catch. Please go ahead and merge the changes. > > - > Denis > > > On Mon, May 4, 2020 at 12:30 AM Ivan Pavlukhin wrote: > > > Folks, > > > > As you might know, not long ago Apache Ignite became a registered > > trademark. I noticed that we still have (TM) symbol in > > https://github.com/apache/ignite/blob/master/README.md > > > > I prepared a ticket [1] and PR. I feel that the used approach might be > > improved, but I tried to do my best in a lack of time. Please review. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12980 > > > > Best regards, > > Ivan Pavlukhin > >
Re: [REVIEW] IGNITE-12980 Use Apache Ignite logo with registered trade mark symbol in README.md
Ivan, thanks for the catch. Please go ahead and merge the changes. - Denis On Mon, May 4, 2020 at 12:30 AM Ivan Pavlukhin wrote: > Folks, > > As you might know, not long ago Apache Ignite became a registered > trademark. I noticed that we still have (TM) symbol in > https://github.com/apache/ignite/blob/master/README.md > > I prepared a ticket [1] and PR. I feel that the used approach might be > improved, but I tried to do my best in a lack of time. Please review. > > [1] https://issues.apache.org/jira/browse/IGNITE-12980 > > Best regards, > Ivan Pavlukhin >
ExecutorServices hide assertions without logging and node stop
Igniters,
I've spent a couple of days in debug mode and found that some of the
configured ExecutorServices of an Ignite instance completely hide
assertion errors without any logging and node killing. This may
violate some internal guarantees and hide serious errors.
Let's consider, for instance, GridDhtPartitionsExchangeFuture [1]. It
has three places of submitting some Runnable on system executor
service. If an assertion error (or even any uncached exception) occurs
in the code block below it will be swallowed without logging, exchange
future completion or node stoping.
cctx.kernalContext().getSystemExecutorService().submit(new Runnable() {
@Override public void run() {
sendPartitions(newCrd);
}
});
I've checked that these executor services and most of them configured
to catch only OutOfMemoryError.
Should we consider catching AssertionErrors as well and treat them as
CRITICAL_ERRORS for the Failure Handler?
Should we log uncached errors on each of them?
The most important list of executor services configured with OOM handler only:
execSvc
svcExecSvc
sysExecSvc
p2pExecSvc
restExecSvc
utilityCacheExecSvc
affExecSvc
qryExecSvc
[1]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java#L4848
[jira] [Created] (IGNITE-12982) NullPointerException on TcpCommunicationMetricsListener for some of the cases
Maxim Muzafarov created IGNITE-12982:
Summary: NullPointerException on TcpCommunicationMetricsListener
for some of the cases
Key: IGNITE-12982
URL: https://issues.apache.org/jira/browse/IGNITE-12982
Project: Ignite
Issue Type: Bug
Reporter: Maxim Muzafarov
The code block below throws an {{NullPointerException}} for some of the cases.
Investigation required.
{code}
@Override public void onMessageSent(GridNioSession ses, Message
msg) {
Object consistentId = ses.meta(CONSISTENT_ID_META);
if (consistentId != null)
metricsLsnr.onMessageSent(msg, consistentId);
}
{code}
{code}
[2020-05-04
18:12:12,991][ERROR][grid-nio-worker-tcp-comm-0-#543%snapshot.IgniteClusterSnapshotSelfTest2%][TestRecordingCommunicationSpi]
Failed to process selector key [ses=GridSelectorNioSessionImpl
[worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=0,
bytesRcvd=42, bytesSent=18, bytesRcvd0=42, bytesSent0=18, select=true,
super=GridWorker [name=grid-nio-worker-tcp-comm-0,
igniteInstanceName=snapshot.IgniteClusterSnapshotSelfTest2, finished=false,
heartbeatTs=1588605131981, hashCode=1038334332, interrupted=false,
runner=grid-nio-worker-tcp-comm-0-#543%snapshot.IgniteClusterSnapshotSelfTest2%]]],
writeBuf=java.nio.DirectByteBuffer[pos=10 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=0,
sentCnt=0, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode
[id=7f78d082-6ce9-42b1-ab08-da1fde40,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1588605131971, loc=false,
ver=2.9.0#20200428-sha1:e551fa71, isClient=false], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=0,
sentCnt=0, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode
[id=7f78d082-6ce9-42b1-ab08-da1fde40,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1588605131971, loc=false,
ver=2.9.0#20200428-sha1:e551fa71, isClient=false], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
closeSocket=true,
outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
super=GridNioSessionImpl [locAddr=/127.0.0.1:47102, rmtAddr=/127.0.0.1:50655,
createTime=1588605131981, closeTime=0, bytesSent=18, bytesRcvd=42,
bytesSent0=18, bytesRcvd0=42, sndSchedTime=1588605131981,
lastSndTime=1588605131981, lastRcvTime=1588605131981, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@fc19b0b, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true, markedForClose=true]]]
java.lang.NullPointerException
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$1.onMessageSent(TcpCommunicationSpi.java:803)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$1.onMessageSent(TcpCommunicationSpi.java:472)
at
org.apache.ignite.internal.util.nio.GridNioServer.onMessageWritten(GridNioServer.java:1764)
at
org.apache.ignite.internal.util.nio.GridNioServer.access$1800(GridNioServer.java:99)
at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processWrite0(GridNioServer.java:1665)
at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processWrite(GridNioServer.java:1365)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2437)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2201)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1842)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
[2020-05-04
18:12:12,993][ERROR][grid-nio-worker-tcp-comm-0-#543%snapshot.IgniteClusterSnapshotSelfTest2%][TestRecordingCommunicationSpi]
Closing NIO session because of unhandled exception.
class org.apache.ignite.internal.util.nio.GridNioException: null
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2460)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2201)
at
[jira] [Created] (IGNITE-12981) Snapshot operation fails on exchange if coordinator left the cluster
Maxim Muzafarov created IGNITE-12981:
Summary: Snapshot operation fails on exchange if coordinator left
the cluster
Key: IGNITE-12981
URL: https://issues.apache.org/jira/browse/IGNITE-12981
Project: Ignite
Issue Type: Bug
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
Fix For: 2.9
Possible steps:
1. Start snapshot operation with {{SnapshotDicoveryMessage}} with triggering
pme-free.
2. Stop the coordinator node.
3. Fails on assertion (SingleMessage should not be sent).
{code}
[2020-05-03
15:05:35,548][ERROR][sys-#94%snapshot.IgniteClusterSnapshotSelfTest1%][IgniteTestResources]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
[type=CRITICAL_ERROR, err=java.lang.AssertionError:
GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryCustomEvent
[customMsg=null, affTopVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ed2e276b-79a3-4de3-925a-68d60200,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1588507532715, loc=false,
ver=2.9.0#20200503-sha1:, isClient=false], topVer=3, nodeId8=f6fa6fad,
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1588507535430]],
crd=TcpDiscoveryNode [id=ed2e276b-79a3-4de3-925a-68d60200,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1588507532715, loc=false,
ver=2.9.0#20200503-sha1:, isClient=false],
exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=3,
minorTopVer=1], discoEvt=DiscoveryCustomEvent [customMsg=null,
affTopVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ed2e276b-79a3-4de3-925a-68d60200,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1588507532715, loc=false,
ver=2.9.0#20200503-sha1:, isClient=false], topVer=3, nodeId8=f6fa6fad,
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1588507535430]], nodeId=ed2e276b,
evt=DISCOVERY_CUSTOM_EVT], added=true, exchangeType=ALL,
initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true,
hash=1771643357], init=false, lastVer=null,
partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion
[topVer=3, minorTopVer=1], futures=[ExplicitLockReleaseFuture
[topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], futures=[]],
AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=3,
minorTopVer=1], futures=[]], DataStreamerReleaseFuture
[topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1], futures=[]],
LocalTxReleaseFuture [topVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
futures=[]], AllTxReleaseFuture [topVer=AffinityTopologyVersion [topVer=3,
minorTopVer=1], futures=[RemoteTxReleaseFuture [topVer=AffinityTopologyVersion
[topVer=3, minorTopVer=1], futures=[]], exchActions=null,
affChangeMsg=null, centralizedAff=false, forceAffReassignment=false,
exchangeLocE=null, cacheChangeFailureMsgSent=false, done=true, state=SRV,
registerCachesFuture=null, partitionsSent=false, partitionsReceived=false,
delayedLatestMsg=null, afterLsnrCompleteFut=GridFutureAdapter
[ignoreInterrupts=false, state=DONE, res=null, hash=197787912],
timeBag=o.a.i.i.util.TimeBag@1d97d94c, startTime=1365700834756,
initTime=1588507535439, rebalanced=false, evtLatch=0, remaining=HashSet
[ed2e276b-79a3-4de3-925a-68d60200], mergedJoinExchMsgs=null,
awaitMergedMsgs=0, super=GridFutureAdapter [ignoreInterrupts=false, state=DONE,
res=AffinityTopologyVersion [topVer=3, minorTopVer=1], hash=530592354
java.lang.AssertionError: GridDhtPartitionsExchangeFuture
[firstDiscoEvt=DiscoveryCustomEvent [customMsg=null,
affTopVer=AffinityTopologyVersion [topVer=3, minorTopVer=1],
super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=ed2e276b-79a3-4de3-925a-68d60200,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1,
intOrder=1, lastExchangeTime=1588507532715, loc=false,
ver=2.9.0#20200503-sha1:, isClient=false], topVer=3, nodeId8=f6fa6fad,
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1588507535430]],
crd=TcpDiscoveryNode [id=ed2e276b-79a3-4de3-925a-68d60200,
consistentId=snapshot.IgniteClusterSnapshotSelfTest0, addrs=ArrayList
[127.0.0.1], sockAddrs=HashSet
Re: [DISCUSSION] Pull-request checks on GitHub
Pavel,
> I think this thread is a good opportunity to discuss commit message
> guidelines.
We had a thread about it recently [1].
[1]
https://lists.apache.org/thread.html/rde6e8258537704433286a60e1d0142485c25228a46561544d35b9704%40%3Cdev.ignite.apache.org%3E
Best regards,
Ivan Pavlukhin
пн, 4 мая 2020 г. в 10:38, Ivan Pavlukhin :
>
> Maxim,
>
> Thanks again for doing great things!
>
> Out of curiosity, could you please shed a light why there are 2 travis
> checks for PR [1]? I am about checks named
> continuous-integration/travis-ci/pr and
> continuous-integration/travis-ci/push.
>
> Best regards,
> Ivan Pavlukhin
>
> вс, 3 мая 2020 г. в 13:08, Pavel Tupitsyn :
> >
> > Igniters, Maxim,
> >
> > I think this thread is a good opportunity to discuss commit message
> > guidelines.
> > I suggest the following:
> >
> > 1. Treat PR title + description as the final squashed commit message.
> > PR author is responsible for writing that properly.
> > Committer who merges the PR is responsible for validating that and using
> > that for the actual squash commit.
> >
> > 2. Adopt the following Git commit message rules (partially from
> > https://chris.beams.io/posts/git-commit/):
> > - Start with IGNITE-
> > - Use imperative mood in the subject line ("Fix foobar crash on start",
> > "Add baz metric")
> > - Capitalize the subject line
> > - Do not end the subject line with a period
> > - Use the body to explain what and why vs. how
> >
> > Thoughts?
> >
> > On Sun, May 3, 2020 at 11:53 AM Maxim Muzafarov wrote:
> >
> > > Hello,
> > >
> > > I have the following in my mind:
> > > 1. This checklist is for discussion and may be changed.
> > > 2. Commits can be squashed in the branch prior to asking a review, but
> > > when the review is in progress a good naming may help to understand
> > > the changes.
> > > 3. It's true that the commit message can be changed prior to merging
> > > the master branch, but it's better to merge the PR with an initial
> > > authored commit message `as is`.
> > >
> > > On Sat, 2 May 2020 at 18:20, Guru Stron wrote:
> > > >
> > > > Maxim,
> > > >
> > > > I have a small question about "Commits have the following pattern..". Is
> > > > it really needed cause AFAIK commits in the PR are squashed. Or am I
> > > > missing something?
> > > >
> > > > On Thu, Apr 30, 2020, 8:33 PM Maxim Muzafarov wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > >
> > > > > I've created the pull request template for GitHub.
> > > > > Please, take a look and let me know what you think [1] [2].
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > https://github.com/apache/ignite/pull/7765/files#diff-195a635ad245ded9076330e31134bd80
> > > > > [2] https://issues.apache.org/jira/browse/IGNITE-12937
> > > > >
> > > > > On Sun, 26 Apr 2020 at 20:35, Saikat Maitra
> > > > > wrote:
> > > > > >
> > > > > > Hi Maxim,
> > > > > >
> > > > > > Thank you for enabling travis ci in ignite repo. It is very helpful
> > > to
> > > > > see
> > > > > > PR build results integrated in PR request.
> > > > > >
> > > > > > I will enable it in ignite-extensions repo as well.
> > > > > >
> > > > > > Regards,
> > > > > > Saikat
> > > > > >
> > > > > > On Mon, Apr 20, 2020 at 12:14 PM Pavel Tupitsyn <
> > > ptupit...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Maxim, pull request template is a great idea.
> > > > > > > We can put a checklist there along with the links to the
> > > guidelines,
> > > > > > > something like this:
> > > > > > >
> > > > > > > [ ] Coding Guidelines are followed
> > > > > > > [ ] TeamCity build passes
> > > > > > > [ ] JIRA ticked is in Patch Available state, review has been
> > > requested
> > > > > in
> > > > > > > comments
> > > > > > > [ ] Something else?
> > > > > > >
> > > > > > > On Mon, Apr 20, 2020 at 8:09 PM Maxim Muzafarov > > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > > > > > Sorry for the incomplete message.
> > > > > > > >
> > > > > > > > 2. I mentioned it too. Hopefully, builds > 4 hrs would not be
> > > > > > > > too
> > > > > often.
> > > > > > > > The reason - there are limited job-workers shared between all
> > > > > > > > the
> > > > > > > > Apache projects. I found some details here [1] [2].
> > > > > > > >
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > >
> > > > >
> > > https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E
> > > > > > > > [2] https://issues.apache.org/jira/browse/INFRA-18533
> > > > > > > >
> > > > > > > > On Mon, 20 Apr 2020 at 20:03, Maxim Muzafarov > > >
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > Pavel,
> > > > > > > > >
> > > > > > > > > 1. Agree here. What if we create a default template pull
> > > request
> > > > > > > > > description with all the links required by our development
> > > process?
> > > > > > > > > [1] It's will be friendly for contributors to have everything
> > > they
> > > > > > > > > need in the PR.
> > >
Re: [DISCUSSION] Pull-request checks on GitHub
Maxim,
Thanks again for doing great things!
Out of curiosity, could you please shed a light why there are 2 travis
checks for PR [1]? I am about checks named
continuous-integration/travis-ci/pr and
continuous-integration/travis-ci/push.
Best regards,
Ivan Pavlukhin
вс, 3 мая 2020 г. в 13:08, Pavel Tupitsyn :
>
> Igniters, Maxim,
>
> I think this thread is a good opportunity to discuss commit message
> guidelines.
> I suggest the following:
>
> 1. Treat PR title + description as the final squashed commit message.
> PR author is responsible for writing that properly.
> Committer who merges the PR is responsible for validating that and using
> that for the actual squash commit.
>
> 2. Adopt the following Git commit message rules (partially from
> https://chris.beams.io/posts/git-commit/):
> - Start with IGNITE-
> - Use imperative mood in the subject line ("Fix foobar crash on start",
> "Add baz metric")
> - Capitalize the subject line
> - Do not end the subject line with a period
> - Use the body to explain what and why vs. how
>
> Thoughts?
>
> On Sun, May 3, 2020 at 11:53 AM Maxim Muzafarov wrote:
>
> > Hello,
> >
> > I have the following in my mind:
> > 1. This checklist is for discussion and may be changed.
> > 2. Commits can be squashed in the branch prior to asking a review, but
> > when the review is in progress a good naming may help to understand
> > the changes.
> > 3. It's true that the commit message can be changed prior to merging
> > the master branch, but it's better to merge the PR with an initial
> > authored commit message `as is`.
> >
> > On Sat, 2 May 2020 at 18:20, Guru Stron wrote:
> > >
> > > Maxim,
> > >
> > > I have a small question about "Commits have the following pattern..". Is
> > > it really needed cause AFAIK commits in the PR are squashed. Or am I
> > > missing something?
> > >
> > > On Thu, Apr 30, 2020, 8:33 PM Maxim Muzafarov wrote:
> > >
> > > > Folks,
> > > >
> > > >
> > > > I've created the pull request template for GitHub.
> > > > Please, take a look and let me know what you think [1] [2].
> > > >
> > > >
> > > > [1]
> > > >
> > https://github.com/apache/ignite/pull/7765/files#diff-195a635ad245ded9076330e31134bd80
> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12937
> > > >
> > > > On Sun, 26 Apr 2020 at 20:35, Saikat Maitra
> > > > wrote:
> > > > >
> > > > > Hi Maxim,
> > > > >
> > > > > Thank you for enabling travis ci in ignite repo. It is very helpful
> > to
> > > > see
> > > > > PR build results integrated in PR request.
> > > > >
> > > > > I will enable it in ignite-extensions repo as well.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > > On Mon, Apr 20, 2020 at 12:14 PM Pavel Tupitsyn <
> > ptupit...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Maxim, pull request template is a great idea.
> > > > > > We can put a checklist there along with the links to the
> > guidelines,
> > > > > > something like this:
> > > > > >
> > > > > > [ ] Coding Guidelines are followed
> > > > > > [ ] TeamCity build passes
> > > > > > [ ] JIRA ticked is in Patch Available state, review has been
> > requested
> > > > in
> > > > > > comments
> > > > > > [ ] Something else?
> > > > > >
> > > > > > On Mon, Apr 20, 2020 at 8:09 PM Maxim Muzafarov > >
> > > > wrote:
> > > > > >
> > > > > > > Pavel,
> > > > > > >
> > > > > > > Sorry for the incomplete message.
> > > > > > >
> > > > > > > 2. I mentioned it too. Hopefully, builds > 4 hrs would not be too
> > > > often.
> > > > > > > The reason - there are limited job-workers shared between all the
> > > > > > > Apache projects. I found some details here [1] [2].
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > >
> > https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E
> > > > > > > [2] https://issues.apache.org/jira/browse/INFRA-18533
> > > > > > >
> > > > > > > On Mon, 20 Apr 2020 at 20:03, Maxim Muzafarov > >
> > > > wrote:
> > > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > > > > > 1. Agree here. What if we create a default template pull
> > request
> > > > > > > > description with all the links required by our development
> > process?
> > > > > > > > [1] It's will be friendly for contributors to have everything
> > they
> > > > > > > > need in the PR.
> > > > > > > >
> > > > > > > > 2.
> > > > > > > >
> > > > > > > > [1]
> > > > > > >
> > > > > >
> > > >
> > https://help.github.com/en/github/building-a-strong-community/creating-a-pull-request-template-for-your-repository
> > > > > > > >
> > > > > > > > On Mon, 20 Apr 2020 at 19:46, Pavel Tupitsyn <
> > ptupit...@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Maxim,
> > > > > > > > >
> > > > > > > > > Good news, thank you.
> > > > > > > > >
> > > > > > > > > However, I see two issues with this:
> > > > > > > > >
> > > > > > > > > 1. False sense of a ready-to-merge PR
> > > > > > > > > Now that we have a
Extended logging for rebalance performance analysis
Hi, Igniters! I'd like to share a new small feature in AI [1]. Current rebalance logging does not allow you to quickly answer following questions: 1)How long was the balance(divided by supplier)? 2)How many records and bytes per supplier were rebalanced? 3)How many times did rebalance restart? 4)Which partitions were rebalanced and from which nodes did they receive them? 5)When did rebalance for all cache groups end? What you can see in logs now: 1)Starting rebalance with order of cache groups. Rebalancing scheduled [order=[ignite-sys-cache, grp1, grp0], top=AffinityTopologyVersion [topVer=2, minorTopVer=0], force=false, evt=NODE_JOINED, node=c2146a04-dc23-4bc9-870d-dfbb55c1] 2)Start rebalance of cache group from a specific supplier, specifying partition ids and mode - historical or full. Starting rebalance routine [ignite-sys-cache, topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], supplier=8c525892-703b-4fc4-b28b-b2f13970, fullPartitions=[0-99], histPartitions=[]] 3)Getting partial or complete partitions of cache group. Completed rebalancing [grp=ignite-sys-cache, supplier=8c525892-703b-4fc4-b28b-b2f13970, topVer=AffinityTopologyVersion [topVer=5, minorTopVer=0], progress=1/2] Completed (final) rebalancing [grp=ignite-sys-cache, supplier=c2146a04-dc23-4bc9-870d-dfbb55c1, topVer=AffinityTopologyVersion [topVer=5, minorTopVer=0], progress=2/2] 4)End rebalance of cache group. Completed rebalance future: RebalanceFuture [grp=CacheGroupContext [grp=ignite-sys-cache], topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], rebalanceId=1, routines=1, receivedBytes=1200, receivedKeys=0, partitionsLeft=0, startTime=1588519707607, endTime=-1, lastCancelledTime=-1] Rebalance statistics: To speed up rebalance analysis, statistics will be output for each cache group and total for all cache groups. If duration rebalance for cache group is greater than threshold value, partition distribution is output. Statistics will you to analyze duration of the balance for each supplier to understand which of them has been transmitting data for longest time. System properties are used to output statistics: IGNITE_QUIET - to output statistics, value must be false; IGNITE_WRITE_REBALANCE_PARTITION_DISTRIBUTION_THRESHOLD - threshold duration rebalance of cache group after which partitions distribution is output, set in milliseconds, default value is 10 minutes. Statistics examples: Successful full and historical rebalance of group cache, without partitions distribution. Rebalance information per cache group (successful rebalance): [id=3181548, name=grp1, startTime=2020-04-13 10:55:16,117, finishTime=2020-04-13 10:55:16,127, d=10 ms, restarted=0] Supplier statistics: [nodeId=0, p=5, d=10 ms] [nodeId=1, p=5, d=10 ms] Aliases: p - partitions, e - entries, b - bytes, d - duration, h - historical, nodeId mapping (nodeId=id,consistentId) [0=rebalancing.RebalanceStatisticsTest1] [1=rebalancing.RebalanceStatisticsTest0] Rebalance information per cache group (successful rebalance): [id=3181547, name=grp0, startTime=2020-04-13 15:01:44,000, finishTime=2020-04-13 15:01:44,116, d=116 ms, restarted=0] Supplier statistics: [nodeId=0, hp=10, he=300, hb=30267, d=116 ms] Aliases: p - partitions, e - entries, b - bytes, d - duration, h - historical, nodeId mapping (nodeId=id,consistentId) [0=rebalancing.RebalanceStatisticsTest0] Successful full and historical rebalance of group cache, with partitions distribution. Rebalance information per cache group (successful rebalance): [id=3181548, name=grp1, startTime=2020-04-13 10:55:16,117, finishTime=2020-04-13 10:55:16,127, d=10 ms, restarted=0] Supplier statistics: [nodeId=0, p=5, d=10 ms] [nodeId=1, p=5, d=10 ms] Aliases: p - partitions, e - entries, b - bytes, d - duration, h - historical, nodeId mapping (nodeId=id,consistentId) [0=rebalancing.RebalanceStatisticsTest1] [1=rebalancing.RebalanceStatisticsTest0] Rebalance duration was greater than 5 ms, printing detailed information about partitions distribution (threshold can be changed by setting number of milliseconds into IGNITE_WRITE_REBALANCE_PARTITION_DISTRIBUTION_THRESHOLD) 0 = [0,bu,su],[1,bu],[2,pr,su] 1 = [0,bu,su],[1,bu],[2,pr,su] 2 = [0,bu,su],[1,bu],[2,pr,su] 3 = [0,bu,su],[1,bu],[2,pr,su] 4 = [0,bu,su],[1,bu],[2,pr,su] 5 = [0,bu,su],[1,bu],[2,pr,su] 6 = [0,bu,su],[1,bu],[2,pr,su] 7 = [0,bu,su],[1,bu],[2,pr,su] 8 = [0,bu,su],[1,bu],[2,pr,su] 9 = [0,bu,su],[1,bu],[2,pr,su] Aliases: pr - primary, bu - backup, su - supplier node, h - historical, nodeId mapping (nodeId=id,consistentId) [0=rebalancing.RebalanceStatisticsTest1] [1=rebalancing.RebalanceStatisticsTest2] [2=rebalancing.RebalanceStatisticsTest0] Rebalance information per cache group (successful rebalance): [id=3181547, name=grp0, startTime=2020-04-13 15:01:44,000, finishTime=2020-04-13 15:01:44,116, d=116 ms, restarted=0] Supplier statistics: [nodeId=0, hp=10, he=300, hb=30267, d=116 ms]
[REVIEW] IGNITE-12980 Use Apache Ignite logo with registered trade mark symbol in README.md
Folks, As you might know, not long ago Apache Ignite became a registered trademark. I noticed that we still have (TM) symbol in https://github.com/apache/ignite/blob/master/README.md I prepared a ticket [1] and PR. I feel that the used approach might be improved, but I tried to do my best in a lack of time. Please review. [1] https://issues.apache.org/jira/browse/IGNITE-12980 Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-12980) Use Apache Ignite logo with registered trade mark symbol in README.md
Ivan Pavlukhin created IGNITE-12980: --- Summary: Use Apache Ignite logo with registered trade mark symbol in README.md Key: IGNITE-12980 URL: https://issues.apache.org/jira/browse/IGNITE-12980 Project: Ignite Issue Type: Task Reporter: Ivan Pavlukhin Logo in the top of https://github.com/apache/ignite/blob/master/README.md should have registered trademark symbol ® instead of unregistered one ™. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[DISCUSS] Remove TC badge from README.md
Hi Igniters, Inspired by a neighbor thread about PR checks [1]. It brought my attention that now we have a neat Travis badge and a strange TC badge which is not very useful from a first glance. What do you think, is it ought to completely remove TC badge from readme? [1] https://lists.apache.org/thread.html/r1580e2fb23728e83afbe1bad2b4cf7e9cece0e19a1d305e5755d7dd1%40%3Cdev.ignite.apache.org%3E Best regards, Ivan Pavlukhin
