echauchot commented on code in PR #22667:
URL: https://github.com/apache/flink/pull/22667#discussion_r1223195930
##########
flink-architecture-tests/flink-architecture-tests-production/archunit-violations/e5126cae-f3fe-48aa-b6fb-60ae6cc3fcd5:
##########
@@ -7,22 +7,22 @@ Method <org.apache.flink.runtime.blob.BlobInputStream.read()>
calls method <org.
Method <org.apache.flink.runtime.blob.BlobInputStream.read([B, int, int)>
calls method <org.apache.flink.runtime.blob.BlobKey.getHash()> in
(BlobInputStream.java:163)
Method
<org.apache.flink.runtime.blob.BlobOutputStream.receiveAndCheckPutResponse(java.io.InputStream,
java.security.MessageDigest, org.apache.flink.runtime.blob.BlobKey$BlobType)>
calls method <org.apache.flink.runtime.blob.BlobKey.getHash()> in
(BlobOutputStream.java:155)
Method
<org.apache.flink.runtime.blob.BlobUtils.checkAndDeleteCorruptedBlobs(java.nio.file.Path,
org.slf4j.Logger)> calls method
<org.apache.flink.runtime.blob.BlobKey.getHash()> in (BlobUtils.java:514)
-Method
<org.apache.flink.runtime.blob.FileSystemBlobStore.get(java.lang.String,
java.io.File, org.apache.flink.runtime.blob.BlobKey)> calls method
<org.apache.flink.runtime.blob.BlobKey.getHash()> in
(FileSystemBlobStore.java:124)
-Method
<org.apache.flink.runtime.executiongraph.Execution.finishPartitionsAndUpdateConsumers()>
calls method
<org.apache.flink.runtime.executiongraph.ExecutionVertex.finishPartitionsIfNeeded()>
in (Execution.java:978)
+Method
<org.apache.flink.runtime.blob.FileSystemBlobStore.get(java.lang.String,
java.io.File, org.apache.flink.runtime.blob.BlobKey)> calls method
<org.apache.flink.runtime.blob.BlobKey.getHash()> in
(FileSystemBlobStore.java:133)
+Method
<org.apache.flink.runtime.executiongraph.Execution.finishPartitionsAndUpdateConsumers()>
calls method
<org.apache.flink.runtime.executiongraph.ExecutionVertex.finishPartitionsIfNeeded()>
in (Execution.java:981)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.grantDispatcherLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.grantLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:83)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.grantJobMasterLeadership(org.apache.flink.api.common.JobID)>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.grantLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:95)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.grantResourceManagerLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.grantLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:109)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.revokeDispatcherLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.revokeLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:77)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.revokeJobMasterLeadership(org.apache.flink.api.common.JobID)>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.revokeLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:89)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.revokeResourceManagerLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.revokeLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:102)
Method
<org.apache.flink.runtime.io.network.partition.ResourceManagerPartitionTrackerImpl.listDataSets()>
calls method
<org.apache.flink.runtime.io.network.partition.DataSetMetaInfo.withNumRegisteredPartitions(int,
int)> in (ResourceManagerPartitionTrackerImpl.java:286)
-Method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.closeAsync(long)>
calls method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.quiesce()>
in (RecreateOnResetOperatorCoordinator.java:343)
+Method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.closeAsync(long)>
calls method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.quiesce()>
in (RecreateOnResetOperatorCoordinator.java:349)
Review Comment:
> > I guess because I rebased on master and prod code has changed ?
>
> Shouldn't it then fail on master if line numbers weren't updated? I had
this issue with Java 17. But the new numbers are definitely correct.
Yes you're right, regular CI on master should fail in that case, I haven't
checked but if it were the case we would have seen. TBH I don't know what
happens with the line numbers, but I used java 11 to update the violations.
>
> > using mvn clean verify does not update the violation store but running
the actual production architecture test does.
>
> TBH I doubt that. You probably just forgot to set one of the magical
archunit properties.
>
I did set `archunit.freeze.refreeze=true` with mvn verify only I set it from
the archunit.properties file not as System property.
> `mvn verify -Darchunit.freeze.refreeze=true` worked fine for me, and I got
the same line number changes locally.
That is comforting.
Could you move these into a separate hotfix commit? They aren't related to
the actual rule fix.
You mean the line numbers update?
##########
flink-architecture-tests/flink-architecture-tests-production/archunit-violations/e5126cae-f3fe-48aa-b6fb-60ae6cc3fcd5:
##########
@@ -7,22 +7,22 @@ Method <org.apache.flink.runtime.blob.BlobInputStream.read()>
calls method <org.
Method <org.apache.flink.runtime.blob.BlobInputStream.read([B, int, int)>
calls method <org.apache.flink.runtime.blob.BlobKey.getHash()> in
(BlobInputStream.java:163)
Method
<org.apache.flink.runtime.blob.BlobOutputStream.receiveAndCheckPutResponse(java.io.InputStream,
java.security.MessageDigest, org.apache.flink.runtime.blob.BlobKey$BlobType)>
calls method <org.apache.flink.runtime.blob.BlobKey.getHash()> in
(BlobOutputStream.java:155)
Method
<org.apache.flink.runtime.blob.BlobUtils.checkAndDeleteCorruptedBlobs(java.nio.file.Path,
org.slf4j.Logger)> calls method
<org.apache.flink.runtime.blob.BlobKey.getHash()> in (BlobUtils.java:514)
-Method
<org.apache.flink.runtime.blob.FileSystemBlobStore.get(java.lang.String,
java.io.File, org.apache.flink.runtime.blob.BlobKey)> calls method
<org.apache.flink.runtime.blob.BlobKey.getHash()> in
(FileSystemBlobStore.java:124)
-Method
<org.apache.flink.runtime.executiongraph.Execution.finishPartitionsAndUpdateConsumers()>
calls method
<org.apache.flink.runtime.executiongraph.ExecutionVertex.finishPartitionsIfNeeded()>
in (Execution.java:978)
+Method
<org.apache.flink.runtime.blob.FileSystemBlobStore.get(java.lang.String,
java.io.File, org.apache.flink.runtime.blob.BlobKey)> calls method
<org.apache.flink.runtime.blob.BlobKey.getHash()> in
(FileSystemBlobStore.java:133)
+Method
<org.apache.flink.runtime.executiongraph.Execution.finishPartitionsAndUpdateConsumers()>
calls method
<org.apache.flink.runtime.executiongraph.ExecutionVertex.finishPartitionsIfNeeded()>
in (Execution.java:981)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.grantDispatcherLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.grantLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:83)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.grantJobMasterLeadership(org.apache.flink.api.common.JobID)>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.grantLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:95)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.grantResourceManagerLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.grantLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:109)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.revokeDispatcherLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.revokeLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:77)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.revokeJobMasterLeadership(org.apache.flink.api.common.JobID)>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.revokeLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:89)
Method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServicesWithLeadershipControl.revokeResourceManagerLeadership()>
calls method
<org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.revokeLeadership()>
in (EmbeddedHaServicesWithLeadershipControl.java:102)
Method
<org.apache.flink.runtime.io.network.partition.ResourceManagerPartitionTrackerImpl.listDataSets()>
calls method
<org.apache.flink.runtime.io.network.partition.DataSetMetaInfo.withNumRegisteredPartitions(int,
int)> in (ResourceManagerPartitionTrackerImpl.java:286)
-Method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.closeAsync(long)>
calls method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.quiesce()>
in (RecreateOnResetOperatorCoordinator.java:343)
+Method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.closeAsync(long)>
calls method
<org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$QuiesceableContext.quiesce()>
in (RecreateOnResetOperatorCoordinator.java:349)
Review Comment:
> These commits should be squashed before the merge as otherwise one commit
alone would fail CI.
>
> ```
> [FLINK-31806] Reverse rule to positive asumption
> [FLINK-31806] Update violations
> ```
sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
