[
https://issues.apache.org/jira/browse/GOBBLIN-2065?focusedWorklogId=918446&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-918446
]
ASF GitHub Bot logged work on GOBBLIN-2065:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 09/May/24 01:37
Start Date: 09/May/24 01:37
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3948:
URL: https://github.com/apache/gobblin/pull/3948#discussion_r1594862828
##########
gobblin-data-management/src/test/java/org/apache/gobblin/data/management/retention/CleanableMysqlDatasetStoreDatasetTest.java:
##########
@@ -112,6 +112,7 @@ public void setUp() throws Exception {
@AfterClass(alwaysRun = true)
public void tearDown() throws Exception {
if (testMetastoreDatabase != null) {
+ // `.close()` to avoid (in the aggregate, across multiple suites) -
java.sql.SQLNonTransientConnectionException: Too many connections
Review Comment:
added doc to alert future maintainers to the reason
##########
gobblin-metastore/src/test/java/org/apache/gobblin/metastore/testing/TestMetastoreDatabaseServer.java:
##########
@@ -96,6 +96,8 @@ class TestMetastoreDatabaseServer implements Closeable {
.withPort(this.dbPort)
.withUser(this.dbUserName, this.dbUserPassword)
.withServerVariable("explicit_defaults_for_timestamp", "off")
+ // default `max_connections` is apparently 151 - see:
https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_connections
+ .withServerVariable("max_connections", "501")
Review Comment:
this seemed to be the deciding factor for successful local execution
##########
gobblin-service/src/test/java/org/apache/gobblin/service/modules/orchestration/proc/ReevaluateDagProcTest.java:
##########
@@ -61,18 +64,36 @@ public class ReevaluateDagProcTest {
private final long flowExecutionId = System.currentTimeMillis();
private final String flowGroup = "fg";
- void mockDMSS(DagManagementStateStore dagManagementStateStore) throws
IOException, SpecNotFoundException {
+ private ITestMetastoreDatabase testMetastoreDatabase;
+ private DagManagementStateStore dagManagementStateStore;
+
+ @BeforeClass
+ public void setUpClass() throws Exception {
+ this.testMetastoreDatabase = TestMetastoreDatabaseFactory.get();
+ }
+
+ @BeforeMethod
+ public void setUp() throws Exception {
+ this.dagManagementStateStore =
spy(MostlyMySqlDagManagementStateStoreTest.getDummyDMSS(this.testMetastoreDatabase));
+ mockDMSSCommonBehavior(dagManagementStateStore);
+ }
+
+ private void mockDMSSCommonBehavior(DagManagementStateStore
dagManagementStateStore) throws IOException, SpecNotFoundException {
doReturn(FlowSpec.builder().build()).when(dagManagementStateStore).getFlowSpec(any());
doNothing().when(dagManagementStateStore).tryAcquireQuota(any());
doNothing().when(dagManagementStateStore).addDagNodeState(any(), any());
+ doNothing().when(dagManagementStateStore).deleteDagNodeState(any(), any());
doReturn(true).when(dagManagementStateStore).releaseQuota(any());
}
+ @AfterClass(alwaysRun = true)
+ public void tearDownClass() throws Exception {
+ // `.close()` to avoid (in the aggregate, across multiple suites) -
java.sql.SQLNonTransientConnectionException: Too many connections
+ this.testMetastoreDatabase.close();
+ }
+
@Test
public void testOneNextJobToRun() throws Exception {
- ITestMetastoreDatabase testMetastoreDatabase =
TestMetastoreDatabaseFactory.get();
- DagManagementStateStore dagManagementStateStore =
spy(MostlyMySqlDagManagementStateStoreTest.getDummyDMSS(testMetastoreDatabase));
- mockDMSS(dagManagementStateStore);
Review Comment:
while each test uses divergent mocking, they can still initialize the bulk
of the spy setup in a `@BeforeMethod`
##########
gobblin-runtime/src/test/java/org/apache/gobblin/runtime/cli/JobStateStoreCliTest.java:
##########
@@ -132,6 +132,14 @@ public void setUp() throws Exception {
jobState);
}
+ @AfterClass(alwaysRun = true)
+ public void tearDown() throws Exception {
Review Comment:
moving `tearDown` whenever it didn't directly follow `setUp`
##########
gobblin-runtime/src/test/java/org/apache/gobblin/runtime/spec_store/MysqlBaseSpecStoreTest.java:
##########
@@ -48,23 +48,23 @@ public class MysqlBaseSpecStoreTest {
private static final String PASSWORD = "testPassword";
private static final String TABLE = "base_spec_store";
+ private ITestMetastoreDatabase testDb;
Review Comment:
mostly this DB is NOT `static`, but sometimes it was. for uniformity making
it always instance-level
Issue Time Tracking
-------------------
Worklog Id: (was: 918446)
Time Spent: 0.5h (was: 20m)
> close ITestMetastoreDatabase in all tests
> -----------------------------------------
>
> Key: GOBBLIN-2065
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2065
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> if test db is not closed, it will eventually give `TooManyConnections`
> exception
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
