[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v10.patch Will likely be committing this patch. Will give QA run another shot. The only flappers I know of at this stage are the ones extending BaseQueryIT and OnDuplicateKeyIT. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, PHOENIX-4110_v10.patch, > PHOENIX-4110_v10.patch, PHOENIX-4110_V2_4.x-HBase-0.98.patch, > PHOENIX-4110_v3_4.x-HBase-0.98.patch, PHOENIX-4110_v3.patch, > PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch, PHOENIX-4110_v6.patch, > PHOENIX-4110_v7.patch, PHOENIX-4110_v8.patch, PHOENIX-4110_v9.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v10.patch > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, PHOENIX-4110_v10.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch, > PHOENIX-4110_v6.patch, PHOENIX-4110_v7.patch, PHOENIX-4110_v8.patch, > PHOENIX-4110_v9.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v9.patch Here is a better more performant approach. It tears down the mini cluster in an async fashion so that tests don't have to wait for it to shut down completely. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch, > PHOENIX-4110_v6.patch, PHOENIX-4110_v7.patch, PHOENIX-4110_v8.patch, > PHOENIX-4110_v9.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v8.patch Another attempt for qa run. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch, > PHOENIX-4110_v6.patch, PHOENIX-4110_v7.patch, PHOENIX-4110_v8.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v7.patch My previous patch had the problem of having the methods as static. Also, there are cases when we override the default driver properties. So in this patch, I am saving the cluster configuration and driver properties before reinitializing the mini cluster and driver. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch, > PHOENIX-4110_v6.patch, PHOENIX-4110_v7.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v6.patch Updated patch. I noticed that some tests like HashJoinIT create almost 100 tables. When shutting down the mini cluster for these many regions, I have seen instances where mini cluster shutdown ran into issues. We can be a bit smarter about it and shutdown and restart the mini cluster in @Before and @After methods. Let's see if this helps. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch, > PHOENIX-4110_v6.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v5.patch > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch, PHOENIX-4110_v5.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v4.patch Patch on top of PHOENIX-4131. I have increased the number of tables to 40. I don't want to increase the number too much since shutting down the mini cluster will take longer since it will have to close at least that many regions. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch, PHOENIX-4110_v4.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v3_4.x-HBase-0.98.patch > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3_4.x-HBase-0.98.patch, > PHOENIX-4110_v3.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_v3.patch Ugh! I don't know why QA cannot apply the patch for 0.98 branch. [~mujtabachohan] or [~elserj] - any ideas? In the mean time, attaching a patch for master. Again, won't commit it. > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, > PHOENIX-4110_V2_4.x-HBase-0.98.patch, PHOENIX-4110_v3.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110_V2_4.x-HBase-0.98.patch > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch, PHOENIX-4110_V2_4.x-HBase-0.98.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4110) ParallelRunListener should monitor number of tables and not number of tests
[ https://issues.apache.org/jira/browse/PHOENIX-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4110: -- Attachment: PHOENIX-4110.patch > ParallelRunListener should monitor number of tables and not number of tests > --- > > Key: PHOENIX-4110 > URL: https://issues.apache.org/jira/browse/PHOENIX-4110 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4110.patch > > > ParallelRunListener today monitors the number of tests that have been run to > determine when mini cluster should be shut down. This helps prevent our test > JVM forks running in OOM. A better heuristic would be to instead check the > number of tables that were created by tests. This way when a particular test > class has created lots of tables, we can shut down the mini cluster sooner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
