[jira] [Commented] (GEODE-5) Check gradle version when running build
[ https://issues.apache.org/jira/browse/GEODE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573344#comment-14573344 ] ASF subversion and git services commented on GEODE-5: - Commit b5e02b6fa9bdca2df75fb7355b8c6a0c4bf64c92 in incubator-geode's branch refs/heads/develop from mbretl [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b5e02b6 ] [GEODE-5] Add Gradle version check and fail if not using minimum Gradle version. Tested by running build with Gradle 2.2.1, which fails, and with Gradle wrapper, which runs successfully. Reviewed by: AMB Check gradle version when running build --- Key: GEODE-5 URL: https://issues.apache.org/jira/browse/GEODE-5 Project: Geode Issue Type: Improvement Affects Versions: 1.0.0-incubating Reporter: Anthony Baker Assignee: Mark Bretl Priority: Minor Labels: build Fix For: 1.0.0-incubating Attachments: GEODE-5-1.patch, GEODE-5-2.patch Gradle behavior is sometimes version-dependent. When not using the supplied gradle wrapper scripts the build should check to ensure that the gradle version is at least 2.3. If the gradle version is 2.3 the build should fail with a nice message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-20) Allow REST web API to be launched. Eliminate duplicate code
[ https://issues.apache.org/jira/browse/GEODE-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598038#comment-14598038 ] ASF subversion and git services commented on GEODE-20: -- Commit 05e4eb92669f6cff9b22021291c94a2591248041 in incubator-geode's branch refs/heads/develop from William Markito [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=05e4eb9 ] [GEODE-20] - Closing PRs #2 and #9 Changing build.gradle on gemfire-assembly for running junits Allow REST web API to be launched. Eliminate duplicate code --- Key: GEODE-20 URL: https://issues.apache.org/jira/browse/GEODE-20 Project: Geode Issue Type: Bug Components: management tools Affects Versions: 1.0.0-incubating Reporter: Wes Williams Assignee: Wes Williams Fix For: 1.0.0-incubating Original Estimate: 168h Remaining Estimate: 168h GitHub user wwilliams-pivotal opened a pull request: https://github.com/apache/incubator-geode/pull/2 Allow REST web API to be launched. Eliminate duplicate code - Refactored duplicate code into a common class - Changed the return for the web rest dev api to return gemfire-web-api instead of gemfire-web - Applied standard formatting conventions against the existing classes per code policy - Added unit test to check for existence of gemfire-web-api jar You can merge this pull request into a Git repository by running: $ git pull https://github.com/wwilliams-pivotal/incubator-geode master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-geode/pull/2.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2 To make the review simpler, the only two material changes are below. All of the rest is/ (should be) duplicate code refactoring or source code formatting according to the conventions that Anthony sent out earlier. ManagementAgent - - //Find developer REST WAR file - final String gemfireAPIWar = getGemFireAPIWarLocation(gemfireHome); + + // Find developer REST WAR file + final String gemfireAPIWar = agentUtil.getGemFireWebApiWarLocation(); ​RestAgent - - final String gemfireAPIWar = getGemFireAPIWarLocation(gemfireHome); -if(gemfireAPIWar == null){ - logger.info(Unable to find GemFire Developer REST API WAR file; the Developer REST API for GemFire will not be exported and accessible.); +final String gemfireAPIWar = agentUtil.getGemFireWebApiWarLocation(); +if (gemfireAPIWar == null) { -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-20) Allow REST web API to be launched. Eliminate duplicate code
[ https://issues.apache.org/jira/browse/GEODE-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598037#comment-14598037 ] ASF subversion and git services commented on GEODE-20: -- Commit 2cae5b717ee21c1f1c2265a991eefbe1c2eae106 in incubator-geode's branch refs/heads/develop from [~wes.williams] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=2cae5b7 ] [GEODE-20] Allow REST API to be launched Allow REST web API to be launched. Eliminate duplicate code --- Key: GEODE-20 URL: https://issues.apache.org/jira/browse/GEODE-20 Project: Geode Issue Type: Bug Components: management tools Affects Versions: 1.0.0-incubating Reporter: Wes Williams Assignee: Wes Williams Fix For: 1.0.0-incubating Original Estimate: 168h Remaining Estimate: 168h GitHub user wwilliams-pivotal opened a pull request: https://github.com/apache/incubator-geode/pull/2 Allow REST web API to be launched. Eliminate duplicate code - Refactored duplicate code into a common class - Changed the return for the web rest dev api to return gemfire-web-api instead of gemfire-web - Applied standard formatting conventions against the existing classes per code policy - Added unit test to check for existence of gemfire-web-api jar You can merge this pull request into a Git repository by running: $ git pull https://github.com/wwilliams-pivotal/incubator-geode master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-geode/pull/2.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2 To make the review simpler, the only two material changes are below. All of the rest is/ (should be) duplicate code refactoring or source code formatting according to the conventions that Anthony sent out earlier. ManagementAgent - - //Find developer REST WAR file - final String gemfireAPIWar = getGemFireAPIWarLocation(gemfireHome); + + // Find developer REST WAR file + final String gemfireAPIWar = agentUtil.getGemFireWebApiWarLocation(); ​RestAgent - - final String gemfireAPIWar = getGemFireAPIWarLocation(gemfireHome); -if(gemfireAPIWar == null){ - logger.info(Unable to find GemFire Developer REST API WAR file; the Developer REST API for GemFire will not be exported and accessible.); +final String gemfireAPIWar = agentUtil.getGemFireWebApiWarLocation(); +if (gemfireAPIWar == null) { -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-38) Gfsh init script ignored
[ https://issues.apache.org/jira/browse/GEODE-38?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592382#comment-14592382 ] ASF subversion and git services commented on GEODE-38: -- Commit 08182b1830091d7a0ccc6288eb49ba39a7a9b9a0 in incubator-geode's branch refs/heads/develop from [~em-chips] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=08182b1 ] [GEODE-38] Add gfsh init script support for .gfsh2rc files. Fix up log4j2 configuration property value for Windows. Use log4j2XML.toURI().toString() to avoid StatusLogger error about MalformedURLException: unknown protocol: c on Windows. Gfsh init script ignored Key: GEODE-38 URL: https://issues.apache.org/jira/browse/GEODE-38 Project: Geode Issue Type: Bug Components: management tools Affects Versions: 1.0.0-incubating Reporter: Neil Stevenson Assignee: Kirk Lund Priority: Trivial Attachments: 0001-Initial-version.patch, 0002-Review-board-changes.patch Original Estimate: 168h Remaining Estimate: 168h Gfsh launcher script implies an init file can be used. This was possible in GF 6.6, but wasn't implemented in the transition to the new Gfsh in GF 7.0 onwards. From gemfire-assembly/src/main/dist/bin/gfsh.bat : # # Copy default .gfshrc to the home directory. Uncomment if needed. # #if [ ! -f $HOME/.gemfire/.gfsh2rc ]; then # cp $GEMFIRE/defaultConfigs/.gemfire/.gfsh2rc $HOME #fi If this file is specified, it is currently ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-51) Property hostname-for-clients has an inconsistent behavior between server and locator
[ https://issues.apache.org/jira/browse/GEODE-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598711#comment-14598711 ] ASF subversion and git services commented on GEODE-51: -- Commit 3c952ac5e5c948e09c7475170496a46da61ec190 in incubator-geode's branch refs/heads/develop from William Markito [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=3c952ac ] [GEODE-51] - Including hostname-for-client parameters for server startup in order allow external clients connect to a distributed system from a different network. Needed for VMs or Docker deployments Property hostname-for-clients has an inconsistent behavior between server and locator - Key: GEODE-51 URL: https://issues.apache.org/jira/browse/GEODE-51 Project: Geode Issue Type: Bug Components: client/server Reporter: William Markito Oliveira Assignee: William Markito Oliveira When running Geode on Docker I've discovered a couple problems with hostname-for-clients property in order to allow external access to the distributed system: - When setting the property for servers through GFSH it seems to be ignored and only works when using cache.xml. Note that GFSH setting does work for locators. - After specifying hostname-for-clients in cache.xml and attempting to connect using Java API, although it doesn't throw an UnknownHostException but returns 0 servers in the DS. Any operations on the regions return a NoAvailableServersException. Will discuss details with [~dschneider] and confirm the problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-51) Property hostname-for-clients has an inconsistent behavior between server and locator
[ https://issues.apache.org/jira/browse/GEODE-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598712#comment-14598712 ] ASF subversion and git services commented on GEODE-51: -- Commit 552477eb8b63281eb097af7ed37c44498526bbcf in incubator-geode's branch refs/heads/develop from William Markito [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=552477e ] [GEODE-51] - Adding help string information Reviewed by RB #35542 Property hostname-for-clients has an inconsistent behavior between server and locator - Key: GEODE-51 URL: https://issues.apache.org/jira/browse/GEODE-51 Project: Geode Issue Type: Bug Components: client/server Reporter: William Markito Oliveira Assignee: William Markito Oliveira When running Geode on Docker I've discovered a couple problems with hostname-for-clients property in order to allow external access to the distributed system: - When setting the property for servers through GFSH it seems to be ignored and only works when using cache.xml. Note that GFSH setting does work for locators. - After specifying hostname-for-clients in cache.xml and attempting to connect using Java API, although it doesn't throw an UnknownHostException but returns 0 servers in the DS. Any operations on the regions return a NoAvailableServersException. Will discuss details with [~dschneider] and confirm the problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-38) Gfsh init script ignored
[ https://issues.apache.org/jira/browse/GEODE-38?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588468#comment-14588468 ] ASF subversion and git services commented on GEODE-38: -- Commit 19a58e3cf4cbf487e76b7c655d70e43906c80a44 in incubator-geode's branch refs/heads/feature/GEODE-38 from [~em-chips] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=19a58e3 ] [GEODE-38] Add gfsh init script support for .gfsh2rc files. Gfsh init script ignored Key: GEODE-38 URL: https://issues.apache.org/jira/browse/GEODE-38 Project: Geode Issue Type: Bug Components: management tools Affects Versions: 1.0.0-incubating Reporter: Neil Stevenson Assignee: Kirk Lund Priority: Trivial Attachments: 0001-Initial-version.patch, 0002-Review-board-changes.patch Original Estimate: 168h Remaining Estimate: 168h Gfsh launcher script implies an init file can be used. This was possible in GF 6.6, but wasn't implemented in the transition to the new Gfsh in GF 7.0 onwards. From gemfire-assembly/src/main/dist/bin/gfsh.bat : # # Copy default .gfshrc to the home directory. Uncomment if needed. # #if [ ! -f $HOME/.gemfire/.gfsh2rc ]; then # cp $GEMFIRE/defaultConfigs/.gemfire/.gfsh2rc $HOME #fi If this file is specified, it is currently ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-5) Check gradle version when running build
[ https://issues.apache.org/jira/browse/GEODE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571442#comment-14571442 ] ASF subversion and git services commented on GEODE-5: - Commit 75fe2f52ca0f2e95b1887aedd9d4ca6b5b0bbc22 in incubator-geode's branch refs/heads/develop from mbretl [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=75fe2f5 ] [GEODE-5] Add Gradle version check and fail if not using minimum Gradle version. Tested by running build with Gradle 2.2.1, which fails, and with Gradle wrapper, which runs successfully. Reviewed by: AMB Check gradle version when running build --- Key: GEODE-5 URL: https://issues.apache.org/jira/browse/GEODE-5 Project: Geode Issue Type: Improvement Affects Versions: 1.0.0-incubating Reporter: Anthony Baker Assignee: Mark Bretl Priority: Minor Labels: build Attachments: GEODE-5-1.patch, GEODE-5-2.patch Gradle behavior is sometimes version-dependent. When not using the supplied gradle wrapper scripts the build should check to ensure that the gradle version is at least 2.3. If the gradle version is 2.3 the build should fail with a nice message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-71) javadocs for the reason IllegalArgumentException is thrown from addLocator, addServer, addPoolLocator, and addPoolServer are not correct
[ https://issues.apache.org/jira/browse/GEODE-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14610795#comment-14610795 ] ASF subversion and git services commented on GEODE-71: -- Commit 498d7cd4a5e3165f1c8e78b114a5cfd985a53664 in incubator-geode's branch refs/heads/feature/GEODE-71 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=498d7cd ] [GEODE-71] javadocs and log message fixed The javadocs are now correct. Changed the log message about the host being unknown to be warn instead of fatal since the whole point of fixing 45348 was to ignore the unknown host exception until you try to use the pool for an operation. I also updated the log message to say: Creating pool with unknown host in case the host becomes known later. instead of just saying the host was unknown. javadocs for the reason IllegalArgumentException is thrown from addLocator, addServer, addPoolLocator, and addPoolServer are not correct Key: GEODE-71 URL: https://issues.apache.org/jira/browse/GEODE-71 Project: Geode Issue Type: Bug Components: client/server Affects Versions: 1.0.0-incubating Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Trivial Fix For: 1.0.0-incubating Original Estimate: 48h Remaining Estimate: 48h The reason why an IllegalArgumentException is thrown from ClientCacheFactory#addPoolLocator,ClientCacheFactory#addPoolServer,PoolFactory#addLocator, and PoolFactory#addServer is wrong. The javadocs say: if host is an unknown host according to InetAddress.getByName(String) or if port is outside the valid range of [1..65535] inclusive. But they should say: if port is outside the valid range of [0..65535] inclusive. The product was changed in 7.0 to no longer throw an exception if the host is unknown. Instead it waits until the pool is used for an operation. Hopefully by that time the host will be know. If it still is not then an exception will be thrown. Another part of this bug is that when the pool is created with an unknown host a fatal log message that just says Hostname is unknown is logged. Since this is not fatal the message should instead be a warning. The message should also explain that the pool will be created with the unknown host. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-9) Spark Integration
[ https://issues.apache.org/jira/browse/GEODE-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615694#comment-14615694 ] ASF subversion and git services commented on GEODE-9: - Commit 85d44f046bfcf34a835141044f99259e2539ae9d in incubator-geode's branch refs/heads/feature/GEODE-9 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=85d44f0 ] GEODE-9: Imported gemfire-spark-connector from geode-1.0.0-SNAPSHOT-2.src.tar Spark Integration - Key: GEODE-9 URL: https://issues.apache.org/jira/browse/GEODE-9 Project: Geode Issue Type: Sub-task Components: core, extensions Reporter: Dan Smith Labels: asf-migration This is a feature that has been under development for GemFire but was not part of the initial drop of code for geode. Geode as a data store for Spark applications is what is being enabled here. By providing a bridge style connector for Spark applications, Geode can become the data store for storing intermediate and final state for Spark applications and allow reference data stored in the in memory tier to be accessed very efficiently for applications Expose Geode regions as Spark RDDs Write Spark RDDs to Geode Regions Execute arbitrary OQL queries in your spark applications -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-8) Merge Remaining GemFire Code to Geode
[ https://issues.apache.org/jira/browse/GEODE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615428#comment-14615428 ] ASF subversion and git services commented on GEODE-8: - Commit 221cce709af662c6633f72835cbb21ce967740c8 in incubator-geode's branch refs/heads/feature/GEODE-8 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=221cce7 ] Merge branch 'develop' into feature/GEODE-8. Resolved conflicts and did a build. Unit test have not yet been run. Conflicts: RUNNING.txt build.gradle gemfire-assembly/build.gradle gemfire-core/build.gradle gemfire-core/src/main/java/com/gemstone/gemfire/management/internal/ManagementAgent.java gemfire-core/src/main/java/com/gemstone/gemfire/management/internal/RestAgent.java gradle.properties Merge Remaining GemFire Code to Geode - Key: GEODE-8 URL: https://issues.apache.org/jira/browse/GEODE-8 Project: Geode Issue Type: New Feature Components: general Reporter: Dan Smith Assignee: Roman Shaposhnik Labels: asf-migration The initial geode source code came from a feature branch of the GemFire codebase. There are several feature branches, not to mention the mainline develop branch, that were already in progress when that drop was made. This is an umbrella issue to track the status of all of this remaining code that still needs to be migrated to geode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-81) AnalyzeSerializablesJUnitTest fails on ShutdownAllGatewayHubsRequest and GatewaySenderProfile
[ https://issues.apache.org/jira/browse/GEODE-81?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615846#comment-14615846 ] ASF subversion and git services commented on GEODE-81: -- Commit e09886acfddbf7fbfe7f0933e6d001147d5434e0 in incubator-geode's branch refs/heads/feature/GEODE-8 from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=e09886a ] GEODE-81 - Fixing unit test failures from AnalyzeSerializablesJUnitTest AnalyzeSerializablesJUnitTest is failing due new the introduction of a couple of classes that provide backwards compatibility support: ShutdownAllGatewayHubsRequest and GatewaySender$OrderPolicy. These classes need to have signatures added to sanctionedDataSerializables. AnalyzeSerializablesJUnitTest fails on ShutdownAllGatewayHubsRequest and GatewaySenderProfile - Key: GEODE-81 URL: https://issues.apache.org/jira/browse/GEODE-81 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Kishor Bachhav Priority: Blocker Original Estimate: 24h Remaining Estimate: 24h On feature/GEODE-8 the integrationTest target has these failures: {noformat} testDataSerializables java.lang.AssertionError: New or moved classes com/gemstone/gemfire/internal/admin/remote/ShutdownAllGatewayHubsRequest,2 fromData,16,2a2bb700052a2bb900060100b50003b1 toData,16,2a2bb700072b2ab40003b900080200b1 Modified classes com/gemstone/gemfire/internal/cache/wan/GatewaySenderAdvisor$GatewaySenderProfile,4 fromData,282,2a2bb700082a2bb80009b5000a2a2bb9000b0100b5000c2a2bb9000d0100b5000e2a2bb9000f0100b500102a2bb9000f0100b500112a2bb9000f0100b500122a2bb9000f0100b500132a2bb9000f0100b500142a2bb9000d0100b500152a2bb9000f0100b500162a2bb80017b500042a2bb80017b500052a2bb80017b500062a2bb9000f0100b500182a2bb9000d0100b500192bb8001ab2001bb6001c9c00552bb8001dc0001e4d2cc600412cb6001fb20020b60021b6002299000d2ab20020b50023a7002c2cb6001fb20024b60021b6002299000d2ab20024b50023a700122ab20025b50023a700082a01b50023a7000e2a2bb8001dc00026b500232bb800273d1c9900162abb002859b70029b5002a2ab4002a2bb8002bb1 fromDataPre_GFE_8_0_0_0,187,2a2bb700082a2bb80009b5000a2a2bb9000b0100b5000c2a2bb9000d0100b5000e2a2bb9000f0100b500102a2bb9000f0100b500112a2bb9000f0100b500122a2bb9000f0100b500132a2bb9000f0100b500142a2bb9000d0100b500152a2bb9000f0100b500162a2bb80017b500042a2bb80017b500052a2bb80017b500062a2bb9000f0100b500182a2bb9000d0100b500192a2bb8001dc00026b500232bb800273d1c9900162abb002859b70029b5002a2ab4002a2bb8002bb1 toData,270,2a2bb7002c2ab4000a2bb8002d2b2ab4000cb9002e03002b2ab4000eb9002f02002b2ab40010b9003002002b2ab40011b9003002002b2ab40012b9003002002b2ab40013b9003002002b2ab40014b9003002002b2ab40015b9002f02002b2ab40016b9003002002ab400042bb800312ab400052bb800312ab400062bb800312b2ab40018b9003002002b2ab40019b9002f02002bb80032b2001bb6001c9c004a2ab40023c600432ab40023b600214d2cb20033b6001fb6002299000db200332bb80034a700212cb20035b6001fb6002299000db200352bb80034a7000ab200362bb80034a7000b2ab400232bb800342ab4002ac6000704a70004033d1c2bb800371c99000b2ab4002a2bb80038b1 toDataPre_GFE_8_0_0_0,235,2a2bb7002c2ab4000a2bb8002d2b2ab4000cb9002e03002b2ab4000eb9002f02002b2ab40010b9003002002b2ab40011b9003002002b2ab40012b9003002002b2ab40013b9003002002b2ab40014b9003002002b2ab40015b9002f02002b2ab40016b9003002002ab400042bb800312ab400052bb800312ab400062bb800312b2ab40018b9003002002ab4001299000d2b04b9002f0200a7001e2ab40023c7000d2b04b9002f0200a7000d2b2ab40019b9002f02002ab4001299000b012bb80034a7000b2ab400232bb800342ab4002ac6000704a70004033d1c2bb800371c99000b2ab4002a2bb80038b1 : toData (len=270,expected=186), and fromData (len=282,expected=187) If the class is not persisted or sent over the wire add it to the excludedClasses.txt file in the com/gemstone/gemfire/codeAnalysis directory. Otherwise if this doesn't break backward compatibility move the file actualDataSerializables.dat to the codeAnalysis test directory and rename to sanctionedDataSerializables.txt at org.junit.Assert.fail(Assert.java:88) at com.gemstone.gemfire.codeAnalysis.AnalyzeSerializablesJUnitTest.testDataSerializables(AnalyzeSerializablesJUnitTest.java:221) testSerializables java.lang.AssertionError: New or moved classes com/gemstone/gemfire/cache/util/Gateway$OrderPolicy,false If the class is not persisted or sent over the wire add it to the excludedClasses.txt file in the /com/gemstone/gemfire/codeAnalysis/ directory. Otherwise if this doesn't break backward compatibility move the file actualSerializables.dat to the codeAnalysis test directory and rename to sanctionedSerializables.txt at
[jira] [Commented] (GEODE-82) Bad import in GatewaySenderAdvisor caused build failures
[ https://issues.apache.org/jira/browse/GEODE-82?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615842#comment-14615842 ] ASF subversion and git services commented on GEODE-82: -- Commit 8a48d0de95844d92c3b456a793e9b080e7463c09 in incubator-geode's branch refs/heads/feature/GEODE-8 from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=8a48d0d ] GEODE-82 Removing unused import of jrockit class. This import causes the build to fail with openjdk, because jrockit is not available with openjdk. Bad import in GatewaySenderAdvisor caused build failures Key: GEODE-82 URL: https://issues.apache.org/jira/browse/GEODE-82 Project: Geode Issue Type: Bug Components: core Reporter: Dan Smith This appears to be a bug on the GEODE-8 branch. A new, unused import was added to GatewaySenderAdvisor: import com.oracle.jrockit.jfr.DataType; It looks like the successfully compiles with JDK 1.7.0_72 but not with 1.7.0_79. Needless to say, this import shouldn't be in there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-12) Merge Pulse Source Code
[ https://issues.apache.org/jira/browse/GEODE-12?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14615718#comment-14615718 ] ASF subversion and git services commented on GEODE-12: -- Commit c4f7ff44e1ab2b65acc818321c961e7a4fd574a1 in incubator-geode's branch refs/heads/feature/GEODE-12 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=c4f7ff4 ] GEODE-12: Imported pulse from geode-1.0.0-SNAPSHOT-2.src.tar Merge Pulse Source Code --- Key: GEODE-12 URL: https://issues.apache.org/jira/browse/GEODE-12 Project: Geode Issue Type: Sub-task Components: management tools Reporter: Dan Smith Pulse is a web application for monitoring a Geode cluster. Pivotal would like to contribute pulse to the Geode project. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-105) Null value in Map causes NPE with Map Indexes
[ https://issues.apache.org/jira/browse/GEODE-105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14686484#comment-14686484 ] ASF subversion and git services commented on GEODE-105: --- Commit fcd03406c13d8f22b0222b337c6309ab94fce69c in incubator-geode's branch refs/heads/develop from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=fcd0340 ] GEODE-105: Null value in Map causes NPE with Map Indexes Convert null to NULL tokens when saving to the map indexes We now ignore non map objects instead of throwing assertion errors. We do not try to index them for map indexes. Null value in Map causes NPE with Map Indexes - Key: GEODE-105 URL: https://issues.apache.org/jira/browse/GEODE-105 Project: Geode Issue Type: Bug Components: querying Reporter: Jason Huynh Assignee: Jason Huynh When a value of null is added into a map that is being indexed on, a NPE will be thrown. This should not happen and instead should be handled correcty with the NULL token. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701686#comment-14701686 ] ASF subversion and git services commented on GEODE-77: -- Commit d4a3f122555ee6daa15c6aea38adea30bc1a057d in incubator-geode's branch refs/heads/feature/GEODE-77 from [~bschuchardt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=d4a3f12 ] GEODE-77 removed shutdown status check. This check was recently added but it is keeping the manager from properly shutting down because the status is set by DirectChannel before the manager is told to shut down. Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702029#comment-14702029 ] ASF subversion and git services commented on GEODE-77: -- Commit c5c8565c654ad681c7171c7fad582e113cb8e720 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~bschuchardt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=c5c8565 ] GEODE-77 fixing a hang caused by wrong DSFID in RemoveMemberMessage also addressing an authentication issue found by Jason with the new GMSJoinLeaveJUnitTest, and adding a test method for removeMember() Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-229) Javadocs for DiskStoreFactory.setCompactionThreshold are incorrect
[ https://issues.apache.org/jira/browse/GEODE-229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703962#comment-14703962 ] ASF subversion and git services commented on GEODE-229: --- Commit 936065f4194f5df97d83b26ca029e66bb69a8d15 in incubator-geode's branch refs/heads/develop from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=936065f ] GEODE-229: Fixed javadoc for DiskStoreFactory.setCompactionThreshold The javadocs and the behavior of the code were backwards. The code compacts when the live data gets below the threshold. I changed the javadocs to avoid screwing up someone who was relied on the old behavior. Adding a unit test of this property. Javadocs for DiskStoreFactory.setCompactionThreshold are incorrect -- Key: GEODE-229 URL: https://issues.apache.org/jira/browse/GEODE-229 Project: Geode Issue Type: Bug Reporter: Dan Smith Assignee: Dan Smith The javadocs for compaction-threshold imply that we compact as soon as the amount of garbage is equal or greater than the threshold. From DiskStoreFactory?.setCompactionThreshold {noformat} When the amount of garbage in an oplog exceeds this percentage then when a compaction * is done this garbage will be cleaned up freeing up disk space. Garbage is created by * entry destroys, entry updates, and region destroys. {noformat} However, looking at the code, it looks like we actually trigger compaction when the amount of non-garbage data is equal or less than the threshold: Here Oplog.needsCompaction, it looks like its saying the oplog needs compaction if the live count is less than the threshold. {code} long rvHWMtmp = this.totalCount.get(); if (rvHWMtmp 0) { long tlc = this.totalLiveCount.get(); if (tlc 0) { tlc = 0; } double rv = tlc; double rvHWM = rvHWMtmp; if (((rv / rvHWM) * 100) = parent.getCompactionThreshold()) { return true; } } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-213) JMX -http manager treats start gateway-sender as start gateway-receiver
[ https://issues.apache.org/jira/browse/GEODE-213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703912#comment-14703912 ] ASF subversion and git services commented on GEODE-213: --- Commit 581702ac1d2a9debe8619680d09fa868b9ca8b05 in incubator-geode's branch refs/heads/develop from [~apa...@the9muses.net] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=581702a ] GEODE-213: New unit test that confirms bug JMX -http manager treats start gateway-sender as start gateway-receiver --- Key: GEODE-213 URL: https://issues.apache.org/jira/browse/GEODE-213 Project: Geode Issue Type: Bug Components: core Affects Versions: 1.0.0-incubating Reporter: Vitaliy Gavrilov Assignee: Kirk Lund WanCommandsControlle in method startGatewaySender builds up a command using the string constant for gateway receiver instead of sender -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-223) RedisDistDUnitTest.testConcCreateDestroy NullPointerException
[ https://issues.apache.org/jira/browse/GEODE-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703910#comment-14703910 ] ASF subversion and git services commented on GEODE-223: --- Commit 83029e0e6c9e367ab9297f38c0d54a4f8d8ebbe6 in incubator-geode's branch refs/heads/develop from Vito Gavrilov [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=83029e0 ] GEODE-223: Handle region create/destroy remote event in Redis adpater Ignore events where region creation initiated remotely attempts to create a local region reference when the region has already been destroyed. Also, the destruction of a region may be caught the query engine, so I have accounted for that by handling com.gemstone.gemfire.cache.query.RegionNotFoundException. Finally, a the Jedis client timeout has been increased for RedisDistDunitTest to account for concurrent region creation/destruction and an expected exception has been added to not fail over the log scanning. Sometimes when a region is destroyed the PooledMessage Processor will log a regiondestroyed exception, which is ok, but makes the test fail. closes #16 RedisDistDUnitTest.testConcCreateDestroy NullPointerException - Key: GEODE-223 URL: https://issues.apache.org/jira/browse/GEODE-223 Project: Geode Issue Type: Bug Components: extensions Reporter: Darrel Schneider Assignee: Vitaliy Gavrilov com.gemstone.gemfire.redis.RedisDistDUnitTest.testConcCreateDestroy failed because of this suspect string: Found suspect string in log4j at line 1132 [error 2015/08/11 11:39:31.020 PDT P2P message reader for cc3-rh6(17559)v0:13796 shared ordered uid=236 port=51035 tid=0x1506] Exception occurred in CacheListener java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1124) at com.gemstone.gemfire.internal.redis.RegionProvider.createRemoteRegionLocally(RegionProvider.java:222) at com.gemstone.gemfire.redis.GemFireRedisServer.afterKeyCreate(GemFireRedisServer.java:532) at com.gemstone.gemfire.redis.GemFireRedisServer.access$400(GemFireRedisServer.java:128) at com.gemstone.gemfire.redis.GemFireRedisServer$MetaCacheListener.afterCreate(GemFireRedisServer.java:559) at com.gemstone.gemfire.internal.cache.EnumListenerEvent$AFTER_CREATE.dispatchEvent(EnumListenerEvent.java:97) at com.gemstone.gemfire.internal.cache.LocalRegion.dispatchEvent(LocalRegion.java:9251) at com.gemstone.gemfire.internal.cache.LocalRegion.dispatchListenerEvent(LocalRegion.java:7700) at com.gemstone.gemfire.internal.cache.LocalRegion.invokePutCallbacks(LocalRegion.java:6466) at com.gemstone.gemfire.internal.cache.EntryEventImpl.invokeCallbacks(EntryEventImpl.java:2623) at com.gemstone.gemfire.internal.cache.AbstractRegionEntry.dispatchListenerEvents(AbstractRegionEntry.java:157) at com.gemstone.gemfire.internal.cache.LocalRegion.basicPutPart2(LocalRegion.java:6316) at com.gemstone.gemfire.internal.cache.AbstractRegionMap.basicPut(AbstractRegionMap.java:3214) at com.gemstone.gemfire.internal.cache.LocalRegion.virtualPut(LocalRegion.java:6125) at com.gemstone.gemfire.internal.cache.DistributedRegion.virtualPut(DistributedRegion.java:394) at com.gemstone.gemfire.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:118) at com.gemstone.gemfire.internal.cache.LocalRegion.basicUpdate(LocalRegion.java:6100) at com.gemstone.gemfire.internal.cache.AbstractUpdateOperation.doPutOrCreate(AbstractUpdateOperation.java:142) at com.gemstone.gemfire.internal.cache.AbstractUpdateOperation$AbstractUpdateMessage.basicOperateOnRegion(AbstractUpdateOperation.java:280) I think the bug is in this code from RegionProvider: {code} r = cache.getRegion(key.toString()); if (type == RedisDataType.REDIS_LIST) doInitializeList(key, r); else if (type == RedisDataType.REDIS_SORTEDSET) doInitializeSortedSet(key, r); this.regions.put(key, r); {code} The NPE happens on the last line (the put call) because r is null. I think this code just needs to handle the race in which cache.getRegion returns null. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-228) NanoTimer2JUnitTest testReset can fail due to race conditions
[ https://issues.apache.org/jira/browse/GEODE-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703911#comment-14703911 ] ASF subversion and git services commented on GEODE-228: --- Commit f246c7f7bcc662f681dc1abcf109db993f9fad6c in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=f246c7f ] GEODE-228: fix intermittent failures in NanoTimer tests - NanoTimer2JUnitTest has been removed since it was an old test that has been replaced by NanoTimerJUnitTest. - NanoTimer can now be constructed with a TimeService. This allows unit tests to insert a different clock. - NanoTimerJUnitTest now uses its own TimeService when testing NanoTimer which prevents intermittent failures. NanoTimer2JUnitTest testReset can fail due to race conditions - Key: GEODE-228 URL: https://issues.apache.org/jira/browse/GEODE-228 Project: Geode Issue Type: Bug Components: tests Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor NanoTimer2JUnitTest can intermittently. The test is dependent and the system clock and the jvm not having any unexpected pauses. Here is a recent failure from the nightly build: https://builds.apache.org/job/Geode-nightly/188/testReport/junit/com.gemstone.gemfire.internal/NanoTimer2JUnitTest/testReset/ junit.framework.AssertionFailedError: create time expected to be in the range [-19899..19899] but was:20551 at junit.framework.Assert.fail(Assert.java:57) at junit.framework.TestCase.fail(TestCase.java:227) at com.gemstone.gemfire.internal.NanoTimer2JUnitTest.assertApproximate(NanoTimer2JUnitTest.java:74) at com.gemstone.gemfire.internal.NanoTimer2JUnitTest.testReset(NanoTimer2JUnitTest.java:60) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-220) DistributedNoAckRegionOffHeapDUnitTest.testRemoteCacheWriter fails intermittently
[ https://issues.apache.org/jira/browse/GEODE-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703909#comment-14703909 ] ASF subversion and git services commented on GEODE-220: --- Commit 96e889ee9fea560d92b33b1b0336369a42032847 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=96e889e ] GEODE-220: Use wait after off-heap destroyRegion DistributedNoAckRegionOffHeapDUnitTest.testRemoteCacheWriter fails intermittently - Key: GEODE-220 URL: https://issues.apache.org/jira/browse/GEODE-220 Project: Geode Issue Type: Bug Components: tests Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 2h Remaining Estimate: 2h DistributedNoAckRegionOffHeapDUnitTest.testRemoteCacheWriter fails intermittently. The intermittent failure is caused by an off-heap region being destroyed and the test expecting the off-heap stats to be synchronously updated during the destroy. off-heap regions release their off-heap references async in a background thread to prevent dead lock. The following assertEquals(0, ...) needs be in a wait criteria: {code} region.destroyRegion(arg); if (region.getAttributes().getOffHeap() !(region instanceof PartitionedRegion)) { GemFireCacheImpl gfc = (GemFireCacheImpl) getCache(); SimpleMemoryAllocatorImpl ma = (SimpleMemoryAllocatorImpl) gfc.getOffHeapStore(); assertEquals(0, ma.getStats().getObjects()); } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703843#comment-14703843 ] ASF subversion and git services commented on GEODE-77: -- Commit 7fd67b8836b1c0ff95034a9a6f25901e805354f2 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=7fd67b8 ] GEODE-77: improve restart locator error handling on corrupted state file Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707472#comment-14707472 ] ASF subversion and git services commented on GEODE-77: -- Commit f894238b5f9f54665728c69145a6c1c72dba3c21 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~bschuchardt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=f894238 ] GEODE-77 adding test coverage for JGroupsMessenger A couple of unit tests to make sure that request routing is working as expected and that message payload fragmentation is being performed. Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-232) gfsh prints ERROR StatusLogger log4j2-cli.xml not found in file system or classpath
[ https://issues.apache.org/jira/browse/GEODE-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706199#comment-14706199 ] ASF subversion and git services commented on GEODE-232: --- Commit 19732b2d8060c5707448364d998ba287ccd9f727 in incubator-geode's branch refs/heads/develop from [~apa...@the9muses.net] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=19732b2 ] GEODE-232: Convert resource string to URI string for log4j2-cli.xml Add lots of new unit and integration tests for LogService. gfsh prints ERROR StatusLogger log4j2-cli.xml not found in file system or classpath --- Key: GEODE-232 URL: https://issues.apache.org/jira/browse/GEODE-232 Project: Geode Issue Type: Bug Reporter: Eric Shu Assignee: Kirk Lund Priority: Critical ./open/gemfire-assembly/build/install/apache-geode/bin/gfsh ERROR StatusLogger File not found in file system or classpath: /com/gemstone/gemfire/internal/logging/log4j/log4j2-cli.xml ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. _ __ / _/ __/ __/ // / / / __/ /___ /_ / _ / / /__/ / / _/ / // / /__/_/ /__/_//_/v1.0.0-incubating-SNAPSHOT Monitor and Manage GemFire gfsh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-223) RedisDistDUnitTest.testConcCreateDestroy NullPointerException
[ https://issues.apache.org/jira/browse/GEODE-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702245#comment-14702245 ] ASF subversion and git services commented on GEODE-223: --- Commit ff9b2f05ffa6312e3fcd3abc2b7bc96077870829 in incubator-geode's branch refs/heads/develop from Vito Gavrilov [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=ff9b2f0 ] GEODE-223: Handle region create/destroy remote event in Redis adpater Ignore events where region creation initiated remotely attempts to create a local region reference when the region has already been destroyed. Also, the destruction of a region may be caught the query engine, so I have accounted for that by handling com.gemstone.gemfire.cache.query.RegionNotFoundException. Finally, a the Jedis client timeout has been increased for RedisDistDunitTest to account for concurrent region creation/destruction and an expected exception has been added to not fail over the log scanning. Sometimes when a region is destroyed the PooledMessage Processor will log a regiondestroyed exception, which is ok, but makes the test fail. closes #16 RedisDistDUnitTest.testConcCreateDestroy NullPointerException - Key: GEODE-223 URL: https://issues.apache.org/jira/browse/GEODE-223 Project: Geode Issue Type: Bug Components: extensions Reporter: Darrel Schneider Assignee: Vitaliy Gavrilov com.gemstone.gemfire.redis.RedisDistDUnitTest.testConcCreateDestroy failed because of this suspect string: Found suspect string in log4j at line 1132 [error 2015/08/11 11:39:31.020 PDT P2P message reader for cc3-rh6(17559)v0:13796 shared ordered uid=236 port=51035 tid=0x1506] Exception occurred in CacheListener java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1124) at com.gemstone.gemfire.internal.redis.RegionProvider.createRemoteRegionLocally(RegionProvider.java:222) at com.gemstone.gemfire.redis.GemFireRedisServer.afterKeyCreate(GemFireRedisServer.java:532) at com.gemstone.gemfire.redis.GemFireRedisServer.access$400(GemFireRedisServer.java:128) at com.gemstone.gemfire.redis.GemFireRedisServer$MetaCacheListener.afterCreate(GemFireRedisServer.java:559) at com.gemstone.gemfire.internal.cache.EnumListenerEvent$AFTER_CREATE.dispatchEvent(EnumListenerEvent.java:97) at com.gemstone.gemfire.internal.cache.LocalRegion.dispatchEvent(LocalRegion.java:9251) at com.gemstone.gemfire.internal.cache.LocalRegion.dispatchListenerEvent(LocalRegion.java:7700) at com.gemstone.gemfire.internal.cache.LocalRegion.invokePutCallbacks(LocalRegion.java:6466) at com.gemstone.gemfire.internal.cache.EntryEventImpl.invokeCallbacks(EntryEventImpl.java:2623) at com.gemstone.gemfire.internal.cache.AbstractRegionEntry.dispatchListenerEvents(AbstractRegionEntry.java:157) at com.gemstone.gemfire.internal.cache.LocalRegion.basicPutPart2(LocalRegion.java:6316) at com.gemstone.gemfire.internal.cache.AbstractRegionMap.basicPut(AbstractRegionMap.java:3214) at com.gemstone.gemfire.internal.cache.LocalRegion.virtualPut(LocalRegion.java:6125) at com.gemstone.gemfire.internal.cache.DistributedRegion.virtualPut(DistributedRegion.java:394) at com.gemstone.gemfire.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:118) at com.gemstone.gemfire.internal.cache.LocalRegion.basicUpdate(LocalRegion.java:6100) at com.gemstone.gemfire.internal.cache.AbstractUpdateOperation.doPutOrCreate(AbstractUpdateOperation.java:142) at com.gemstone.gemfire.internal.cache.AbstractUpdateOperation$AbstractUpdateMessage.basicOperateOnRegion(AbstractUpdateOperation.java:280) I think the bug is in this code from RegionProvider: {code} r = cache.getRegion(key.toString()); if (type == RedisDataType.REDIS_LIST) doInitializeList(key, r); else if (type == RedisDataType.REDIS_SORTEDSET) doInitializeSortedSet(key, r); this.regions.put(key, r); {code} The NPE happens on the last line (the put call) because r is null. I think this code just needs to handle the race in which cache.getRegion returns null. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701538#comment-14701538 ] ASF subversion and git services commented on GEODE-77: -- Commit 26fea7a161f2336aa5577b0f4b2126d11c9ae37f in incubator-geode's branch refs/heads/feature/GEODE-77 from [~bschuchardt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=26fea7a ] GEODE-77 added Messenger statistics and removed old JGroups statistics This also fixes a few bugs that I found during testing. GMSMember wasn't serializing correctly in some cases. I also found that gfsh has a showDeadlock command and hooked in the new findDeepestGraph DependencyGraph search. If gfsh can't find a deadlock it will now report on the deepest call chain it can find, which often points to the source of a problem in the distributed system. Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-86) KeyValueOperationContext getValue returns a byte array when the value was not a byte array
[ https://issues.apache.org/jira/browse/GEODE-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646500#comment-14646500 ] ASF subversion and git services commented on GEODE-86: -- Commit 298ff7911ffcc7b5f7b026de340b95804b7fab5a in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=298ff79 ] GEODE-86: add a way to get the deserialized value to KeyValueOperationContext KeyValueOperationContext now has a getDeserializedValue method that can be used to get the operation value as a deserialized object. The javadocs on getValue now explain that the result may be a serialized blob. Unit test coverage has been added for PutOperationContext and GetOperationContextImpl which are subclasses of KeyValueOperationContext. KeyValueOperationContext getValue returns a byte array when the value was not a byte array -- Key: GEODE-86 URL: https://issues.apache.org/jira/browse/GEODE-86 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 48h Remaining Estimate: 48h I am using a plain-old Java object and Geode PDX to PUT and GET values from a client app to a Geode region. Thanks to PDX, I don't have to install the class on the Geode cluster. In my implementation of the AccessControl interface, I receive the OperationContext object as part of the pre- and post-authorization callbacks. I was expecting to receive a PdxInstance object but instead I am receiving a byte[] object. In the future, It would be nice to receive a PdxInstance to make the process much cleaner. For now I have to do this something like this to get my PdxInstance object: PdxInputStream pdxIs = new PdxInputStream((byte[]) ((KeyValueOperationContext) context).getValue()); userData = (PdxInstance) DataSerializer.readObject(pdxIs); This looks like a bug in com.gemstone.gemfire.cache.operations.KeyValueOperationContext.getValue(). This method should return to deserialized value for the operation. The javadocs don't actually say this but I think that was the idea. If you wanted to serialized value you could call getSerializedValue(). The problem with the current getValue implementation is that if the context has a serialized value it just returns it. But the serialized value will always be a byte[] that contains the serialized bytes. getValue should only return a byte array if the actual value passed to the operation was a byte array. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646774#comment-14646774 ] ASF subversion and git services commented on GEODE-124: --- Commit 60ebf92b7bd6600441ef5f8ee3b4bed29937c345 in incubator-geode's branch refs/heads/develop from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=60ebf92 ] GEODE-124: Merge 'feature/GEODE-124' into develop Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646772#comment-14646772 ] ASF subversion and git services commented on GEODE-124: --- Commit 1683361bc87368e48d8cfefebf6dbcda932ed90c in incubator-geode's branch refs/heads/develop from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=1683361 ] GEODE-124: Add min size check and optimize calls * Member stats are needed for computing member load skew and total data size. The gathering step is common and need not be executed twice * If the total transfer size during rebalance is low, then avoid rebalance. This could happen when a new cluster is created is data is being loaded Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646770#comment-14646770 ] ASF subversion and git services commented on GEODE-124: --- Commit a0ada7603d2447c7b70d791a8ae1fa75a4bd413f in incubator-geode's branch refs/heads/develop from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=a0ada76 ] GEODE-124: Address review comments * Fix spelling mistakes * Add more details in logs Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-169) Bug36853EventsExpiryDUnitTest.testEventsExpiryBug fails because of unexpected IOException
[ https://issues.apache.org/jira/browse/GEODE-169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646864#comment-14646864 ] ASF subversion and git services commented on GEODE-169: --- Commit 377beeb3a17637fb287c6b0bddef66af3050d22c in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=377beeb ] GEODE-169: testEventsExpiryBug should expect Unexpected IOException. Bug36853EventsExpiryDUnitTest.testEventsExpiryBug fails because of unexpected IOException - Key: GEODE-169 URL: https://issues.apache.org/jira/browse/GEODE-169 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Bug36853EventsExpiryDUnitTest.testEventsExpiryBug fails intermittently like so: java.lang.AssertionError: Suspicious strings were written to the log during this run. Fix the strings or use DistributedTestCase.addExpectedException to ignore. --- Found suspect string in log4j at line 647 [warn 2015/07/28 04:32:47.018 PDT ServerConnection on port 25082 Thread 1 tid=0x4cf1] Server connection from [identity(cc1-co(27992:loner):34963:63876fd4,connection=1; port=34963]: Unexpected IOException: The full stack of this exception is:[vm_0][warn 2015/07/28 04:32:47.018 PDT ServerConnection on port 25082 Thread 1 tid=0x4cf1] Server connection from [identity(cc1-co(27992:loner):34963:63876fd4,connection=1; port=34963]: Unexpected IOException: [vm_0]java.net.SocketException: Connection reset [vm_0]at java.net.SocketInputStream.read(SocketInputStream.java:196) [vm_0]at java.net.SocketInputStream.read(SocketInputStream.java:122) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.Message.fetchHeader(Message.java:699) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.Message.readHeaderAndPayload(Message.java:723) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.Message.read(Message.java:666) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.Message.recv(Message.java:1156) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.Message.recv(Message.java:1170) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.BaseCommand.readRequest(BaseCommand.java:939) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:741) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:914) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1159) [vm_0]at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [vm_0]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [vm_0]at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:580) [vm_0]at java.lang.Thread.run(Thread.java:745) The test will allow a Connection reset but in this case only sees: Unexpected IOException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646771#comment-14646771 ] ASF subversion and git services commented on GEODE-124: --- Commit d307eda9253e9d8929806f5421ef12059eed71eb in incubator-geode's branch refs/heads/develop from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=d307eda ] GEODE-124: Add logic to compute getTotalSize getTotalSize collects current size of a partitioned region on all members. The sum of these sizes is used to check if rebalance is needed. Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-95) PartitionedRegionSingleHopDUnitTest.test_MetadataContents fails intermittently
[ https://issues.apache.org/jira/browse/GEODE-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647604#comment-14647604 ] ASF subversion and git services commented on GEODE-95: -- Commit b460df43b1f2dbf06ad37f4175ca58ea49c450b3 in incubator-geode's branch refs/heads/develop from kbachhav [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b460df4 ] GEODE-95:PartitionedRegionSingleHopDUnitTest.test_MetadataContents fails intermittently There is a race between updating the metadata by client and creation of secondary bucket. Once, the client fetches the metadata without information of secondary bucket(due to race), any further put/get will go to the correct server which hosts the bucket for which client has information. This will not cause any re-fetching of metadata from server as there is no hop. Hence only making test changes. PartitionedRegionSingleHopDUnitTest.test_MetadataContents fails intermittently -- Key: GEODE-95 URL: https://issues.apache.org/jira/browse/GEODE-95 Project: Geode Issue Type: Bug Components: client/server Reporter: Dan Smith Assignee: Kishor Bachhav This test appears to have a race condition. It failed on of the nightly builds: https://builds.apache.org/view/All/job/Geode-nightly/lastBuild/testReport/junit/com.gemstone.gemfire.internal.cache/PartitionedRegionSingleHopDUnitTest/test_MetadataContents/ {noformat} junit.framework.AssertionFailedError: expected:2 but was:1 at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:234) at junit.framework.Assert.assertEquals(Assert.java:241) at junit.framework.TestCase.assertEquals(TestCase.java:409) at com.gemstone.gemfire.internal.cache.PartitionedRegionSingleHopDUnitTest.verifyMetadata(PartitionedRegionSingleHopDUnitTest.java:2221) at com.gemstone.gemfire.internal.cache.PartitionedRegionSingleHopDUnitTest.test_MetadataContents(PartitionedRegionSingleHopDUnitTest.java:386) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:252) at junit.framework.TestSuite.run(TestSuite.java:247) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50) at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106) at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at
[jira] [Commented] (GEODE-141) DistributedTransactionDUnitTest.testRemoveAllWithTransactions fails with CommitIncompleteException
[ https://issues.apache.org/jira/browse/GEODE-141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647511#comment-14647511 ] ASF subversion and git services commented on GEODE-141: --- Commit 64a9bd411058127571be61809e0e28d9ce174e2a in incubator-geode's branch refs/heads/develop from Vivek Bhaskar [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=64a9bd4 ] GEODE-172: Fixing an intermittent failure as reported by closing cache after each individual test. Also temporarily taking care of an assertion failure that is reflected in GEODE-110 and GEODE-141 that are still under investigation. DistributedTransactionDUnitTest.testRemoveAllWithTransactions fails with CommitIncompleteException -- Key: GEODE-141 URL: https://issues.apache.org/jira/browse/GEODE-141 Project: Geode Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: Kirk Lund Assignee: vivek bhaskar This test is failing intermittently in nightly builds. See #159. Complete stack is below: {code} dunit.RMIException: While invoking com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest$34.call in VM 0 running on Host asf911.gq1.ygridcore.net with 4 VMs at dunit.VM.invoke(VM.java:359) at dunit.VM.invoke(VM.java:303) at dunit.VM.invoke(VM.java:271) at com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest.execute(DistributedTransactionDUnitTest.java:118) at com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest.testRemoveAllWithTransactions(DistributedTransactionDUnitTest.java:1086) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:252) at junit.framework.TestSuite.run(TestSuite.java:247) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360) at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64) at
[jira] [Commented] (GEODE-110) DistributedTransactionDUnitTest.testPutAllWithTransactions failed with CommitIncompleteException
[ https://issues.apache.org/jira/browse/GEODE-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647510#comment-14647510 ] ASF subversion and git services commented on GEODE-110: --- Commit 64a9bd411058127571be61809e0e28d9ce174e2a in incubator-geode's branch refs/heads/develop from Vivek Bhaskar [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=64a9bd4 ] GEODE-172: Fixing an intermittent failure as reported by closing cache after each individual test. Also temporarily taking care of an assertion failure that is reflected in GEODE-110 and GEODE-141 that are still under investigation. DistributedTransactionDUnitTest.testPutAllWithTransactions failed with CommitIncompleteException Key: GEODE-110 URL: https://issues.apache.org/jira/browse/GEODE-110 Project: Geode Issue Type: Bug Reporter: Kirk Lund Assignee: vivek bhaskar {code} com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest testPutAllWithTransactions FAILED dunit.RMIException: While invoking com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest$33.call in VM 0 running on Host pietas.apache.org with 4 VMs at dunit.VM.invoke(VM.java:359) at dunit.VM.invoke(VM.java:303) at dunit.VM.invoke(VM.java:271) at com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest.execute(DistributedTransactionDUnitTest.java:118) at com.gemstone.gemfire.disttx.DistributedTransactionDUnitTest.testPutAllWithTransactions(DistributedTransactionDUnitTest.java:983) Caused by: com.gemstone.gemfire.cache.CommitIncompleteException: Incomplete commit of transaction TXId: pietas(30637)v362:53897:3. Caused by the following exceptions: From member: pietas(30663)v366:34083 com.gemstone.gemfire.distributed.internal.ReplyException: From pietas(30663)v366:34083: com.gemstone.gemfire.cache.UnsupportedOperationInTransactionException: Expected entry size of 1 for region /__PR/_B__customerPRRegion_3 during a distributed transaction but got 2 at com.gemstone.gemfire.internal.cache.TXMessage.process(TXMessage.java:114) at com.gemstone.gemfire.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:371) at com.gemstone.gemfire.distributed.internal.DistributionMessage.schedule(DistributionMessage.java:434) at com.gemstone.gemfire.distributed.internal.DistributionManager.scheduleIncomingMessage(DistributionManager.java:3862) at com.gemstone.gemfire.distributed.internal.DistributionManager.handleIncomingDMsg(DistributionManager.java:3486) at com.gemstone.gemfire.distributed.internal.DistributionManager$MyListener.messageReceived(DistributionManager.java:4683) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.processMessage(JGroupMembershipManager.java:2160) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager.handleOrDeferMessage(JGroupMembershipManager.java:2069) at com.gemstone.gemfire.distributed.internal.membership.jgroup.JGroupMembershipManager$MyDCReceiver.messageReceived(JGroupMembershipManager.java:668) at com.gemstone.gemfire.distributed.internal.direct.DirectChannel.receive(DirectChannel.java:804) at com.gemstone.gemfire.internal.tcp.TCPConduit.messageReceived(TCPConduit.java:835) at com.gemstone.gemfire.internal.tcp.Connection.dispatchMessage(Connection.java:3932) at com.gemstone.gemfire.internal.tcp.Connection.processNIOBuffer(Connection.java:3515) at com.gemstone.gemfire.internal.tcp.Connection.runNioReader(Connection.java:1827) at com.gemstone.gemfire.internal.tcp.Connection.run(Connection.java:1702) at java.lang.Thread.run(Thread.java:745) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-155) RegionWithHDFS*BasicDUnitTest.testWOTimeForRollOverParam failed due to incorrect number of files created.
[ https://issues.apache.org/jira/browse/GEODE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654882#comment-14654882 ] ASF subversion and git services commented on GEODE-155: --- Commit 8e4c70429f30d39c65330d9397325928780e9c56 in incubator-geode's branch refs/heads/develop from [~ashetkar] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=8e4c704 ] GEODE-155: - Fixes intermittent failure in RegionWithHDFS*DUnitTest about incorrect number of hoplog files getting created. - Caused due to an earlier test not cleaning up static variables in ParallelGatewaySenderQueue. - These are indeed cleaned up during GemFireCacheImpl.close() if there are any GatewaySenders running in the system. - But if a region with gateway-senders associated with it, is destroyed first before a cache.close(), then the gateway senders are only stopped, and removed from allGatewaySenders list. But the static variables are not cleaned up. - Later, during GemFireCacheImpl.close(), as the allGatewaySenders list is empty, it skips cleaning the static variables. - As a fix, invoking a static clean-up method of ParallelGatewaySenderQueue explicitly during cache.close(). - Retained the non-static ParallelGatewaySenderQueue.cleanUp(), if required in future development for performing instance-specific clean-up. - Minor formatting changes in the unit tests. RegionWithHDFS*BasicDUnitTest.testWOTimeForRollOverParam failed due to incorrect number of files created. - Key: GEODE-155 URL: https://issues.apache.org/jira/browse/GEODE-155 Project: Geode Issue Type: Bug Components: hdfs Affects Versions: 1.0.0-incubating Reporter: Amogh Shetkar Assignee: Amogh Shetkar The same test case in two distributed tests failed with the same error in Jenkins build #165. {code} com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSOffHeapBasicDUnitTest testWOTimeForRollOverParam FAILED junit.framework.AssertionFailedError: there should be exactly two files, but there are 1 com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSBasicDUnitTest testWOTimeForRollOverParam FAILED junit.framework.AssertionFailedError: there should be exactly two files, but there are 1 at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.assertTrue(Assert.java:22) at junit.framework.TestCase.assertTrue(TestCase.java:192) at com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSBasicDUnitTest.verifyTwoHDFSFiles(RegionWithHDFSBasicDUnitTest.java:1075) at com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSBasicDUnitTest.testWOTimeForRollOverParam(RegionWithHDFSBasicDUnitTest.java:1023) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-187) AsyncEventQueue with Event Filter test log shows IllegalStateException
[ https://issues.apache.org/jira/browse/GEODE-187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654951#comment-14654951 ] ASF subversion and git services commented on GEODE-187: --- Commit 01145b8e9eeec8d7fdf8e6fed500ff54021ab609 in incubator-geode's branch refs/heads/develop from kbachhav [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=01145b8 ] GEODE-187:AsyncEventQueue with Event Filter test log shows IllegalStateException -AbstractGatewaySenderEventProcessor#processqueue iterates events to get filtered. While iterating 2 consditions are checked and event is removed in thses both condition. for single itr.next() when itr.remove is called second time, it causes IllegalStateException. -Added 'continue' after first itr.remove is called AsyncEventQueue with Event Filter test log shows IllegalStateException Key: GEODE-187 URL: https://issues.apache.org/jira/browse/GEODE-187 Project: Geode Issue Type: Bug Reporter: Kishor Bachhav Assignee: Kishor Bachhav AbstractGatewaySenderEventProcessor#processqueue iterates events to get filtered. While iterating 2 consditions are checked and event is removed in thses both condition. for single itr.next() when itr.remove is called second time, it causes IllegalStateException. java.lang.IllegalStateException at java.util.ArrayList$Itr.remove(ArrayList.java:844) at com.gemstone.gemfire.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:509) We need to continue with other event if the event is already removed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-186) ClientServerTransactionDUnitTest.testSuspendTimeout takes too long to run
[ https://issues.apache.org/jira/browse/GEODE-186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659015#comment-14659015 ] ASF subversion and git services commented on GEODE-186: --- Commit ea9f03e778631f35366716a664e1ee6a714ef7b0 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=ea9f03e ] Fix GEODE-186 by removing sleeps in tests The old test scheduled tx suspension to timeout after 1 minute. So the test always run for at least 1 minute. A test hook now exists that allows the test to specify a different time unit (default is still minutes) for tx suspension expiration. The sleeps in a bunch of other tests were not needed since the tx operation is synchronous. So those sleeps have simply been removed. A couple of sleeps in clients waiting for something to arrive that was done on a server have been converted to a wait since server to client distribution is async. Reviewed by Swapnil. ClientServerTransactionDUnitTest.testSuspendTimeout takes too long to run - Key: GEODE-186 URL: https://issues.apache.org/jira/browse/GEODE-186 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor ClientServerTransactionDUnitTest.testSuspendTimeout takes too long to run. It takes 70 seconds. It has a 70 second sleep. It is waiting for a suspended tx to expire. It should use a test hook to change the expiration units from minutes to millis. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652520#comment-14652520 ] ASF subversion and git services commented on GEODE-124: --- Commit 0ff54f3e85f078e6c775e36192a5863b18c45f81 in incubator-geode's branch refs/heads/develop from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=0ff54f3 ] GEODE-124: Rename size-minimum to minimum-size Mimimum-size is easier to read and represents the property better. Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-030815.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-193) Enable subset of CopyOnReadIndexJUnitTests
[ https://issues.apache.org/jira/browse/GEODE-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14682107#comment-14682107 ] ASF subversion and git services commented on GEODE-193: --- Commit 96fd856db31f491c2d1977fbcf01b72b1ded25c0 in incubator-geode's branch refs/heads/develop from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=96fd856 ] GEODE-193: Enable subset of CopyOnReadIndexJUnitTests Enable subset of CopyOnReadIndexJUnitTests -- Key: GEODE-193 URL: https://issues.apache.org/jira/browse/GEODE-193 Project: Geode Issue Type: Improvement Components: querying Reporter: Jason Huynh Assignee: Jason Huynh Priority: Minor There are a subset of tests in CopyOnReadIndexJUnitTest that are disabled. They should be enabled and the number of objects created per test should be lowered. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-177) Merge Dockerfile into Geode repository
[ https://issues.apache.org/jira/browse/GEODE-177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660864#comment-14660864 ] ASF subversion and git services commented on GEODE-177: --- Commit fcb42ad52ba6b01c7fde18f0864946d0200f990d in incubator-geode's branch refs/heads/develop from William Markito [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=fcb42ad ] [GEODE-177] - Moving Dockerfile into Geode codebase Merge Dockerfile into Geode repository -- Key: GEODE-177 URL: https://issues.apache.org/jira/browse/GEODE-177 Project: Geode Issue Type: New Feature Components: extensions Reporter: William Markito Oliveira Assignee: William Markito Oliveira Priority: Minor With [GEODE-35] we created a Dockerfile and provisioned on DockerHub but due to [GEODE-8] the work stayed in a public GitHub repository. Now it's time to move the Dockerfile into the product and automate the image build on DockerHub, keeping the unstable/nightly tag for developers monitoring our *develop* branch. Until Geode has a release those images would still have a disclaimer about not being production ready and meant only for testing and development purposes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-181) FastLogger does not always honor Debug or Trace log levels when there is a MarkerFilter
[ https://issues.apache.org/jira/browse/GEODE-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654395#comment-14654395 ] ASF subversion and git services commented on GEODE-181: --- Commit 1c4cd8f7909ba20f3c321001d26424e6853fe6a9 in incubator-geode's branch refs/heads/develop from [~apa...@the9muses.net] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=1c4cd8f ] GEODE-181: Add new unit and integration tests for FastLogger Add new TestSuites for logging tests. Extract default config testing to FastLoggerWithDefaultConfigJUnitTest. Move FastLoggerJUnitTest integration tests to FastLoggerIntegrationJUnitTest. Create new unit tests in FastLoggerJUnitTest. Update tests to full JUnit 4 syntax and use Rules (GEODE-89). Add mockito dependency to build for unit testing. Completes: GEODE-89, GEODE-181, GEODE-10 FastLogger does not always honor Debug or Trace log levels when there is a MarkerFilter --- Key: GEODE-181 URL: https://issues.apache.org/jira/browse/GEODE-181 Project: Geode Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: Kirk Lund Assignee: Kirk Lund While working on GEODE-89, I wrote some new unit tests and found that FastLogger does not always honor Debug or Trace log levels when there is a MarkerFilter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-181) FastLogger does not always honor Debug or Trace log levels when there is a MarkerFilter
[ https://issues.apache.org/jira/browse/GEODE-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654398#comment-14654398 ] ASF subversion and git services commented on GEODE-181: --- Commit 1c4cd8f7909ba20f3c321001d26424e6853fe6a9 in incubator-geode's branch refs/heads/develop from [~apa...@the9muses.net] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=1c4cd8f ] GEODE-181: Add new unit and integration tests for FastLogger Add new TestSuites for logging tests. Extract default config testing to FastLoggerWithDefaultConfigJUnitTest. Move FastLoggerJUnitTest integration tests to FastLoggerIntegrationJUnitTest. Create new unit tests in FastLoggerJUnitTest. Update tests to full JUnit 4 syntax and use Rules (GEODE-89). Add mockito dependency to build for unit testing. Completes: GEODE-89, GEODE-181, GEODE-10 FastLogger does not always honor Debug or Trace log levels when there is a MarkerFilter --- Key: GEODE-181 URL: https://issues.apache.org/jira/browse/GEODE-181 Project: Geode Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: Kirk Lund Assignee: Kirk Lund While working on GEODE-89, I wrote some new unit tests and found that FastLogger does not always honor Debug or Trace log levels when there is a MarkerFilter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-137) Spark Connector: should connect to local GemFire server if possible
[ https://issues.apache.org/jira/browse/GEODE-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660366#comment-14660366 ] ASF subversion and git services commented on GEODE-137: --- Commit e596a4eecc3aa09a751843656c4e47c1b252282b in incubator-geode's branch refs/heads/feature/GEODE-137 from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=e596a4e ] GEODE-137: use local GemFire server to initialize ClientCache whenever possible. Spark Connector: should connect to local GemFire server if possible --- Key: GEODE-137 URL: https://issues.apache.org/jira/browse/GEODE-137 Project: Geode Issue Type: Bug Reporter: Qihong Chen Assignee: Qihong Chen Original Estimate: 96h Remaining Estimate: 96h DefaultGemFireConnection uses ClientCacheFactory with locator info to create ClientCache instance. In this case, the ClientCache doesn't connect to the GemFire/Geode server on the same host if there's one. This cause more network traffic and less efficient. ClientCacheFactory can create ClientCache based on GemFire server(s) info as well. Therefore, we can force the ClientCache connects to local GemFire server if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-188) Gfsh uses the wrong name for the gateway substitution filter in 'create async-event-queue' command
[ https://issues.apache.org/jira/browse/GEODE-188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660364#comment-14660364 ] ASF subversion and git services commented on GEODE-188: --- Commit e7e24267e8c3f840ed5e43ba09dc5f19cb0ef2ef in incubator-geode's branch refs/heads/feature/GEODE-137 from kbachhav [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=e7e2426 ] GEODE-188:wrong name for the gateway substitution filter in 'create async-event-queue' command -replaced string gateway-event-substitution-listener with gateway-event-substitution-filter and GatewayEventSubstitutionListener with GatewayEventSubstitutionFilter Ran: QueueCommandsDUnitTest Gfsh uses the wrong name for the gateway substitution filter in 'create async-event-queue' command -- Key: GEODE-188 URL: https://issues.apache.org/jira/browse/GEODE-188 Project: Geode Issue Type: Bug Reporter: Kishor Bachhav Assignee: Kishor Bachhav 'create async-event-queue' command uses gateway-event-substitution-listener and GatewayEventSubstitutionListener instead of gateway-event-substitution-filter and GatewayEventSubstitutionFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-154) HDFS region with HeapLRU reports LowMemoryExceptions even when the region size is less then the eviction threshold
[ https://issues.apache.org/jira/browse/GEODE-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660363#comment-14660363 ] ASF subversion and git services commented on GEODE-154: --- Commit 88ed3ed59b96f8ce84327ed2bad132d4dc6e3a51 in incubator-geode's branch refs/heads/feature/GEODE-137 from Namrata Thanvi [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=88ed3ed ] Fix GEODE-154: HDFS region with HeapLRU reports LowMemoryExceptions even when region size is less then the eviction threshold - calling setUsageThreshold() on MemoryPoolMXBean resulted in it reporting incorrect memory usage causing temporary spikes. Since we already set CollectionThreshold at 1 byte and also have a stat and heap poller thread, we now skip setting the usage threshold. HDFS region with HeapLRU reports LowMemoryExceptions even when the region size is less then the eviction threshold -- Key: GEODE-154 URL: https://issues.apache.org/jira/browse/GEODE-154 Project: Geode Issue Type: Bug Components: hdfs Affects Versions: 1.0.0-incubating Reporter: Namrata Assignee: Namrata Fix For: 1.0.0-incubating I discovered this while writing a test for GEODE-10. When a HDFS region with eviction is configured then it reports LowMemoryException even when we are not even close to the eviction threshold. With the stat It looks like that at the LowMemoryException was reported only with 20% of the heap usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-137) Spark Connector: should connect to local GemFire server if possible
[ https://issues.apache.org/jira/browse/GEODE-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660447#comment-14660447 ] ASF subversion and git services commented on GEODE-137: --- Commit 2e2a795db57e21cc83784f0b3111b0b5e355ad87 in incubator-geode's branch refs/heads/develop from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=2e2a795 ] GEODE-137: use local GemFire server to initialize LocalCache whenever possible. Spark Connector: should connect to local GemFire server if possible --- Key: GEODE-137 URL: https://issues.apache.org/jira/browse/GEODE-137 Project: Geode Issue Type: Bug Reporter: Qihong Chen Assignee: Qihong Chen Original Estimate: 96h Remaining Estimate: 96h DefaultGemFireConnection uses ClientCacheFactory with locator info to create ClientCache instance. In this case, the ClientCache doesn't connect to the GemFire/Geode server on the same host if there's one. This cause more network traffic and less efficient. ClientCacheFactory can create ClientCache based on GemFire server(s) info as well. Therefore, we can force the ClientCache connects to local GemFire server if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-137) Spark Connector: should connect to local GemFire server if possible
[ https://issues.apache.org/jira/browse/GEODE-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660448#comment-14660448 ] ASF subversion and git services commented on GEODE-137: --- Commit d0d48588c895cff7418090c50b57f9f4a740763c in incubator-geode's branch refs/heads/develop from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=d0d4858 ] GEODE-137: use local GemFire server to initialize ClientCache whenever possible. Spark Connector: should connect to local GemFire server if possible --- Key: GEODE-137 URL: https://issues.apache.org/jira/browse/GEODE-137 Project: Geode Issue Type: Bug Reporter: Qihong Chen Assignee: Qihong Chen Original Estimate: 96h Remaining Estimate: 96h DefaultGemFireConnection uses ClientCacheFactory with locator info to create ClientCache instance. In this case, the ClientCache doesn't connect to the GemFire/Geode server on the same host if there's one. This cause more network traffic and less efficient. ClientCacheFactory can create ClientCache based on GemFire server(s) info as well. Therefore, we can force the ClientCache connects to local GemFire server if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-146) Queries are not thread safe due to scopeId variable
[ https://issues.apache.org/jira/browse/GEODE-146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660567#comment-14660567 ] ASF subversion and git services commented on GEODE-146: --- Commit 1a6a0ef53acfdc3931f48881b4b90907c16dc972 in incubator-geode's branch refs/heads/develop from Vito Gavrilov [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=1a6a0ef ] Fix for GEODE-109 1) To fix this floating meta data problem for the redis list implementation, this meta data has been moved into the list region itself 2) Sorted set query failures are fixed by using fully qualified names and also depends on GEODE-146 3) Concurrent region creation/destruction logic completely changed to use dedicated locks for synchronization, fixed distributed deadlock 4) Added integration tests 5) Add javadocs where necessary This closes #13 Queries are not thread safe due to scopeId variable --- Key: GEODE-146 URL: https://issues.apache.org/jira/browse/GEODE-146 Project: Geode Issue Type: Bug Components: querying Reporter: Jason Huynh Assignee: Jason Huynh Currently queries are not thread safe although they should be. This is due to the way the scopeId is stored in the CompiledSelect. The fix consists of storing the scopeId in the execution context. public void testThreadSafetyOfCompiledSelectScopeId() throws Exception { Cache cache = CacheUtils.getCache(); RegionFactoryInteger, Portfolio rf = cache .createRegionFactory(RegionShortcut.PARTITION); Region r = rf.create(keyzset); for (int i = 0; i 100; i++) { r.put(i, new Portfolio(i)); } ScopeThreadingTestHook scopeIDTestHook = new ScopeThreadingTestHook(3); DefaultQuery.testHook = scopeIDTestHook; QueryService qs = cache.getQueryService(); Query q = qs .newQuery(SELECT DISTINCT * FROM /keyzset.keySet key WHERE key.id 0 AND key.id = 0 ORDER BY key asc LIMIT $3); Thread q1 = new Thread(new QueryRunnable(q, new Object[] { 10, 20, 10 })); Thread q2 = new Thread(new QueryRunnable(q, new Object[] { 5, 10, 5 })); Thread q3 = new Thread(new QueryRunnable(q, new Object[] { 2, 10, 8 })); q1.start(); q2.start(); q3.start(); q1.join(); q2.join(); q3.join(); assertEquals(Exceptions were thrown due to DefaultQuery not being thread-safe, true, scopeIDTestHook.isOk()); } private class QueryRunnable implements Runnable { private Query q; private Object[] params; public QueryRunnable(Query q, Object[] params) { this.q = q; this.params = params; } public void run() { try { q.execute(params); } catch (Exception e) { throw new TestException(exception occured while executing query, e); } } } public class ScopeThreadingTestHook implements DefaultQuery.TestHook { private CyclicBarrier barrier; private ListException exceptionsThrown = new LinkedListException(); public ScopeThreadingTestHook(int numThreads) { barrier = new CyclicBarrier(numThreads); } @Override public void doTestHook(int spot) { this.doTestHook(spot + ); } @Override public void doTestHook(String spot) { if (spot.equals(1)) { try { barrier.await(8, TimeUnit.SECONDS); } catch (InterruptedException e) { exceptionsThrown.add(e); Thread.currentThread().interrupt(); } catch (Exception e) { exceptionsThrown.add(e); } } } public boolean isOk() { return exceptionsThrown.size() == 0; } } -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-109) Bugs in redis adapter when running with multiple vms
[ https://issues.apache.org/jira/browse/GEODE-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660566#comment-14660566 ] ASF subversion and git services commented on GEODE-109: --- Commit 1a6a0ef53acfdc3931f48881b4b90907c16dc972 in incubator-geode's branch refs/heads/develop from Vito Gavrilov [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=1a6a0ef ] Fix for GEODE-109 1) To fix this floating meta data problem for the redis list implementation, this meta data has been moved into the list region itself 2) Sorted set query failures are fixed by using fully qualified names and also depends on GEODE-146 3) Concurrent region creation/destruction logic completely changed to use dedicated locks for synchronization, fixed distributed deadlock 4) Added integration tests 5) Add javadocs where necessary This closes #13 Bugs in redis adapter when running with multiple vms Key: GEODE-109 URL: https://issues.apache.org/jira/browse/GEODE-109 Project: Geode Issue Type: Bug Components: core Affects Versions: 1.0.0-incubating Reporter: Vitaliy Gavrilov 1) The meta data attached with the redis lists implementation does not always stay synchronized with the list. 2) Queries run by sorted set requests fail in execution 3) Potential deadlock when regions are created simultaneously in different vms -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-190) ReflectionBasedAutoSerializer leaks heap memory every time the Geode cache is closed
[ https://issues.apache.org/jira/browse/GEODE-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660631#comment-14660631 ] ASF subversion and git services commented on GEODE-190: --- Commit 6cb4a5682667eacc0017898b81367c3e5029c888 in incubator-geode's branch refs/heads/feature/GEODE-190 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=6cb4a56 ] GEODE-190: Fix ReflectionBasedAutoSerializer leak A weak map value references its key keeping the key from ever being garbage. The only reason the weak map was added was to allow internal code to get the AutoSerializableManager given a ReflectionBasedAutoSerializer. But the map was not really needed since the ReflectionBasedAutoSerializer has a final field named manager. The only problem is that ReflectionBasedAutoSerializer is a public api and AutoSerializableManager is internal. This fix adds a getManager method on ReflectionBasedAutoSerializer that returns object and is javadoc'd for internal use only. The internal code casts the result to AutoSerializableManager. This allows the weak map that was causing the memory leak to be removed. ReflectionBasedAutoSerializer leaks heap memory every time the Geode cache is closed Key: GEODE-190 URL: https://issues.apache.org/jira/browse/GEODE-190 Project: Geode Issue Type: Bug Components: serialization Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor If the ReflectionBasedAutoSerializer then every time the Geode cache is closed some of the object that should become garbage do not. They are strongly referenced. If an app repeatedly closes and recreates the cache then this will eventually cause an OutOfMemoryException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-196) Remove unneeded system out in RuntimeIterator
[ https://issues.apache.org/jira/browse/GEODE-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695715#comment-14695715 ] ASF subversion and git services commented on GEODE-196: --- Commit 1eccf917198b0d2ce1e3f350fbf7317d322f2eed in incubator-geode's branch refs/heads/develop from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=1eccf91 ] GEODE-196: Remove unneeded system out in RuntimeIterator Remove unneeded system out in RuntimeIterator - Key: GEODE-196 URL: https://issues.apache.org/jira/browse/GEODE-196 Project: Geode Issue Type: Bug Reporter: Jason Huynh Assignee: Jason Huynh Priority: Minor Remove the system out and if block in RuntimeIterator. The lines are: if(current == UNINITIALIZED) { System.out.println(asif); } -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695651#comment-14695651 ] ASF subversion and git services commented on GEODE-77: -- Commit 0a70d5140277b366be4d0b9607a5e68936837c2f in incubator-geode's branch refs/heads/feature/GEODE-77 from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=0a70d51 ] GEODE-77: Implement Authenticator interface in class GMSAuthenticator with unit tests. Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-212) ResourceManager implementation impacts off-heap performance
[ https://issues.apache.org/jira/browse/GEODE-212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696088#comment-14696088 ] ASF subversion and git services commented on GEODE-212: --- Commit 587f5896a893d6512b1ae7b0fd6892e451ca0265 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=587f589 ] GEODE-212: improve off-heap + ResourceManager performance The synchronous code that notifies the async listener now does a quick check to see if an event might be needed. This is done with no synchronization. It just does two volatile reads. The only time the synchronized notify is now needed is if we might have crossed a threshold boundary or if the listener indicates it really wants an event. The old code called System.currentTimeMillis everytime it created a MemoryEvent. This timestamp was only used by off-heap to reduce how many notifications were done when the state is not normal. Heap does notifications in abnormal states every time the amount of memory changes. Off-heap does it when abnormal at most every second by using the timestamp. To get rid of all these currentTimeMillis calls the new code has the background OffHeapMemoryUsageListener times out its wait call every second. If it times out it will set a volatile that indicates it wants the next abnormal event. ResourceManager implementation impacts off-heap performance --- Key: GEODE-212 URL: https://issues.apache.org/jira/browse/GEODE-212 Project: Geode Issue Type: Bug Components: offheap Reporter: Darrel Schneider Assignee: Darrel Schneider Original Estimate: 24h Remaining Estimate: 24h When the resource manager is configured with either a eviction or critical threshold then it registers a listener with the off-heap SimpleMemoryAllocator that causes every off-heap allocate and free to call this listener. The current listener does a synchronize on a singleton and notifies it. This will impact the rate at which multiple threads can allocate and free off-heap objects. I simple test using 8 threads showed that without the resource manager they could do 16 million allocate/frees in 3 seconds. With the resource manager it took 12 seconds. The notify is done to a singleton thread that simple checks to see if the new off-heap memory size would result in a change to the resource manager state. This can be done without synchronizing. Only if a change is possible do we need to do the sync and notify. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-216) Failure to process GetMessage due RegionDestroyedException
[ https://issues.apache.org/jira/browse/GEODE-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696165#comment-14696165 ] ASF subversion and git services commented on GEODE-216: --- Commit 41d04928be3a455b02e58432713ed694f5f2b238 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=41d0492 ] GEODE-216: Handle any exception that might be thrown The old code was trying to prevent a deadlock under some certain conditions. But if it can not get a PR because of an exception then a deadlock is not possible. So the new code just catches and ignores exceptions and falls through to the code that will process the message the old way. This code will encounter the same exception and has the proper logic to handle it. Failure to process GetMessage due RegionDestroyedException -- Key: GEODE-216 URL: https://issues.apache.org/jira/browse/GEODE-216 Project: Geode Issue Type: Bug Reporter: Dan Smith Assignee: Darrel Schneider This failure showed up in a nightly build, the test RedisDistDUnitTest.testConcCreateDestroy timed out on a read. But to me it looks like the underlying issue is some missing exception handling code in GetMessage that caused a hang due to a RegionDestroyedException. https://builds.apache.org/job/Geode-nightly/183/testReport/junit/com.gemstone.gemfire.redis/RedisDistDUnitTest/testConcCreateDestroy/ {noformat} dunit.RMIException: While invoking com.gemstone.gemfire.redis.RedisDistDUnitTest$1ConcCreateDestroy.call in VM 3 running on Host jenkins-ubuntu-1404-4gb-f9a with 4 VMs at dunit.VM.invoke(VM.java:359) at dunit.VM.invoke(VM.java:303) at dunit.VM.invoke(VM.java:271) at com.gemstone.gemfire.redis.RedisDistDUnitTest.testConcCreateDestroy(RedisDistDUnitTest.java:168) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:252) at junit.framework.TestSuite.run(TestSuite.java:247) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696169#comment-14696169 ] ASF subversion and git services commented on GEODE-77: -- Commit ab68f4e683ac784b2602adb3d57323d0ace75db0 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~bschuchardt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=ab68f4e ] GEODE-77 fixing the inability to create multiple MembershipManagers ina single JVM The static MemberAttributes.DEFAULT was being used to pull together all of the bits of a new members identity (InternalDistributedMember) but this prevented multiple MembershipManagers from being instantiated w/o ones identity bleeding into anothers. Now the MembershipManager is the assembling a single InternalDistributedMember instance, putting info into it as it becomes available. A new unit test ensures that this is functioning correctly. Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-179) DistributedNoAckRegionCCEOffHeapDUnitTest.testEntryTtlDestroyEvent intermittent assertion falure
[ https://issues.apache.org/jira/browse/GEODE-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680769#comment-14680769 ] ASF subversion and git services commented on GEODE-179: --- Commit e255c75e9be33e3d274c42822923ba3db0629739 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=e255c75 ] GEODE-179: fix race in testEntryTtlDestroyEvent A race existed that caused no-ack regions to fail because they did not see the entry expire. This race was fixed by suspending expiration until the create is seen on the remote side and then after permitting expiration using a wait for the expire to happen. DistributedNoAckRegionCCEOffHeapDUnitTest.testEntryTtlDestroyEvent intermittent assertion falure Key: GEODE-179 URL: https://issues.apache.org/jira/browse/GEODE-179 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor See: https://builds.apache.org/view/All/job/Geode-nightly/172/testReport/com.gemstone.gemfire.cache30/DistributedNoAckRegionCCEOffHeapDUnitTest/testEntryTtlDestroyEvent/ dunit.RMIException: While invoking com.gemstone.gemfire.cache30.MultiVMRegionTestCase$183.run in VM 1 running on Host asf903.gq1.ygridcore.net with 4 VMs at dunit.VM.invoke(VM.java:359) at dunit.VM.invoke(VM.java:303) at dunit.VM.invoke(VM.java:257) at com.gemstone.gemfire.cache30.MultiVMRegionTestCase.testEntryTtlDestroyEvent(MultiVMRegionTestCase.java:3927) Caused by: junit.framework.AssertionFailedError: Expected: null but was: NonTXEntry@655cb9ec VersionedThinRegionEntryOffHeapStringKey1@2ab268f1 (key=KEY; rawValue=SimpleMemoryAllocatorImpl$GemFireChunk@-392327287:dataSize=8 refCount=1 addr=7f81e89df008; version={v1; rv1; time=1438231062905};member=null) at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.assertTrue(Assert.java:22) at junit.framework.Assert.assertNull(Assert.java:277) at junit.framework.Assert.assertNull(Assert.java:268) at junit.framework.TestCase.assertNull(TestCase.java:438) at com.gemstone.gemfire.cache30.MultiVMRegionTestCase$183.run2(MultiVMRegionTestCase.java:3934) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-138) MemoryThresholdsOffHeapDUnitTest.testEventDelivery failed
[ https://issues.apache.org/jira/browse/GEODE-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680949#comment-14680949 ] ASF subversion and git services commented on GEODE-138: --- Commit bd84581e3d5a93cbbb3fcdf06ebe60bb9460e702 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=bd84581 ] GEODE-138: remove race condition from testEventDelivery The race due to the test expecting local event delivery to be synchronous but the implementation of off-heap resource manager events being async. The test has been changed to always use a wait criteria when waiting for a memory event to arrive. MemoryThresholdsOffHeapDUnitTest.testEventDelivery failed -- Key: GEODE-138 URL: https://issues.apache.org/jira/browse/GEODE-138 Project: Geode Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: Kirk Lund Assignee: Darrel Schneider Priority: Minor This end-to-end test failed in Jenkins in build #159. Need to tighten up any race conditions and make this test more reliable. {code} dunit.RMIException: While invoking com.gemstone.gemfire.cache.management.MemoryThresholdsOffHeapDUnitTest$46.call in VM 1 running on Host asf911.gq1.ygridcore.net with 4 VMs at dunit.VM.invoke(VM.java:359) at dunit.VM.invoke(VM.java:303) at dunit.VM.invoke(VM.java:271) at com.gemstone.gemfire.cache.management.MemoryThresholdsOffHeapDUnitTest.verifyListenerValue(MemoryThresholdsOffHeapDUnitTest.java:1568) at com.gemstone.gemfire.cache.management.MemoryThresholdsOffHeapDUnitTest.testEventDelivery(MemoryThresholdsOffHeapDUnitTest.java:149) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:252) at junit.framework.TestSuite.run(TestSuite.java:247) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360) at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
[jira] [Commented] (GEODE-191) region.size() operation sometimes returns incorrect results
[ https://issues.apache.org/jira/browse/GEODE-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14680565#comment-14680565 ] ASF subversion and git services commented on GEODE-191: --- Commit b1e7466cb64fd6413c4658026498e5e6f4b8a7c8 in incubator-geode's branch refs/heads/develop from [~sbawaska] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b1e7466 ] GEODE-191 fix for transactional region.size() In the merge from sga2 branch, entryCount() no longer consulted the dataView() which led to getting wrong size when a transaction is hosted on a remote member. region.size() operation sometimes returns incorrect results --- Key: GEODE-191 URL: https://issues.apache.org/jira/browse/GEODE-191 Project: Geode Issue Type: Bug Components: transactions Reporter: Swapnil Bawaskar Attachments: testForSize.txt In case where a transaction is hosted on a remote member, doing a region.size() operation on a partitioned region returns an incorrect size. The operation returns 0 for size of all the buckets hosted locally. This seems to have been introduced in the merge from sga2 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-214) Azul Zing support
[ https://issues.apache.org/jira/browse/GEODE-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697501#comment-14697501 ] ASF subversion and git services commented on GEODE-214: --- Commit eb7e7b778d8b64741d1e377022c620feaa6483ca in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=eb7e7b7 ] GEODE-214: improve Azul support - isTenured now looks for GenPauseless Old Gen for azul. - Azul jvm version no longer logged as unsupported. - Fatal log message about jvm version not being supported is now a warning since we continue to run. - Fixed a bug in ReflectionSingleObjectSizer in how it calculated the size of a field. It was calling Field.getClass instead of Field.getType. Field.getClass always returns an instance of Field.class which the sizer always says is an objref size. getType will return a primitive class for primitive fields which is what we want. - Improved ObjectSizerJUnitTest. - The object header size and reference size are now correctly computed for azul. - Added some java 8 support to the ObjectSizer. - Fix tests that are unintentionally spawning processes that use the default mcast-port. This fixes intermittent failures caused by finding another member of the wrong GemFire version. Azul Zing support - Key: GEODE-214 URL: https://issues.apache.org/jira/browse/GEODE-214 Project: Geode Issue Type: Improvement Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 48h Remaining Estimate: 48h Some issues with running Geode on the Azul Zing jvm have been identified. They are: - the Resource Manager does not know the name of azul's tenured memory pool - geode does not correctly compute the size of java objects on azul -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-214) Azul Zing support
[ https://issues.apache.org/jira/browse/GEODE-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697711#comment-14697711 ] ASF subversion and git services commented on GEODE-214: --- Commit eb7e7b778d8b64741d1e377022c620feaa6483ca in incubator-geode's branch refs/heads/feature/GEODE-77 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=eb7e7b7 ] GEODE-214: improve Azul support - isTenured now looks for GenPauseless Old Gen for azul. - Azul jvm version no longer logged as unsupported. - Fatal log message about jvm version not being supported is now a warning since we continue to run. - Fixed a bug in ReflectionSingleObjectSizer in how it calculated the size of a field. It was calling Field.getClass instead of Field.getType. Field.getClass always returns an instance of Field.class which the sizer always says is an objref size. getType will return a primitive class for primitive fields which is what we want. - Improved ObjectSizerJUnitTest. - The object header size and reference size are now correctly computed for azul. - Added some java 8 support to the ObjectSizer. - Fix tests that are unintentionally spawning processes that use the default mcast-port. This fixes intermittent failures caused by finding another member of the wrong GemFire version. Azul Zing support - Key: GEODE-214 URL: https://issues.apache.org/jira/browse/GEODE-214 Project: Geode Issue Type: Improvement Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 48h Remaining Estimate: 48h Some issues with running Geode on the Azul Zing jvm have been identified. They are: - the Resource Manager does not know the name of azul's tenured memory pool - geode does not correctly compute the size of java objects on azul -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-209) FailWithTimeoutOfWaitForOutputToMatchJUnitTest.testFailWithTimeoutOfWaitForOutputToMatch is not stable
[ https://issues.apache.org/jira/browse/GEODE-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697710#comment-14697710 ] ASF subversion and git services commented on GEODE-209: --- Commit 29145676960c7d75f1da04cf618f71f6a399eda7 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~apa...@the9muses.net] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=2914567 ] GEODE-209: Change subTearDown to destroy process FailWithTimeoutOfWaitForOutputToMatchJUnitTest.testFailWithTimeoutOfWaitForOutputToMatch is not stable -- Key: GEODE-209 URL: https://issues.apache.org/jira/browse/GEODE-209 Project: Geode Issue Type: Bug Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Kirk Lund Priority: Minor Fix For: 1.0.0-incubating FailWithTimeoutOfWaitForOutputToMatchJUnitTest.testFailWithTimeoutOfWaitForOutputToMatch failed 5 times in the last 30 runs. It is failing in the tearDown phase. {quote} java.lang.AssertionError at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertFalse(Assert.java:64) at org.junit.Assert.assertFalse(Assert.java:74) at com.gemstone.gemfire.test.golden.FailWithTimeoutOfWaitForOutputToMatchJUnitTest.subTearDown(FailWithTimeoutOfWaitForOutputToMatchJUnitTest.java:20) at com.gemstone.gemfire.test.golden.GoldenTestCase.tearDownGoldenTest(GoldenTestCase.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) {quote} {code} public void subTearDown() throws Exception { this.process.waitFor(); assertFalse(this.process.isAlive()); } {code} Fix to GEODE-127 may have caused this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-216) Failure to process GetMessage due RegionDestroyedException
[ https://issues.apache.org/jira/browse/GEODE-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697709#comment-14697709 ] ASF subversion and git services commented on GEODE-216: --- Commit 41d04928be3a455b02e58432713ed694f5f2b238 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=41d0492 ] GEODE-216: Handle any exception that might be thrown The old code was trying to prevent a deadlock under some certain conditions. But if it can not get a PR because of an exception then a deadlock is not possible. So the new code just catches and ignores exceptions and falls through to the code that will process the message the old way. This code will encounter the same exception and has the proper logic to handle it. Failure to process GetMessage due RegionDestroyedException -- Key: GEODE-216 URL: https://issues.apache.org/jira/browse/GEODE-216 Project: Geode Issue Type: Bug Components: core Reporter: Dan Smith Assignee: Darrel Schneider This failure showed up in a nightly build, the test RedisDistDUnitTest.testConcCreateDestroy timed out on a read. But to me it looks like the underlying issue is some missing exception handling code in GetMessage that caused a hang due to a RegionDestroyedException. https://builds.apache.org/job/Geode-nightly/183/testReport/junit/com.gemstone.gemfire.redis/RedisDistDUnitTest/testConcCreateDestroy/ {noformat} dunit.RMIException: While invoking com.gemstone.gemfire.redis.RedisDistDUnitTest$1ConcCreateDestroy.call in VM 3 running on Host jenkins-ubuntu-1404-4gb-f9a with 4 VMs at dunit.VM.invoke(VM.java:359) at dunit.VM.invoke(VM.java:303) at dunit.VM.invoke(VM.java:271) at com.gemstone.gemfire.redis.RedisDistDUnitTest.testConcCreateDestroy(RedisDistDUnitTest.java:168) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:252) at junit.framework.TestSuite.run(TestSuite.java:247) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at
[jira] [Commented] (GEODE-212) ResourceManager implementation impacts off-heap performance
[ https://issues.apache.org/jira/browse/GEODE-212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697708#comment-14697708 ] ASF subversion and git services commented on GEODE-212: --- Commit 587f5896a893d6512b1ae7b0fd6892e451ca0265 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=587f589 ] GEODE-212: improve off-heap + ResourceManager performance The synchronous code that notifies the async listener now does a quick check to see if an event might be needed. This is done with no synchronization. It just does two volatile reads. The only time the synchronized notify is now needed is if we might have crossed a threshold boundary or if the listener indicates it really wants an event. The old code called System.currentTimeMillis everytime it created a MemoryEvent. This timestamp was only used by off-heap to reduce how many notifications were done when the state is not normal. Heap does notifications in abnormal states every time the amount of memory changes. Off-heap does it when abnormal at most every second by using the timestamp. To get rid of all these currentTimeMillis calls the new code has the background OffHeapMemoryUsageListener times out its wait call every second. If it times out it will set a volatile that indicates it wants the next abnormal event. ResourceManager implementation impacts off-heap performance --- Key: GEODE-212 URL: https://issues.apache.org/jira/browse/GEODE-212 Project: Geode Issue Type: Bug Components: offheap Reporter: Darrel Schneider Assignee: Darrel Schneider Original Estimate: 24h Remaining Estimate: 24h When the resource manager is configured with either a eviction or critical threshold then it registers a listener with the off-heap SimpleMemoryAllocator that causes every off-heap allocate and free to call this listener. The current listener does a synchronize on a singleton and notifies it. This will impact the rate at which multiple threads can allocate and free off-heap objects. I simple test using 8 threads showed that without the resource manager they could do 16 million allocate/frees in 3 seconds. With the resource manager it took 12 seconds. The notify is done to a singleton thread that simple checks to see if the new off-heap memory size would result in a change to the resource manager state. This can be done without synchronizing. Only if a change is possible do we need to do the sync and notify. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-220) DistributedNoAckRegionOffHeapDUnitTest.testRemoteCacheWriter fails intermittently
[ https://issues.apache.org/jira/browse/GEODE-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697775#comment-14697775 ] ASF subversion and git services commented on GEODE-220: --- Commit 8eb6cef8a3428d77d6e8661fda6b2a9292f7f367 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=8eb6cef ] GEODE-220: Use wait after off-heap destroyRegion DistributedNoAckRegionOffHeapDUnitTest.testRemoteCacheWriter fails intermittently - Key: GEODE-220 URL: https://issues.apache.org/jira/browse/GEODE-220 Project: Geode Issue Type: Bug Components: tests Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 2h Remaining Estimate: 2h DistributedNoAckRegionOffHeapDUnitTest.testRemoteCacheWriter fails intermittently. The intermittent failure is caused by an off-heap region being destroyed and the test expecting the off-heap stats to be synchronously updated during the destroy. off-heap regions release their off-heap references async in a background thread to prevent dead lock. The following assertEquals(0, ...) needs be in a wait criteria: {code} region.destroyRegion(arg); if (region.getAttributes().getOffHeap() !(region instanceof PartitionedRegion)) { GemFireCacheImpl gfc = (GemFireCacheImpl) getCache(); SimpleMemoryAllocatorImpl ma = (SimpleMemoryAllocatorImpl) gfc.getOffHeapStore(); assertEquals(0, ma.getStats().getObjects()); } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-77) Replace JGroups 2.2.9
[ https://issues.apache.org/jira/browse/GEODE-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697712#comment-14697712 ] ASF subversion and git services commented on GEODE-77: -- Commit e0d1c4f908b30bdf239041403bf9c6369f2be22b in incubator-geode's branch refs/heads/feature/GEODE-77 from [~bschuchardt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=e0d1c4f ] Merge remote-tracking branch 'origin/develop' into feature/GEODE-77 Replace JGroups 2.2.9 - Key: GEODE-77 URL: https://issues.apache.org/jira/browse/GEODE-77 Project: Geode Issue Type: Bug Reporter: Bruce Schuchardt Assignee: Bruce Schuchardt Priority: Blocker Fix For: 1.0.0-incubating Attachments: GEODE-MembershipManagerFunctionalSpecification-130715-1604-29054.pdf The JGroups 2.2.9 sources that are currently included in Geode must be replaced in order for Geode to leave incubation. A wiki document has been created to investigate alternatives. https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-211) ConcurrentRegionOperationsJUnitTest takes too long to run
[ https://issues.apache.org/jira/browse/GEODE-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693913#comment-14693913 ] ASF subversion and git services commented on GEODE-211: --- Commit 9de95d63275c5e5a13b35d5bb7744c399916d7f5 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=9de95d6 ] GEODE-211: shorten the time ConcurrentRegionOperationsJUnitTest runs ConcurrentRegionOperationsJUnitTest takes too long to run - Key: GEODE-211 URL: https://issues.apache.org/jira/browse/GEODE-211 Project: Geode Issue Type: Bug Components: tests Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 1h Remaining Estimate: 1h ConcurrentRegionOperationsJUnitTest takes over 200 seconds to run. A couple of simple changes can cut this time in half. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-176) Validating offheap orphan on bucket region in unit test may hit RegionDestroyedException
[ https://issues.apache.org/jira/browse/GEODE-176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654392#comment-14654392 ] ASF subversion and git services commented on GEODE-176: --- Commit 3e3acef6de451966fb4180b458b6d7432ab5687d in incubator-geode's branch refs/heads/feature/GEODE-181 from [~eshu] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=3e3acef ] [GEODE-176] fixes the issue Validating offheap orphan on bucket region in unit test may hit RegionDestroyedException Key: GEODE-176 URL: https://issues.apache.org/jira/browse/GEODE-176 Project: Geode Issue Type: Bug Components: core Reporter: Eric Shu Assignee: Eric Shu Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-10) HDFS Integration
[ https://issues.apache.org/jira/browse/GEODE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637104#comment-14637104 ] ASF subversion and git services commented on GEODE-10: -- Commit b3e216740d75223252c95c0a016828d63cc13398 in incubator-geode's branch refs/heads/feature/GEODE-119 from [~ashetkar] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b3e2167 ] - GEODE-119: Re-introducing the distributed unit tests that were excluded in the initial code drop to Geode (GEODE-10). - The failures were mostly related to the change in dunit-results directory structure. - Also, did some code formatting and clean up. HDFS Integration Key: GEODE-10 URL: https://issues.apache.org/jira/browse/GEODE-10 Project: Geode Issue Type: New Feature Components: hdfs Reporter: Dan Smith Assignee: Ashvin Attachments: GEODE-HDFSPersistence-Draft-060715-2109-21516.pdf Ability to persist data on HDFS had been under development for GemFire. It was part of the latest code drop, GEODE-8. As part of this feature we are proposing some changes to the HdfsStore management API (see attached doc for details). # The current API has nested configuration for compaction and async queue. This nested structure forces user to execute multiple steps to manage a store. It also does not seem to be consistent with other management APIs # Some member names in current API are confusing HDFS Integration: Geode as a transactional layer that microbatches data out to Hadoop. This capability makes Geode a NoSQL store that can sit on top of Hadoop and parallelize the process of moving data from the in memory tier into Hadoop, making it very useful for capturing and processing fast data while making it available for Hadoop jobs relatively quickly. The key requirements being met here are # Ingest data into HDFS parallely # Cache bloom filters and allow fast lookups of individual elements # Have programmable policies for deciding what stays in memory # Roll files in HDFS # Index data that is in memory # Have expiration policies that allows the transactional set to decay out older data # Solution needs to support replicated and partitioned regions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-156) unit tests that call pauseIfNecessary will intermittently fail
[ https://issues.apache.org/jira/browse/GEODE-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643017#comment-14643017 ] ASF subversion and git services commented on GEODE-156: --- Commit 73d0dcd2ae7c4d3ffcd490472d88faf3ec264885 in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=73d0dcd ] GEODE-156: fix unit tests that call pauseIfNecessary all calls to pauseIfNecessary have been changed to flushIfNecessary. The new flushIfNecessary method uses StateFlushOperation. unit tests that call pauseIfNecessary will intermittently fail -- Key: GEODE-156 URL: https://issues.apache.org/jira/browse/GEODE-156 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Many of the unit tests in MultiVMRegionTestCase call pauseIfNecessary. They do this to allow previous operations done on a no-ack region to complete. But since the pause is simply a sleep it is possible for these tests to not sleep long enough and still fail. They may also sleep longer than is needed. We have a StateFlushOperation that can be used by the sender to wait until all the no-ack ops it did on a particular region have completed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643239#comment-14643239 ] ASF subversion and git services commented on GEODE-124: --- Commit d307eda9253e9d8929806f5421ef12059eed71eb in incubator-geode's branch refs/heads/feature/GEODE-124 from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=d307eda ] GEODE-124: Add logic to compute getTotalSize getTotalSize collects current size of a partitioned region on all members. The sum of these sizes is used to check if rebalance is needed. Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-83) Support AUTH for Redis protocol adaptor
[ https://issues.apache.org/jira/browse/GEODE-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643173#comment-14643173 ] ASF subversion and git services commented on GEODE-83: -- Commit 4af9bb3815f59a71605f212c27269a5eeb4a4f7d in incubator-geode's branch refs/heads/develop from [~sbawaska] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=4af9bb3 ] Merge branch 'feature/GEODE-83' into develop close #10. Support AUTH for Redis protocol adaptor --- Key: GEODE-83 URL: https://issues.apache.org/jira/browse/GEODE-83 Project: Geode Issue Type: Improvement Components: extensions Reporter: Swapnil Bawaskar The Redis protocol supports basic authentication through the AUTH command: http://redis.io/topics/security Rather than storing the password in redis.conf, we should ask the users to store it in gfsecurity.conf file as a redis-password property. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-83) Support AUTH for Redis protocol adaptor
[ https://issues.apache.org/jira/browse/GEODE-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643171#comment-14643171 ] ASF subversion and git services commented on GEODE-83: -- Commit 111d61546f2958cf4ad36872677b95ca50269dfd in incubator-geode's branch refs/heads/develop from Vito Gavrilov [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=111d615 ] GEODE-83 Add support for Redis AUTH command Also added Geode property redis-password so the authentication password can be set by xml or through GFSH Support AUTH for Redis protocol adaptor --- Key: GEODE-83 URL: https://issues.apache.org/jira/browse/GEODE-83 Project: Geode Issue Type: Improvement Components: extensions Reporter: Swapnil Bawaskar The Redis protocol supports basic authentication through the AUTH command: http://redis.io/topics/security Rather than storing the password in redis.conf, we should ask the users to store it in gfsecurity.conf file as a redis-password property. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-9) Spark Integration
[ https://issues.apache.org/jira/browse/GEODE-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643461#comment-14643461 ] ASF subversion and git services commented on GEODE-9: - Commit b387e3587c1a8f5566017d322f4ff606b9bc5e21 in incubator-geode's branch refs/heads/develop from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b387e35 ] Merge branch 'feature/GEODE-9' into develop Spark Integration - Key: GEODE-9 URL: https://issues.apache.org/jira/browse/GEODE-9 Project: Geode Issue Type: New Feature Components: core, extensions Reporter: Dan Smith Assignee: Jason Huynh Labels: asf-migration This is a feature that has been under development for GemFire but was not part of the initial drop of code for geode. Geode as a data store for Spark applications is what is being enabled here. By providing a bridge style connector for Spark applications, Geode can become the data store for storing intermediate and final state for Spark applications and allow reference data stored in the in memory tier to be accessed very efficiently for applications Expose Geode regions as Spark RDDs Write Spark RDDs to Geode Regions Execute arbitrary OQL queries in your spark applications -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-121) Document Change: Rename GemFire to Geode
[ https://issues.apache.org/jira/browse/GEODE-121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643456#comment-14643456 ] ASF subversion and git services commented on GEODE-121: --- Commit 60d074cc7adbafc4651e4326dc917e6f67f1a19e in incubator-geode's branch refs/heads/develop from [~jchen21] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=60d074c ] GEODE-121: Document Change: Rename GemFire to Geode Rename GemFire to Geode. But keep gemfire in package names, APIs till related Geode packages are changed. Document Change: Rename GemFire to Geode Key: GEODE-121 URL: https://issues.apache.org/jira/browse/GEODE-121 Project: Geode Issue Type: Sub-task Components: core, extensions Reporter: Jianxia Chen Assignee: Jianxia Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-120) RDD.saveToGemfire() can not handle big dataset (1M entries per partition)
[ https://issues.apache.org/jira/browse/GEODE-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643459#comment-14643459 ] ASF subversion and git services commented on GEODE-120: --- Commit 70448c5dffeb29ae285720b904cfc04f2ef377ec in incubator-geode's branch refs/heads/develop from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=70448c5 ] GEODE-120 Add batch size to RDD.saveToGemfire() RDD.saveToGemfire() can not handle big dataset (1M entries per partition) - Key: GEODE-120 URL: https://issues.apache.org/jira/browse/GEODE-120 Project: Geode Issue Type: Sub-task Components: core, extensions Affects Versions: 1.0.0-incubating Reporter: Qihong Chen Assignee: Qihong Chen Original Estimate: 48h Remaining Estimate: 48h the connector use single region.putAll() call to save each RDD partition. But putAll() doesn't handle big dataset well (such as 1M record). Need to split the dataset into smaller chunks, and invoke putAll() for each chunk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-9) Spark Integration
[ https://issues.apache.org/jira/browse/GEODE-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643458#comment-14643458 ] ASF subversion and git services commented on GEODE-9: - Commit 89b9aaff835cd3545a94bd5b06ce7f0a4c8a2b4b in incubator-geode's branch refs/heads/develop from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=89b9aaf ] Merge branch 'GEODE-114' into feature/GEODE-9 Spark Integration - Key: GEODE-9 URL: https://issues.apache.org/jira/browse/GEODE-9 Project: Geode Issue Type: New Feature Components: core, extensions Reporter: Dan Smith Assignee: Jason Huynh Labels: asf-migration This is a feature that has been under development for GemFire but was not part of the initial drop of code for geode. Geode as a data store for Spark applications is what is being enabled here. By providing a bridge style connector for Spark applications, Geode can become the data store for storing intermediate and final state for Spark applications and allow reference data stored in the in memory tier to be accessed very efficiently for applications Expose Geode regions as Spark RDDs Write Spark RDDs to Geode Regions Execute arbitrary OQL queries in your spark applications -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-9) Spark Integration
[ https://issues.apache.org/jira/browse/GEODE-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643460#comment-14643460 ] ASF subversion and git services commented on GEODE-9: - Commit b73127ce793ecff664d9cdae9132b641c7c9edd0 in incubator-geode's branch refs/heads/develop from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b73127c ] Merge branch 'develop' into feature/GEODE-9 Spark Integration - Key: GEODE-9 URL: https://issues.apache.org/jira/browse/GEODE-9 Project: Geode Issue Type: New Feature Components: core, extensions Reporter: Dan Smith Assignee: Jason Huynh Labels: asf-migration This is a feature that has been under development for GemFire but was not part of the initial drop of code for geode. Geode as a data store for Spark applications is what is being enabled here. By providing a bridge style connector for Spark applications, Geode can become the data store for storing intermediate and final state for Spark applications and allow reference data stored in the in memory tier to be accessed very efficiently for applications Expose Geode regions as Spark RDDs Write Spark RDDs to Geode Regions Execute arbitrary OQL queries in your spark applications -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-9) Spark Integration
[ https://issues.apache.org/jira/browse/GEODE-9?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643594#comment-14643594 ] ASF subversion and git services commented on GEODE-9: - Commit b387e3587c1a8f5566017d322f4ff606b9bc5e21 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b387e35 ] Merge branch 'feature/GEODE-9' into develop Spark Integration - Key: GEODE-9 URL: https://issues.apache.org/jira/browse/GEODE-9 Project: Geode Issue Type: New Feature Components: core, extensions Reporter: Dan Smith Assignee: Jason Huynh Labels: asf-migration This is a feature that has been under development for GemFire but was not part of the initial drop of code for geode. Geode as a data store for Spark applications is what is being enabled here. By providing a bridge style connector for Spark applications, Geode can become the data store for storing intermediate and final state for Spark applications and allow reference data stored in the in memory tier to be accessed very efficiently for applications Expose Geode regions as Spark RDDs Write Spark RDDs to Geode Regions Execute arbitrary OQL queries in your spark applications -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-91) Zero results when querying against a HashIndex indexed on PdxStrings
[ https://issues.apache.org/jira/browse/GEODE-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643591#comment-14643591 ] ASF subversion and git services commented on GEODE-91: -- Commit 27b37efcc34f1d338bd0f6fdfb75e6265a4fefbe in incubator-geode's branch refs/heads/feature/GEODE-77 from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=27b37ef ] GEODE-91: Zero results when querying against a HashIndex indexed on PdxStrings Pdx string fields indexed with a hash index were not being resolved properly when doing comparisons. Zero results when querying against a HashIndex indexed on PdxStrings - Key: GEODE-91 URL: https://issues.apache.org/jira/browse/GEODE-91 Project: Geode Issue Type: Bug Components: querying Reporter: Jason Huynh Assignee: Jason Huynh When a hash index is indexed on a pdx string, the hash index on reevaluation will not correctly compare the strings. The following test will fail: public void testPdxWithStringIndexKeyValues() throws Exception { createPartitionedRegion(test_region); int numEntries = 10; Index index = qs.createHashIndex(idHash, p.id, /test_region p); for (int i = 0; i numEntries; i++) { PdxInstance record = CacheUtils.getCache().createPdxInstanceFactory(test_region).writeString(id, + i).writeString(domain, A).create(); region.put( + i, record); } SelectResults results = (SelectResults) qs.newQuery( SELECT DISTINCT tr.domain FROM /test_region tr WHERE tr.id='1').execute(); assertEquals(1, results.size()); assertTrue(observer.indexUsed); } -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-156) unit tests that call pauseIfNecessary will intermittently fail
[ https://issues.apache.org/jira/browse/GEODE-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643592#comment-14643592 ] ASF subversion and git services commented on GEODE-156: --- Commit 73d0dcd2ae7c4d3ffcd490472d88faf3ec264885 in incubator-geode's branch refs/heads/feature/GEODE-77 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=73d0dcd ] GEODE-156: fix unit tests that call pauseIfNecessary all calls to pauseIfNecessary have been changed to flushIfNecessary. The new flushIfNecessary method uses StateFlushOperation. unit tests that call pauseIfNecessary will intermittently fail -- Key: GEODE-156 URL: https://issues.apache.org/jira/browse/GEODE-156 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Many of the unit tests in MultiVMRegionTestCase call pauseIfNecessary. They do this to allow previous operations done on a no-ack region to complete. But since the pause is simply a sleep it is possible for these tests to not sleep long enough and still fail. They may also sleep longer than is needed. We have a StateFlushOperation that can be used by the sender to wait until all the no-ack ops it did on a particular region have completed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-83) Support AUTH for Redis protocol adaptor
[ https://issues.apache.org/jira/browse/GEODE-83?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643593#comment-14643593 ] ASF subversion and git services commented on GEODE-83: -- Commit 4af9bb3815f59a71605f212c27269a5eeb4a4f7d in incubator-geode's branch refs/heads/feature/GEODE-77 from [~sbawaska] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=4af9bb3 ] Merge branch 'feature/GEODE-83' into develop close #10. Support AUTH for Redis protocol adaptor --- Key: GEODE-83 URL: https://issues.apache.org/jira/browse/GEODE-83 Project: Geode Issue Type: Improvement Components: extensions Reporter: Swapnil Bawaskar The Redis protocol supports basic authentication through the AUTH command: http://redis.io/topics/security Rather than storing the password in redis.conf, we should ask the users to store it in gfsecurity.conf file as a redis-password property. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-161) Remove rerunTest task from gemfire-core/build.gradle
[ https://issues.apache.org/jira/browse/GEODE-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643642#comment-14643642 ] ASF subversion and git services commented on GEODE-161: --- Commit 4f86d40b15e7889c107193249b0c300ab217c732 in incubator-geode's branch refs/heads/develop from [~mbretl] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=4f86d40 ] [GEODE-161] Remove commented rerunTest task Executed 'clean build -Dskip.tests=true' for verfication. Remove rerunTest task from gemfire-core/build.gradle Key: GEODE-161 URL: https://issues.apache.org/jira/browse/GEODE-161 Project: Geode Issue Type: Task Components: build Affects Versions: 1.0.0-incubating Reporter: Mark Bretl Assignee: Mark Bretl Priority: Minor Original Estimate: 1h Remaining Estimate: 1h There is a commented task 'rerunTest' which was added revision 31d1b20ee47088f0518001d87d821838cb614771, this should be removed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639635#comment-14639635 ] ASF subversion and git services commented on GEODE-124: --- Commit dad71e90f2b7d4ef8d354ea7c77ac1bef0d71a6c in incubator-geode's branch refs/heads/feature/GEODE-124 from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=dad71e9 ] GEODE-124: Add Auto-Rebalance trigger logic * Add size threshold configuration * Add logic to trigger rebalance if size to move is above threshold Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-143) PdxDeserializationDUnitTest and PdxClientServerDUnitTest failing because debug level logging left enabled by previous test
[ https://issues.apache.org/jira/browse/GEODE-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639505#comment-14639505 ] ASF subversion and git services commented on GEODE-143: --- Commit aeafb0151c28d397961a64bf9e8f9541dcb508ed in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=aeafb01 ] GEODE-143: logging must not cause cached values to be deserialized A more powerful StringUtils.forceToString now exists that will also print out the elements of arrays. It used to just to a toString on the array which only shows the array identity. The new code will (by default) show the first 16 element values and after that say how many more the array had that are not shown. If you set the sys prop StringUtils.MAX_ARRAY_ELEMENTS_TO_CONVERT you can change the default of 16. PdxDeserializationDUnitTest and PdxClientServerDUnitTest failing because debug level logging left enabled by previous test -- Key: GEODE-143 URL: https://issues.apache.org/jira/browse/GEODE-143 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 24h Remaining Estimate: 24h PdxDeserializationDUnitTest and PdxClientServerDUnitTest started failing in this build: https://builds.apache.org/job/Geode-nightly/159/testReport/junit/ The PdxDeserializationDUnitTest is failing for sure because a previous test left the log-level set to debug. This causes a debug level log message to deserialize the value for logging which causes this test to fail with: Caused by: com.gemstone.gemfire.SerializationException: Deserialization should not be happening in this VM I think the PdxClientServerDUnitTest is also caused by this because it is failing because it see a domain class instance and expect a PdxInstance. Two things need to be done to fix this ticket: 1. determine what test is leaving the log-level at debug/fine and fix it. 2. change the product to tolerate not being able to deserialize a value when it is being done for a log statement. Also logging should not cause the form stored in a VMCachedDeserializable to change from serialized to deserialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-92) PR with entry eviction 1 leaves 3 entries in memory with async overflow
[ https://issues.apache.org/jira/browse/GEODE-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635869#comment-14635869 ] ASF subversion and git services commented on GEODE-92: -- Commit b8686b52b97f8cbe1b46df8cda81a5e2bbd4cfaf in incubator-geode's branch refs/heads/develop from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b8686b5 ] GEODE-56 Decrementing bytesOverflowedOnDisk when an update happens to an evicted entry. For persistent regions, we were correctly updating this stat. But for non persistent regions we did not read the old size on disk, so we were not updating the stat. If the entry was in the async queue, we only increment the stat once when the final value is actually written to disk. So we will only decrement the stat if the oldValue is not pending asynchronous write to disk. I'm still seeing issues with the entryCount. I've tracked this down to what I think is a problem with tombstones. It appears when we modify an entry that is a tombstone, we increment the entry count once for the new value, and once for removing the tombstone. I've filed GEODE-92 and GEODE-93 for the issues I've found with this new test. PR with entry eviction 1 leaves 3 entries in memory with async overflow --- Key: GEODE-92 URL: https://issues.apache.org/jira/browse/GEODE-92 Project: Geode Issue Type: Bug Reporter: Dan Smith Attachments: 0001-Test-that-demonstrates-two-many-entries-in-memory-wi.patch I discovered this while working on a unit test for another issue. It appears that when configuring a PR with entry eviction 1, in some cases there ends up being more than 1 entry left in memory when using overflow to disk with disk-synchronous=false. With synchronous disk, this issue does not occur. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-93) Entry count stats are incorrect with PR with entry eviction and async disk
[ https://issues.apache.org/jira/browse/GEODE-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635870#comment-14635870 ] ASF subversion and git services commented on GEODE-93: -- Commit b8686b52b97f8cbe1b46df8cda81a5e2bbd4cfaf in incubator-geode's branch refs/heads/develop from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b8686b5 ] GEODE-56 Decrementing bytesOverflowedOnDisk when an update happens to an evicted entry. For persistent regions, we were correctly updating this stat. But for non persistent regions we did not read the old size on disk, so we were not updating the stat. If the entry was in the async queue, we only increment the stat once when the final value is actually written to disk. So we will only decrement the stat if the oldValue is not pending asynchronous write to disk. I'm still seeing issues with the entryCount. I've tracked this down to what I think is a problem with tombstones. It appears when we modify an entry that is a tombstone, we increment the entry count once for the new value, and once for removing the tombstone. I've filed GEODE-92 and GEODE-93 for the issues I've found with this new test. Entry count stats are incorrect with PR with entry eviction and async disk -- Key: GEODE-93 URL: https://issues.apache.org/jira/browse/GEODE-93 Project: Geode Issue Type: Bug Reporter: Dan Smith Attachments: 0001-Test-that-demonstrates-incorrect-entriesOnDisk-and-e.patch I discovered this while writing a unit test for GEODE-56. It looks like when doing operations on a PR with entry eviction of 1, overflow to disk, and asynchronous persistence, the stats fo entries in memory and entries on disk are incorrect. Entries in memory is too high and entriesOnDisk goes negative. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-91) Zero results when querying against a HashIndex indexed on PdxStrings
[ https://issues.apache.org/jira/browse/GEODE-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14641042#comment-14641042 ] ASF subversion and git services commented on GEODE-91: -- Commit 27b37efcc34f1d338bd0f6fdfb75e6265a4fefbe in incubator-geode's branch refs/heads/develop from [~huynhja] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=27b37ef ] GEODE-91: Zero results when querying against a HashIndex indexed on PdxStrings Pdx string fields indexed with a hash index were not being resolved properly when doing comparisons. Zero results when querying against a HashIndex indexed on PdxStrings - Key: GEODE-91 URL: https://issues.apache.org/jira/browse/GEODE-91 Project: Geode Issue Type: Bug Components: querying Reporter: Jason Huynh Assignee: Jason Huynh When a hash index is indexed on a pdx string, the hash index on reevaluation will not correctly compare the strings. The following test will fail: public void testPdxWithStringIndexKeyValues() throws Exception { createPartitionedRegion(test_region); int numEntries = 10; Index index = qs.createHashIndex(idHash, p.id, /test_region p); for (int i = 0; i numEntries; i++) { PdxInstance record = CacheUtils.getCache().createPdxInstanceFactory(test_region).writeString(id, + i).writeString(domain, A).create(); region.put( + i, record); } SelectResults results = (SelectResults) qs.newQuery( SELECT DISTINCT tr.domain FROM /test_region tr WHERE tr.id='1').execute(); assertEquals(1, results.size()); assertTrue(observer.indexUsed); } -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-148) Many expiration unit tests take too long to run
[ https://issues.apache.org/jira/browse/GEODE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14640915#comment-14640915 ] ASF subversion and git services commented on GEODE-148: --- Commit 9d7949555fce852b10d0b2e023a578aa446ea23a in incubator-geode's branch refs/heads/develop from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=9d79495 ] Merge branch 'feature/GEODE-148' into develop Many expiration unit tests take too long to run --- Key: GEODE-148 URL: https://issues.apache.org/jira/browse/GEODE-148 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Some examples are: - MultiVMRegionTestCase.testUpdateResetsIdleTime - RegionTestCase.testEntryIdleReset -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-119) Re-introduce the HDFS unit tests that were disabled before code-drop (GEODE-8) due to failures
[ https://issues.apache.org/jira/browse/GEODE-119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638947#comment-14638947 ] ASF subversion and git services commented on GEODE-119: --- Commit b3e216740d75223252c95c0a016828d63cc13398 in incubator-geode's branch refs/heads/develop from [~ashetkar] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b3e2167 ] - GEODE-119: Re-introducing the distributed unit tests that were excluded in the initial code drop to Geode (GEODE-10). - The failures were mostly related to the change in dunit-results directory structure. - Also, did some code formatting and clean up. Re-introduce the HDFS unit tests that were disabled before code-drop (GEODE-8) due to failures -- Key: GEODE-119 URL: https://issues.apache.org/jira/browse/GEODE-119 Project: Geode Issue Type: Test Components: hdfs Reporter: Amogh Shetkar Assignee: Amogh Shetkar Priority: Minor Labels: test The dunits that need to be re-introduced: 1. com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSBasicDUnitTest 2. com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSOffHeapBasicDUnitTest 3. com.gemstone.gemfire.cache.hdfs.internal.RegionWithHDFSPersistenceBasicDUnitTest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-10) HDFS Integration
[ https://issues.apache.org/jira/browse/GEODE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638948#comment-14638948 ] ASF subversion and git services commented on GEODE-10: -- Commit b3e216740d75223252c95c0a016828d63cc13398 in incubator-geode's branch refs/heads/develop from [~ashetkar] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b3e2167 ] - GEODE-119: Re-introducing the distributed unit tests that were excluded in the initial code drop to Geode (GEODE-10). - The failures were mostly related to the change in dunit-results directory structure. - Also, did some code formatting and clean up. HDFS Integration Key: GEODE-10 URL: https://issues.apache.org/jira/browse/GEODE-10 Project: Geode Issue Type: New Feature Components: hdfs Reporter: Dan Smith Assignee: Ashvin Attachments: GEODE-HDFSPersistence-Draft-060715-2109-21516.pdf Ability to persist data on HDFS had been under development for GemFire. It was part of the latest code drop, GEODE-8. As part of this feature we are proposing some changes to the HdfsStore management API (see attached doc for details). # The current API has nested configuration for compaction and async queue. This nested structure forces user to execute multiple steps to manage a store. It also does not seem to be consistent with other management APIs # Some member names in current API are confusing HDFS Integration: Geode as a transactional layer that microbatches data out to Hadoop. This capability makes Geode a NoSQL store that can sit on top of Hadoop and parallelize the process of moving data from the in memory tier into Hadoop, making it very useful for capturing and processing fast data while making it available for Hadoop jobs relatively quickly. The key requirements being met here are # Ingest data into HDFS parallely # Cache bloom filters and allow fast lookups of individual elements # Have programmable policies for deciding what stays in memory # Roll files in HDFS # Index data that is in memory # Have expiration policies that allows the transactional set to decay out older data # Solution needs to support replicated and partitioned regions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-133) testCustomEntryTtl3 RegionTestCase method takes too long and does not validate correctly
[ https://issues.apache.org/jira/browse/GEODE-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639073#comment-14639073 ] ASF subversion and git services commented on GEODE-133: --- Commit e8672d0dbe2fb46671148b538d1570b42d7336ed in incubator-geode's branch refs/heads/feature/GEODE-9 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=e8672d0 ] GEODE-133: fix testCustomEntryTtl3 to do meaningful validation and run faster Test now validates that you can change the custom expiry action and key. The test used to try to change the amount of time but these tests have a hard time validating elapsed time because intermittent changes to system load and the system clock changing due to clock sync services. testCustomEntryTtl3 RegionTestCase method takes too long and does not validate correctly Key: GEODE-133 URL: https://issues.apache.org/jira/browse/GEODE-133 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 4h Remaining Estimate: 4h testCustomEntryTtl3 takes about 10 seconds to run and it does not need to take that long. It also attempts to make sure that custom expiry can be changed on an existing region but the way this test has evolved causes it to no longer validate this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-92) PR with entry eviction 1 leaves 3 entries in memory with async overflow
[ https://issues.apache.org/jira/browse/GEODE-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639075#comment-14639075 ] ASF subversion and git services commented on GEODE-92: -- Commit b8686b52b97f8cbe1b46df8cda81a5e2bbd4cfaf in incubator-geode's branch refs/heads/feature/GEODE-9 from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=b8686b5 ] GEODE-56 Decrementing bytesOverflowedOnDisk when an update happens to an evicted entry. For persistent regions, we were correctly updating this stat. But for non persistent regions we did not read the old size on disk, so we were not updating the stat. If the entry was in the async queue, we only increment the stat once when the final value is actually written to disk. So we will only decrement the stat if the oldValue is not pending asynchronous write to disk. I'm still seeing issues with the entryCount. I've tracked this down to what I think is a problem with tombstones. It appears when we modify an entry that is a tombstone, we increment the entry count once for the new value, and once for removing the tombstone. I've filed GEODE-92 and GEODE-93 for the issues I've found with this new test. PR with entry eviction 1 leaves 3 entries in memory with async overflow --- Key: GEODE-92 URL: https://issues.apache.org/jira/browse/GEODE-92 Project: Geode Issue Type: Bug Reporter: Dan Smith Attachments: 0001-Test-that-demonstrates-two-many-entries-in-memory-wi.patch I discovered this while working on a unit test for another issue. It appears that when configuring a PR with entry eviction 1, in some cases there ends up being more than 1 entry left in memory when using overflow to disk with disk-synchronous=false. With synchronous disk, this issue does not occur. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-124) Ability to auto-rebalance data
[ https://issues.apache.org/jira/browse/GEODE-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639148#comment-14639148 ] ASF subversion and git services commented on GEODE-124: --- Commit 8abba2046c7e6316ce189f18e63dda5a22e673e0 in incubator-geode's branch refs/heads/feature/GEODE-124 from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=8abba20 ] GEODE-124: Create basic auto-balancer skeleton * Create a sub-project for auto-rebalancer * Commit quartz based scheduler to parse and interpret cron strings * Add a new DistributedLockService and Object for AutoBalancer * Create stat for auto-rebalance-attempts in ResourceStats Ability to auto-rebalance data -- Key: GEODE-124 URL: https://issues.apache.org/jira/browse/GEODE-124 Project: Geode Issue Type: New Feature Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Attachments: GEODE-AutomatedRebalancing-150715.pdf Rebalancing relocates data from heavily loaded members to lightly loaded members. Currently Geode only supports manual rebalancing by issuing a gfsh command or a java function call. In most cases, the decision to rebalance is based on the size of the member. As Geode monitors the data size, it can also automatically trigger rebalancing. Auto-Rebalancing is expected to prevent failures seen in an unbalanced cluster more than with a manual rebalance. Latest spec: https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-144) Try-with-resources for Geode cache
[ https://issues.apache.org/jira/browse/GEODE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639343#comment-14639343 ] ASF subversion and git services commented on GEODE-144: --- Commit 7f72d8da6bfc97a4f1d4b82727dc6934addfab6a in incubator-geode's branch refs/heads/develop from William Markito [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=7f72d8d ] [GEODE-144] - Adding try-with-resources to cache API. Try-with-resources for Geode cache -- Key: GEODE-144 URL: https://issues.apache.org/jira/browse/GEODE-144 Project: Geode Issue Type: Improvement Components: core Reporter: William Markito Oliveira Assignee: William Markito Oliveira Priority: Minor I'd like to propose implementing AutoClosable interface in order to allow Geode users to treat Cache as an AutoClosable resource. Most of old GemFire examples and Geode docs[1] as well include a very common practice as follows: {code} Cache cache = new CacheFactory().set(name, nodeName) .set(log-level, fine) .set(statistic-sampling-enabled, true)... .create(); // application code // Close the cache and disconnect System.out.println(Closing the cache and disconnecting.); cache.close(); {code} The idea would be to avoid such closing calls when user think it's appropriate by using try-with-resources syntax: {code} try ( Cache cache = new CacheFactory().set(name, nodeName).create() ) { // application code } {code} There would be no need to call cache.close() or use try/finally blocks. Please find in the link below more examples [3] of try-with-resources. [1] http://gemfire.docs.pivotal.io/latest/userguide/index.html#basic_config/the_cache/managing_a_client_cache.html [2] http://docs.oracle.com/javase/7/docs/api/java/lang/AutoCloseable.html [3] http://docs.oracle.com/javase/7/docs/technotes/guides/language/try-with-resources.html [4] http://gemfire.docs.pivotal.io/latest/javadocs/japi/com/gemstone/gemfire/cache/RegionService.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-134) AnalyzeSerializablesJUnitTest is failing after hdfs api changes
[ https://issues.apache.org/jira/browse/GEODE-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639072#comment-14639072 ] ASF subversion and git services commented on GEODE-134: --- Commit a571fc6c67d1fe45a5473b312859f040805dd953 in incubator-geode's branch refs/heads/feature/GEODE-9 from Ashvin Agrawal [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=a571fc6 ] GEODE-134: Fix Serializable tests HDFS classes are new and do not break backward compatibility. AnalyzeSerializablesJUnitTest is failing after hdfs api changes --- Key: GEODE-134 URL: https://issues.apache.org/jira/browse/GEODE-134 Project: Geode Issue Type: Bug Components: core Affects Versions: 1.0.0-incubating Reporter: Ashvin Assignee: Ashvin Fix For: 1.0.0-incubating As part of updates for GEODE-10, serialized hdfs classes got modified. The changes are causing AnalyzeSerializablesJUnitTest to fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-143) PdxDeserializationDUnitTest and PdxClientServerDUnitTest failing because debug level logging left enabled by previous test
[ https://issues.apache.org/jira/browse/GEODE-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637959#comment-14637959 ] ASF subversion and git services commented on GEODE-143: --- Commit 59e5e40d726fffdb54570d647118d6096bd0730a in incubator-geode's branch refs/heads/feature/GEODE-143 from [~dschneider] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=59e5e40 ] GEODE-143: logging must not cause cached values to be deserialized A more powerful StringUtils.forceToString now exists that will also print out the elements of arrays. It used to just to a toString on the array which only shows the array identity. The new code will (by default) show the first 16 element values and after that say how many more the array had that are not shown. If you set the sys prop StringUtils.MAX_ARRAY_ELEMENTS_TO_CONVERT you can change the default of 16. PdxDeserializationDUnitTest and PdxClientServerDUnitTest failing because debug level logging left enabled by previous test -- Key: GEODE-143 URL: https://issues.apache.org/jira/browse/GEODE-143 Project: Geode Issue Type: Bug Reporter: Darrel Schneider Assignee: Darrel Schneider Priority: Minor Original Estimate: 24h Remaining Estimate: 24h PdxDeserializationDUnitTest and PdxClientServerDUnitTest started failing in this build: https://builds.apache.org/job/Geode-nightly/159/testReport/junit/ The PdxDeserializationDUnitTest is failing for sure because a previous test left the log-level set to debug. This causes a debug level log message to deserialize the value for logging which causes this test to fail with: Caused by: com.gemstone.gemfire.SerializationException: Deserialization should not be happening in this VM I think the PdxClientServerDUnitTest is also caused by this because it is failing because it see a domain class instance and expect a PdxInstance. Two things need to be done to fix this ticket: 1. determine what test is leaving the log-level at debug/fine and fix it. 2. change the product to tolerate not being able to deserialize a value when it is being done for a log statement. Also logging should not cause the form stored in a VMCachedDeserializable to change from serialized to deserialized. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-149) gradle build does not rerun passed test
[ https://issues.apache.org/jira/browse/GEODE-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635944#comment-14635944 ] ASF subversion and git services commented on GEODE-149: --- Commit 77d7fa5b8c6e7c36fc752b5403872a2ed52ddaa5 in incubator-geode's branch refs/heads/develop from [~upthewaterspout] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=77d7fa5 ] GEODE-149 Reexecute a test even it previously passed Modified build.gradle to make the test task as not up to date. This forces gradle to rerun the test target every time it is invoked. gradle build does not rerun passed test --- Key: GEODE-149 URL: https://issues.apache.org/jira/browse/GEODE-149 Project: Geode Issue Type: Bug Reporter: Dan Smith Assignee: Dan Smith Priority: Minor When running a test multiple times, gradle does not reexecute the test because the output is considered up to date. This makes it difficult to try to reproduce a test that only fails intermittently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (GEODE-114) There's race condition in DefaultGemFireConnection.getRegionProxy
[ https://issues.apache.org/jira/browse/GEODE-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14630045#comment-14630045 ] ASF subversion and git services commented on GEODE-114: --- Commit dc1c1559dd8a38ffade204f5440755338de2a5c8 in incubator-geode's branch refs/heads/feature/GEODE-9 from [~qihong] [ https://git-wip-us.apache.org/repos/asf?p=incubator-geode.git;h=dc1c155 ] GEODE-114: fix race condition in DefaultGemFireConnection.getRegionProxy There's race condition in DefaultGemFireConnection.getRegionProxy - Key: GEODE-114 URL: https://issues.apache.org/jira/browse/GEODE-114 Project: Geode Issue Type: Sub-task Components: core, extensions Affects Versions: 1.0.0-incubating Reporter: Qihong Chen Assignee: Qihong Chen Original Estimate: 24h Remaining Estimate: 24h when multiple threads try to call getRegionProxy with the same region at the same time, the following exception was thrown: com.gemstone.gemfire.cache.RegionExistsException: /debs at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.createVMRegion(GemFireCacheImpl.java:2880) at com.gemstone.gemfire.internal.cache.GemFireCacheImpl.basicCreateRegion(GemFireCacheImpl.java:2835) at com.gemstone.gemfire.cache.client.internal.ClientRegionFactoryImpl.create(ClientRegionFactoryImpl.java:223) at io.pivotal.gemfire.spark.connector.internal.DefaultGemFireConnection.getRegionProxy(DefaultGemFireConnection.scala:87) at io.pivotal.gemfire.spark.connector.internal.rdd.GemFirePairRDDWriter.write(GemFireRDDWriter.scala:47) at io.pivotal.gemfire.spark.connector.GemFirePairRDDFunctions$$anonfun$saveToGemfire$2.apply(GemFirePairRDDFunctions.scala:24) at io.pivotal.gemfire.spark.connector.GemFirePairRDDFunctions$$anonfun$saveToGemfire$2.apply(GemFirePairRDDFunctions.scala:24) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61) at org.apache.spark.scheduler.Task.run(Task.scala:64) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)