----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36397/#review91487 -----------------------------------------------------------
gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 19) <https://reviews.apache.org/r/36397/#comment144900> How is this disk store used in later steps? Is it configured on the HDFS store or the Region? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 20) <https://reviews.apache.org/r/36397/#comment144882> Is "HDFS buffers..." a step the user performs? This list is supposed to be a list of steps done by the user gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 22) <https://reviews.apache.org/r/36397/#comment144883> Is it also connected to the optional disk store in step 1? Does a new step or sentence begin between "Store Uses"? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 38) <https://reviews.apache.org/r/36397/#comment144884> All the "public static" constants on HDFSStore should have javadocs gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 67) <https://reviews.apache.org/r/36397/#comment144885> The last sentence "... is also fine" should probably be removed. I think you should define the precedence of the different ways of specifying the name node url. Also if the explicit one is defined will it only be used or if is not found will it still pick up one from hdfs-site.xml and/or the classpath. gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 119) <https://reviews.apache.org/r/36397/#comment144888> change "means that less number of bigger batches are persisted" to "causes fewer and bigger batches to be persisted" gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 121) <https://reviews.apache.org/r/36397/#comment144890> consume more memory gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 138) <https://reviews.apache.org/r/36397/#comment144896> Is this really per region or would is it per HDFS store? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 149) <https://reviews.apache.org/r/36397/#comment144897> Change "to buffer prevent data" to "to prevent buffer data"? Also "Persisting data may" to "Persisting buffer data may" gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 168) <https://reviews.apache.org/r/36397/#comment144901> What used sync disk writes? Is it only when the store writes to its diskStore? What is the default? What are the risks of setting it to false? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 177) <https://reviews.apache.org/r/36397/#comment144902> Change "This clause" to "This option"? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 184) <https://reviews.apache.org/r/36397/#comment144903> Change MaxWriteOnlyFileSize to WriteOnlyFileSizeMaximum I think in general Max should be a suffix instead of a prefix. I'm ok with Max instead of Maximum but thought it might be better to use the full word. gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 193) <https://reviews.apache.org/r/36397/#comment144906> This one seems related to the previous size limit. Perhaps better names for both are: WriteOnlyFileSizeLimit WriteOnlyFileTimeLimit which ever limit is hit first causes the file to be closed. In general seems like defaults should be doc'd (you have constants so put a link in these javadocs to the default constant javadocs). gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 226) <https://reviews.apache.org/r/36397/#comment144910> make sure and mention the units (in this case seconds) in the first line of the javadocs. Make sure this is done for all the attributes. For example: Number of seconds to wait before starting the next major compaction. gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 236) <https://reviews.apache.org/r/36397/#comment144911> Add some info on the life cycle of these threads. Will they all immediately be created and live forever? Or do they only get created as needed. Do they ever timeout? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 245) <https://reviews.apache.org/r/36397/#comment144914> instead of "amount of time" change to "number of UNITS" (seconds?). Also the first sentence of the javadocs should describe the attribute with a quick summary. In this case it is intro material. gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 253) <https://reviews.apache.org/r/36397/#comment144916> "this this" -> "this" gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 254) <https://reviews.apache.org/r/36397/#comment144917> Shouldn't the IllegalStateException info be moved to @throws IllegalStateException? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 278) <https://reviews.apache.org/r/36397/#comment144919> The first line should describe this attribute. The generic "This advanced..." info could be in a later sentence. For example: A file larger than this size, in megabytes, will not be considered for minor compaction. This option is advanced ... gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java (line 282) <https://reviews.apache.org/r/36397/#comment144918> Added "MinorCompaction" to these advanced configs? Get rid of "MB" in the method name. None of the other methods encode the units in the name. I think Max and Min should be a suffix instead of a prefix. gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStoreFactory.java (line 27) <https://reviews.apache.org/r/36397/#comment144922> Should these settor methods (also on the mutator interface) throw IllegalArgumentException for values out of range? For example should you be able to configure the number of threads to be < 0? gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStoreMutator.java (line 11) <https://reviews.apache.org/r/36397/#comment144920> Add an interface comment saying what this is, how you get and instance, and how to use it. gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStoreMutator.java (line 20) <https://reviews.apache.org/r/36397/#comment144921> Why the magic -1? For booleans you return Boolean and use null. Why not use this for ordinal's too? In this case it would return Integer. It seems unlikely that code will use the get methods on the mutator interface. - Darrel Schneider On July 10, 2015, 1:45 p.m., Ashvin A wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36397/ > ----------------------------------------------------------- > > (Updated July 10, 2015, 1:45 p.m.) > > > Review request for geode, Amogh Shetkar and Dan Smith. > > > Bugs: GEODE-10 > https://issues.apache.org/jira/browse/GEODE-10 > > > Repository: geode > > > Description > ------- > > GEODE-10: Refactor HdfsStore api to match spec > > * Currently HdfsStore's configuration object is nested and a user needs to > create multiple sub objects to manage the store instance. This is less > usable > and gets confusing at times. User also gets exposed to a lot of internal > details. So replacing nested configuration with a flat structure will be > better. > * Rename members > > > Diffs > ----- > > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSEventQueueAttributes.java > ef7e863 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSEventQueueAttributesFactory.java > fc09b7a > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStore.java > f5bd943 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStoreFactory.java > 516d2aa > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/HDFSStoreMutator.java > 47b1708 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSEventQueueAttributesImpl.java > df89841 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSIntegrationUtil.java > 242923b > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSStoreConfigHolder.java > 50ea3c6 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSStoreCreation.java > b5fbfe8 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSStoreFactoryImpl.java > e7e75dc > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSStoreImpl.java > 8e7e358 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSStoreMutatorImpl.java > e4e2093 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/SizeTieredHdfsCompactionConfigHolder.java > 57d58b7 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HDFSCompactionManager.java > d96cd11 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HDFSUnsortedHoplogOrganizer.java > c9be401 > > gemfire-core/src/main/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HdfsSortedOplogOrganizer.java > 7bb025b > > gemfire-core/src/main/java/com/gemstone/gemfire/internal/cache/GemFireCacheImpl.java > 76488dd > > gemfire-core/src/main/java/com/gemstone/gemfire/internal/cache/xmlcache/CacheCreation.java > a5fd3f1 > > gemfire-core/src/main/java/com/gemstone/gemfire/internal/cache/xmlcache/CacheXmlParser.java > e015624 > > gemfire-core/src/main/java/com/gemstone/gemfire/management/internal/cli/commands/HDFSStoreCommands.java > b044156 > > gemfire-core/src/main/java/com/gemstone/gemfire/management/internal/cli/functions/AlterHDFSStoreFunction.java > 13ba5fa > > gemfire-core/src/main/java/com/gemstone/gemfire/management/internal/cli/functions/CreateHDFSStoreFunction.java > ac9c895 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/ColocatedRegionWithHDFSDUnitTest.java > ea9dc11 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/HDFSConfigJUnitTest.java > dc3f404 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/HdfsStoreMutatorJUnitTest.java > f12078e > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/BaseHoplogTestCase.java > 06c3c4b > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HDFSCacheLoaderJUnitTest.java > 3e4b8e2 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HDFSCompactionManagerJUnitTest.java > 2c32d61 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HDFSUnsortedHoplogOrganizerJUnitTest.java > 8d2fae3 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/HdfsSortedOplogOrganizerJUnitTest.java > 0884a40 > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/TieredCompactionJUnitTest.java > 007e73b > > gemfire-core/src/test/java/com/gemstone/gemfire/cache/hdfs/internal/hoplog/mapreduce/HoplogUtilJUnitTest.java > 576144f > > gemfire-core/src/test/java/com/gemstone/gemfire/internal/cache/HDFSRegionOperationsJUnitTest.java > d068cb4 > > gemfire-core/src/test/java/com/gemstone/gemfire/management/internal/cli/commands/HDFSStoreCommandsJUnitTest.java > b46abc7 > > gemfire-core/src/test/java/com/gemstone/gemfire/management/internal/cli/functions/AlterHDFSStoreFunctionJUnitTest.java > f66f54b > > gemfire-core/src/test/java/com/gemstone/gemfire/management/internal/cli/functions/CreateHDFSStoreFunctionJUnitTest.java > a20297b > > gemfire-core/src/test/java/com/gemstone/gemfire/management/internal/cli/functions/DescribeHDFSStoreFunctionJUnitTest.java > fe630fe > > Diff: https://reviews.apache.org/r/36397/diff/ > > > Testing > ------- > > > Thanks, > > Ashvin A > >
