> On April 28, 2015, 5:43 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala, > > line 88 > > <https://reviews.apache.org/r/33453/diff/2/?file=939982#file939982line88> > > > > qq: Is there any checksum in the offset file to make sure that we don't > > read in a corrupted value? > > Navina Ramesh wrote: > No. As of now, we don't have any checksum. Do you think it is better to > add it now than later ?
Yi, I created SAMZA-666 to work on adding checksum data to the offset file. - Navina ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33453/#review81838 ----------------------------------------------------------- On May 4, 2015, 8:57 p.m., Navina Ramesh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33453/ > ----------------------------------------------------------- > > (Updated May 4, 2015, 8:57 p.m.) > > > Review request for samza, Yan Fang, Chris Riccomini, Naveen Somasundaram, and > Yi Pan (Data Infrastructure). > > > Repository: samza > > > Description > ------- > > Added Unit Tests for TaskStorageManager and refactored some code > > Changed default to yarn cwd instead of io.tmpDir and refactored code > > > Diffs > ----- > > samza-core/src/main/scala/org/apache/samza/config/ShellCommandConfig.scala > e94a4735217f59d074510ce1556c8c439e6a72f0 > samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala > ac4793afe1e6868933e750181bee1e27c157b5e6 > samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala > f68a7fee24614fce101e91c4f933d9b4e65dda0a > > samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala > PRE-CREATION > > samza-kv-rocksdb/src/main/scala/org/apache/samza/storage/kv/RocksDbKeyValueStore.scala > 1b44a517129b35affac802929087eaa0061e6b5d > > Diff: https://reviews.apache.org/r/33453/diff/ > > > Testing > ------- > > Tested locally using hello-samza. > Note: you have to set an environment variable LOGGED_STORE_BASE_DIR pointing > to the new location to persist the changelog attached stores. Otherwise, it > will default to YARN's cwd and will not re-use local state. > > > Thanks, > > Navina Ramesh > >
