[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2778 LGTM ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2778 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8912/ ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2778 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/844/ ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2778 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/649/ ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/2778 @manishgupta88 yeah, I go through the code again and finally find the root cause: the key used to load cache and the key used to clear cache is different due to the difference in path separator between linux and windows. Please check the modification again. ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2778 @xuchuanyin .`CarbonLRUCache` instance is one per JVM but cache implementation is different for different purpose like we have separate cache provider implementation for blockDataMap and dictionary. You can check the `CacheProvider` class to get some more details on it. Cache is a standard interface which has very generic methods. Because we are storing all the entries in LRU cache to control the memory through LRU eviction all the keys are being stored in LRU cache map. BloomDataMap should also be aware about its keys and then use the invalidate API to clean all the required keys. Check the `clear` method implementation of BlockletDataMpaFactory ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/2778 @manishgupta88 Oh, if it is so, then I think the interfaces of cache are confusing. Currently we use only one cache instance for all purpose like mainDataMap, bloomDataMap, dictionary, but we didn't provide a way to separate them well. Here my initial intention is to drop all the cache for one datamap. If I call `invalidate(K key)`, I need to provide all the keys, which is inconvenient. ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2778 @xuchuanyin ...I think this PR changes are not correct...you are calling `lruCache.clear()` which will clear the complete LRU cache mapthat means all the entries for all the tables will be cleared because LRUCache instance is only 1 per JVM.I believe through this PR you are only trying to clear the stale entry for table being recreated You can check the flow of drop table to understand how the dataMap cache gets cleared on dropping the table and try to incorporate Bloom dataMap cache clearing changes as per that. You can use the existing API `void invalidate(K key)` and try not to introduce a new API `void invalidateAll()` ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2778 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/806/ ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2778 Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8875/ ---
[GitHub] carbondata issue #2778: [CARBONDATA-2980][BloomDataMap] Fix bug in clearing ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2778 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/614/ ---
