[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/3053 > Does this PR fix two problems? > If it is yes, better to separate it into two. > the one line change of rowId to rowId + 1 is coupled with this, when i removed the compress method in unSafeFixLengthColumnPage, i got this issue and fixed in this, so this is required in this PR only ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/3053 @kumarvishal09 ...I agree with you that it is a functional issue and we need to merge it. My point was before merging we can do one load performance test to see if there is any performance degrade and if there is any then we can update the benchmark results ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/3053 @kumarvishal09 i have tested the fallback scenario by changing code, it is even failing with that also and i have raised discussion in snappy community also [https://groups.google.com/forum/#!topic/snappy-compression/4noNVKCMBqM](url) ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user kumarvishal09 commented on the issue: https://github.com/apache/carbondata/pull/3053 @manishgupta88 @xuchuanyin I think if it's really a problem with snappy then whether any performance impact is there or not we have to merge as its a functional issue. :) @akashrn5 May be this issue is coming because of offheap to onheap fallback in UnsafeMemoryManager can u please verify once. Please try discuss with snappy community also. ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/3053 @akashrn5 .I agree with @xuchuanyin before merging the PR it is better to get the PR tested for performance. We can observe 2 things during the benchmark test - performance and compression ratio of rawcompress Vs compressDouble and then take the final decision ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/3053 Does this PR fix two problems? If it is yes, better to separate it into two. And for the first problem, I'm also concerning about the performance decrease. The rawCompress can save some memory copy operations, that's why we add a check there and try to use that feature if the compressor supports that. It may needs more observations about the performance decreasement OR we can just add a switch there to control the behavior and it will be helpful for comparison. ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2220/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10477/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2440/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2218/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10472/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2436/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user akashrn5 commented on the issue: https://github.com/apache/carbondata/pull/3053 > i think the performance of rawCompress is better than compressLong,compressInt, can we find the root cause of JVM crashï¼ i dont think there is much difference we get with timing, but problem is JVM crash happens randomly, since we get maxsizefor compression from snapy itself, we even allocated that memory sucessfully and passed the address to snappy, after that JVM crashed. This is very random. So better to remove that. ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user qiuchenjian commented on the issue: https://github.com/apache/carbondata/pull/3053 i think the performance of rawCompress is better than compressLong,compressInt, can we find the root cause of JVM crashï¼ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10455/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2415/ ---
[GitHub] carbondata issue #3053: [CARBONDATA-3233]Fix JVM crash issue in snappy compr...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/3053 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2198/ ---
