[GitHub] carbondata pull request #2525: [CARBONDATA-2756] refactored code to use ZSTD...
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2525#discussion_r203991119
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/filesystem/LocalCarbonFile.java
---
@@ -375,7 +381,15 @@ public DataOutputStream getDataOutputStream(String
path, FileFactory.FileType fi
} else if ("ZSTD".equalsIgnoreCase(compressor)) {
// compression level 1 is cost-effective for sort temp file
// which is not used for storage
- outputStream = new ZstdOutputStream(new FileOutputStream(path), 1);
+ FileOutputStream fileOutputStream = new FileOutputStream(path);
+ try {
+outputStream = (OutputStream)
Class.forName("com.github.luben.zstd.ZstdOutputStream")
+.getConstructor(OutputStream.class,
int.class).newInstance(fileOutputStream, 1);
+ } catch (ReflectiveOperationException e) {
+throw new IOException(e);
+ } finally {
+fileOutputStream.close();
--- End diff --
dont close the stream hereit should be closed by the caller after its
usage is complete
Only in case of failure you close in catch block and catch for
Throwable...same comment is applicable for above code changes also..
---
[GitHub] carbondata pull request #2525: [CARBONDATA-2756] refactored code to use ZSTD...
Github user KanakaKumar commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2525#discussion_r203637393
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/filesystem/LocalCarbonFile.java
---
@@ -293,7 +291,12 @@ public boolean delete() {
} else if ("LZ4".equalsIgnoreCase(compressor)) {
inputStream = new LZ4BlockInputStream(new FileInputStream(path));
} else if ("ZSTD".equalsIgnoreCase(compressor)) {
- inputStream = new ZstdInputStream(new FileInputStream(path));
+ try {
+inputStream = (InputStream)
Class.forName("com.github.luben.zstd.ZstdInputStream")
+.getConstructors()[0].newInstance(new FileInputStream(path));
+ } catch (ReflectiveOperationException e) {
+throw new RuntimeException(e);
--- End diff --
Throw IOException with a hint to "Ensure ZSTD lib is in class path or not"
---
[GitHub] carbondata pull request #2525: [CARBONDATA-2756] refactored code to use ZSTD...
GitHub user kunal642 opened a pull request: https://github.com/apache/carbondata/pull/2525 [CARBONDATA-2756] refactored code to use ZSTD compression using Reflection 1. refactored code to use ZSTD compression using Reflection 2. add license Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kunal642/carbondata zstd_fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2525.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2525 commit 1cada8b2b23f846d31ab80c1592346804583d17b Author: kunal642 Date: 2018-07-19T06:47:04Z refactored code to use ZSTD compression using Reflection ---
