[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-15 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r10343 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-10 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r132589551 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #152: ORC-227: [C++] Fix docker failure due to ExternalProj...

2017-08-10 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/152#discussion_r132522783 --- Diff: c++/src/ByteRLE.cc --- @@ -26,9 +26,9 @@ namespace orc { - const size_t MINIMUM_REPEAT = 3; - const size_t MAXIMUM_REPEAT

[GitHub] orc pull request #152: ORC-227: [C++] Fix docker failure due to ExternalProj...

2017-08-10 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/152#discussion_r132531200 --- Diff: c++/src/ByteRLE.cc --- @@ -26,9 +26,9 @@ namespace orc { - const size_t MINIMUM_REPEAT = 3; - const size_t MAXIMUM_REPEAT

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-14 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r133008869 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-07-20 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r128647874 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-07-19 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r128392669 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124718336 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124720134 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124718523 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124720147 --- Diff: c++/src/ColumnWriter.cc --- @@ -0,0 +1,507 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-28 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124715175 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124718827 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #134: Orc 17

2017-07-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r125728350 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -66,22 +64,22 @@ namespace orc { options = config->GetOptions(); } hdfs::IoServ

[GitHub] orc pull request #134: Orc 17

2017-07-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r125728966 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -123,17 +123,21 @@ namespace orc { uint64_t length, uint64_t offset) override

[GitHub] orc pull request #134: Orc 17

2017-07-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r125729396 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -34,15 +34,13 @@ #include "common/hdfs_configuration.h" #include "common/configu

[GitHub] orc issue #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-07-05 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/128 Hi @omalley , please see my replies and the new commit. Let me know if you have further questions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] orc issue #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-08-07 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/128 @omalley Update change again, please take a look. I also reduce the initial stream memory capacity from 4M to 1M, and greatly reduce the pre-allocated memory for footers and ps. This should alleviate

[GitHub] orc pull request #142: [ORC-218] Cache timezone information in the library.

2017-07-31 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/142#discussion_r130473949 --- Diff: c++/src/CMakeLists.txt --- @@ -125,6 +125,83 @@ include_directories ( ${LZ4_INCLUDE_DIRS} ) +# To avoid reading the Timezone

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-08-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r131780372 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,228 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #119: ORC-177 OutputStream interface and implementation

2017-05-10 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/119 ORC-177 OutputStream interface and implementation 1. Create output stream interface and implementation classes 2. Add memory input output stream for the convenince of UT 3. Add corresponding unit

[GitHub] orc issue #119: ORC-177 OutputStream interface and implementation

2017-05-15 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/119 hi @omalley , can you please take a look at this? Thx. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-16 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/122 ORC-192 Implement zlib compresion stream Implement zlib compressor based on the output stream. Add corresponding UTs. Also rename the existing test suite from TestCompression

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120799267 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc issue #132: ORC-202. Add writer implementation enum to file format

2017-06-21 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/132 That makes sense. I add the writer code in c++ writer. We might need a similar change for c++ reader. Will do it in a separate PR. --- If your project is set up for it, you can reply to this email

[GitHub] orc pull request #134: Orc 17

2017-06-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123853204 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -34,15 +34,13 @@ #include "common/hdfs_configuration.h" #include "common/configu

[GitHub] orc pull request #134: Orc 17

2017-06-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123852932 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -121,39 +116,24 @@ namespace orc { } uint64_t getNaturalReadSize() const override

[GitHub] orc pull request #134: Orc 17

2017-06-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123853053 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -66,22 +64,22 @@ namespace orc { options = config->GetOptions(); } hdfs::IoServ

[GitHub] orc pull request #134: Orc 17

2017-06-22 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123646956 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -0,0 +1,170 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-22 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117822601 --- Diff: c++/src/Compression.cc --- @@ -636,6 +884,33 @@ DIAGNOSTIC_POP return static_cast(result); } + std::unique_ptr

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118044090 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048038 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048019 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118046740 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048655 --- Diff: c++/src/RLEv1.hh --- @@ -26,6 +26,59 @@ namespace orc { +class RleEncoderV1 : public RleEncoder { +public

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048627 --- Diff: c++/src/RLEv1.cc --- @@ -26,8 +26,173 @@ namespace orc { const uint64_t MINIMUM_REPEAT = 3; +const uint64_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048556 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc issue #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/126 Thanks @majetideepak for reviewing. @prasanthj @omalley please help review and/or accept the PR. Thx! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118057495 --- Diff: c++/test/TestByteRLEEncoder.cc --- @@ -0,0 +1,231 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118057983 --- Diff: c++/test/TestByteRLEEncoder.cc --- @@ -0,0 +1,231 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-20 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117622782 --- Diff: c++/src/Compression.cc --- @@ -636,6 +884,33 @@ DIAGNOSTIC_POP return static_cast(result); } + std::unique_ptr

[GitHub] orc issue #122: ORC-192 Implement zlib compresion stream

2017-05-19 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/122 @majetideepak @omalley please take another look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117381049 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117379895 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117380313 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117378369 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117363862 --- Diff: c++/src/Compression.cc --- @@ -636,6 +884,33 @@ DIAGNOSTIC_POP return static_cast(result); } + std::unique_ptr

[GitHub] orc issue #126: ORC-191 Implement RLE v1 encoder

2017-05-19 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/126 @omalley @majetideepak Can you also take a look at this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-18 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/126 ORC-191 Implement RLE v1 encoder Implement integer, byte and boolean RLE encoder, and add corresponding UTs. Only v1 is implemented at this point. Change-Id

[GitHub] orc issue #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/122 @omalley can you please take a look at this? Thx. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] orc issue #122: ORC-192 Implement zlib compresion stream

2017-05-22 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/122 Hi @majetideepak , what happens next after you approved? are you able to accept this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120708078 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120706989 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120699358 --- Diff: c++/include/orc/OrcFile.hh --- @@ -119,6 +120,17 @@ namespace orc { * @param path the name of the file in the local file system

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120701079 --- Diff: c++/src/ColumnWriter.cc --- @@ -0,0 +1,507 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120704852 --- Diff: c++/src/ColumnWriter.cc --- @@ -0,0 +1,507 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120739743 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120751420 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc issue #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-01 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/128 @majetideepak have you got a chance to take a look? Sorry for the big diff. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] orc pull request #116: ORC-185: [C++] Simplify Statististics Implementation

2017-05-08 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/116#discussion_r115346953 --- Diff: c++/src/Statistics.hh --- @@ -41,49 +41,181 @@ namespace orc { }; /** + * Internal Statistics Implementation

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-05-24 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/128 ORC-178 Implement Basic C++ Writer and Writer Option 1. Add basic Writer and WriterOption 2. Add StructColumnWriter and IntegerColumnWriter. With them, we will be able to write a complete ORC

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-15 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 @majetideepak how do I do that? :) ---

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-11 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r138235902 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,18 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc issue #169: [WIP] ORC-203 Modify the StringStatistics to trim the minimu...

2017-09-22 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/169 @dain if the concern is the performance, should we also limit the string length when generate stats in writer path, which in my opinion is more costly? I think if we keep the string compare up to 1024

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-20 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 Squash commit. Thanks @ajayyadava @majetideepak ---

[GitHub] orc issue #169: [WIP] ORC-203 Modify the StringStatistics to trim the minimu...

2017-09-21 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/169 I don't understand why you need to trim the strings. Protobuf doesn't support strings over 1024 characters? ---

[GitHub] orc issue #159: ORC-175: ORC-232: add jmh-generator-annprocess in pom.xml. i...

2017-09-11 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/159 @iamhumanbeing did you compare it with zstd? Based on my experience, zstd is way better than igzip. I would expect a similar result with ISA-L. It doesn't seem to be adding a lot of value if we plan

[GitHub] orc pull request #134: Orc 17

2017-09-11 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r138018152 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -0,0 +1,172 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #134: Orc 17

2017-09-11 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r138019713 --- Diff: c++/include/orc/OrcFile.hh --- @@ -103,12 +103,18 @@ namespace orc { }; /** - * Create a stream to a local file

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-04 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 @omalley please take another look. thanks. ---

[GitHub] orc pull request #134: Orc 17

2017-09-13 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r138531251 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -0,0 +1,172 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc issue #134: Orc 17

2017-09-13 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/134 LGTM ---

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-13 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 @majetideepak pls take another look. Thx! ---

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r134897076 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-02 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r136697977 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #199: ORC-276: [C++] Create a simple tool to import CSV fil...

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/199#discussion_r155596136 --- Diff: tools/src/CSVFileImport.cc --- @@ -0,0 +1,411 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #199: ORC-276: [C++] Create a simple tool to import CSV fil...

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/199#discussion_r155593796 --- Diff: tools/src/CSVFileImport.cc --- @@ -0,0 +1,411 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #199: ORC-276: [C++] Create a simple tool to import CSV fil...

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/199#discussion_r155594426 --- Diff: tools/src/CSVFileImport.cc --- @@ -0,0 +1,411 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc issue #198: Orc-272: Minor porting changes.

2017-12-07 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/198 LGTM ---

[GitHub] orc pull request #184: Orc 256 unmask range option

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/184#discussion_r154762146 --- Diff: java/core/src/java/org/apache/orc/impl/mask/RedactMaskFactory.java --- @@ -619,7 +646,7 @@ public double maskDouble(double value) { } else

[GitHub] orc pull request #184: Orc 256 unmask range option

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/184#discussion_r154761512 --- Diff: java/core/src/java/org/apache/orc/impl/mask/RedactMaskFactory.java --- @@ -245,8 +271,8 @@ public void maskData(ColumnVector original, ColumnVector

[GitHub] orc pull request #184: Orc 256 unmask range option

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/184#discussion_r155623157 --- Diff: java/core/src/test/org/apache/orc/impl/mask/TestUnmaskRange.java --- @@ -0,0 +1,165 @@ +package org.apache.orc.impl.mask

[GitHub] orc pull request #268: ORC-363 Enable zstd decompression in ORC Java reader

2018-05-14 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/268 ORC-363 Enable zstd decompression in ORC Java reader 1. Upgrade aircompressor lib to 0.11 2. Enable Zstd decompression in Java reader 3. Zstd compression is still not availiable. Will throw

[GitHub] orc issue #268: ORC-363 Enable zstd decompression in ORC Java reader

2018-05-17 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/268 The current solution is not perfect. But at least it gives us some ability to read zstd Orc files, which I believe is important from the compatibility perspective - our in-house system has zstd Orc

[GitHub] orc issue #273: ORC-343 Enable C++ writer to support RleV2

2018-05-25 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/273 @majetideepak this is RLEv2 change that was promised. @yuruiz Could you also include some perf data obtained from offline testing? ---

[GitHub] orc pull request #211: ORC-290 [C++] Update Readme to include C++ writer inf...

2018-01-18 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/211 ORC-290 [C++] Update Readme to include C++ writer info Change-Id: Ic419427b7441c96f63221ba650f9965e5342680e You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] orc issue #212: ORC-293: [C++] Fix RleEncoderV1 for case when sizeof(long) <...

2018-01-24 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/212 +1 ---

[GitHub] orc issue #214: ORC-273: [C++] add range check to prevent bad memory access.

2018-02-06 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/214 LGTM. ---

[GitHub] orc issue #199: ORC-276: [C++] Create a simple tool to import CSV files

2018-01-01 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/199 Hi @majetideepak , Gang is on vacation and will look into your feedback after he's back next week. Thx. ---

[GitHub] orc issue #293: ORC-388: Fix isSafeSubtract to use logic operator instead of...

2018-07-25 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/293 LGTM ---

[GitHub] orc issue #268: ORC-363 Enable zstd decompression in ORC Java reader

2018-08-10 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/268 Hi all, I'd like to bring this up again. It's almost another three months, and we haven't seen the zstd java library yet. I would suggest we move forward to add zstd support in C++ reader/writer

[GitHub] orc issue #300: [ORC-394][C++] Add addUserMetadata() function to C++ write

2018-08-10 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/300 Looks good. Thanks for adding this. ---

[GitHub] orc pull request #301: ORC-395: Support ZSTD in C++ writer/reader

2018-09-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/301#discussion_r215486619 --- Diff: c++/src/Compression.cc --- @@ -899,6 +907,166 @@ DIAGNOSTIC_POP return static_cast(result); } + /** + * Block

[GitHub] orc pull request #301: ORC-395: Support ZSTD in C++ writer/reader

2018-09-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/301#discussion_r215486726 --- Diff: c++/src/Compression.cc --- @@ -899,6 +907,166 @@ DIAGNOSTIC_POP return static_cast(result); } + /** + * Block

[GitHub] orc pull request #212: ORC-293: [C++] Fix RleEncoderV1 for case when sizeof(...

2018-01-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/212#discussion_r163463896 --- Diff: c++/include/orc/Common.hh --- @@ -69,7 +69,7 @@ namespace orc { UNKNOWN_WRITER = INT32_MAX }; - enum CompressionKind

[GitHub] orc pull request #245: ORC-161: Proposal for new decimal encodings and stati...

2018-04-12 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/245#discussion_r181149073 --- Diff: site/_docs/encodings.md --- @@ -123,6 +127,41 @@ DIRECT_V2 | PRESENT | Yes | Boolean RLE | DATA

[GitHub] orc pull request #247: ORC-339. Reorganize the ORC file format specification...

2018-04-13 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/247#discussion_r181530787 --- Diff: site/specification/ORCv2.md --- @@ -0,0 +1,1032 @@ +--- +layout: page +title: Evolving Draft for ORC Specification v2

[GitHub] orc issue #240: ORC-331: [C++] Initial support of Windows/MSVC

2018-04-08 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/240 We should also add an MSVC build in travis. Otherwise there's no way to verify it. ---

[GitHub] orc issue #241: ORC-332: Add syntax version to orc_proto.proto

2018-04-09 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/241 LGTM ---

[GitHub] orc issue #243: Update the site with more information about developing.

2018-04-10 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/243 LGTM. Thanks for putting together a document. ---

  1   2   >