[GitHub] orc pull request #301: ORC-395: Support ZSTD in C++ writer/reader

2018-09-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/301#discussion_r215486726 --- Diff: c++/src/Compression.cc --- @@ -899,6 +907,166 @@ DIAGNOSTIC_POP return static_cast(result); } + /** + * Block

[GitHub] orc pull request #301: ORC-395: Support ZSTD in C++ writer/reader

2018-09-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/301#discussion_r215486619 --- Diff: c++/src/Compression.cc --- @@ -899,6 +907,166 @@ DIAGNOSTIC_POP return static_cast(result); } + /** + * Block

[GitHub] orc issue #268: ORC-363 Enable zstd decompression in ORC Java reader

2018-08-10 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/268 Hi all, I'd like to bring this up again. It's almost another three months, and we haven't seen the zstd java library yet. I would suggest we move forward to add zstd support in C++ reader/writer

[GitHub] orc issue #300: [ORC-394][C++] Add addUserMetadata() function to C++ write

2018-08-10 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/300 Looks good. Thanks for adding this. ---

[GitHub] orc issue #293: ORC-388: Fix isSafeSubtract to use logic operator instead of...

2018-07-25 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/293 LGTM ---

[GitHub] orc issue #273: ORC-343 Enable C++ writer to support RleV2

2018-05-25 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/273 @majetideepak this is RLEv2 change that was promised. @yuruiz Could you also include some perf data obtained from offline testing? ---

[GitHub] orc issue #268: ORC-363 Enable zstd decompression in ORC Java reader

2018-05-17 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/268 The current solution is not perfect. But at least it gives us some ability to read zstd Orc files, which I believe is important from the compatibility perspective - our in-house system has zstd Orc

[GitHub] orc pull request #268: ORC-363 Enable zstd decompression in ORC Java reader

2018-05-14 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/268 ORC-363 Enable zstd decompression in ORC Java reader 1. Upgrade aircompressor lib to 0.11 2. Enable Zstd decompression in Java reader 3. Zstd compression is still not availiable. Will throw

[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-18 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/245 +1 for Gang's proposal. ---

[GitHub] orc pull request #247: ORC-339. Reorganize the ORC file format specification...

2018-04-13 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/247#discussion_r181530787 --- Diff: site/specification/ORCv2.md --- @@ -0,0 +1,1032 @@ +--- +layout: page +title: Evolving Draft for ORC Specification v2

[GitHub] orc pull request #245: ORC-161: Proposal for new decimal encodings and stati...

2018-04-12 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/245#discussion_r181149073 --- Diff: site/_docs/encodings.md --- @@ -123,6 +127,41 @@ DIRECT_V2 | PRESENT | Yes | Boolean RLE | DATA

[GitHub] orc issue #243: Update the site with more information about developing.

2018-04-10 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/243 LGTM. Thanks for putting together a document. ---

[GitHub] orc issue #241: ORC-332: Add syntax version to orc_proto.proto

2018-04-09 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/241 LGTM ---

[GitHub] orc issue #240: ORC-331: [C++] Initial support of Windows/MSVC

2018-04-08 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/240 We should also add an MSVC build in travis. Otherwise there's no way to verify it. ---

[GitHub] orc pull request #233: ORC-322: [C++] Fix writing & reading timestamp

2018-03-24 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/233#discussion_r176909659 --- Diff: c++/src/ColumnWriter.cc --- @@ -1194,9 +1194,8 @@ namespace orc { bool hasNull = false; for (uint64_t i = 0; i < numValues;

[GitHub] orc pull request #233: ORC-322: [C++] Fix writing & reading timestamp

2018-03-24 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/233#discussion_r176910052 --- Diff: c++/src/ColumnReader.cc --- @@ -336,8 +336,7 @@ namespace orc { } } int64_t writerTime = secsBuffer[i

[GitHub] orc issue #233: ORC-322: [C++] Fix writing & reading timestamp

2018-03-23 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/233 IMO the ideal approach of achieving the timestamp semantics is to have the caller pass in session timezone, and the reader return timestamp batch based on that. That would be similar to what we do

[GitHub] orc issue #233: ORC-322: [C++] Fix writing & reading timestamp

2018-03-22 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/233 Sorry, I am confused after reading the discussions above. The key question I have is - do we implement ORC TIMESTAMP as SQL "TIMESTAMP with Timezone" or "TIMESTAMP without Timezone&q

[GitHub] orc issue #214: ORC-273: [C++] add range check to prevent bad memory access.

2018-02-06 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/214 LGTM. ---

[GitHub] orc issue #212: ORC-293: [C++] Fix RleEncoderV1 for case when sizeof(long) <...

2018-01-24 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/212 +1 ---

[GitHub] orc pull request #212: ORC-293: [C++] Fix RleEncoderV1 for case when sizeof(...

2018-01-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/212#discussion_r163463896 --- Diff: c++/include/orc/Common.hh --- @@ -69,7 +69,7 @@ namespace orc { UNKNOWN_WRITER = INT32_MAX }; - enum CompressionKind

[GitHub] orc pull request #211: ORC-290 [C++] Update Readme to include C++ writer inf...

2018-01-18 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/211 ORC-290 [C++] Update Readme to include C++ writer info Change-Id: Ic419427b7441c96f63221ba650f9965e5342680e You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] orc issue #199: ORC-276: [C++] Create a simple tool to import CSV files

2018-01-01 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/199 Hi @majetideepak , Gang is on vacation and will look into your feedback after he's back next week. Thx. ---

[GitHub] orc pull request #184: Orc 256 unmask range option

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/184#discussion_r155623157 --- Diff: java/core/src/test/org/apache/orc/impl/mask/TestUnmaskRange.java --- @@ -0,0 +1,165 @@ +package org.apache.orc.impl.mask

[GitHub] orc pull request #184: Orc 256 unmask range option

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/184#discussion_r154761512 --- Diff: java/core/src/java/org/apache/orc/impl/mask/RedactMaskFactory.java --- @@ -245,8 +271,8 @@ public void maskData(ColumnVector original, ColumnVector

[GitHub] orc pull request #184: Orc 256 unmask range option

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/184#discussion_r154762146 --- Diff: java/core/src/java/org/apache/orc/impl/mask/RedactMaskFactory.java --- @@ -619,7 +646,7 @@ public double maskDouble(double value) { } else

[GitHub] orc issue #198: Orc-272: Minor porting changes.

2017-12-07 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/198 LGTM ---

[GitHub] orc pull request #199: ORC-276: [C++] Create a simple tool to import CSV fil...

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/199#discussion_r155596136 --- Diff: tools/src/CSVFileImport.cc --- @@ -0,0 +1,411 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #199: ORC-276: [C++] Create a simple tool to import CSV fil...

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/199#discussion_r155593796 --- Diff: tools/src/CSVFileImport.cc --- @@ -0,0 +1,411 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #199: ORC-276: [C++] Create a simple tool to import CSV fil...

2017-12-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/199#discussion_r155594426 --- Diff: tools/src/CSVFileImport.cc --- @@ -0,0 +1,411 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc issue #169: [WIP] ORC-203 Modify the StringStatistics to trim the minimu...

2017-09-22 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/169 @dain if the concern is the performance, should we also limit the string length when generate stats in writer path, which in my opinion is more costly? I think if we keep the string compare up to 1024

[GitHub] orc issue #169: [WIP] ORC-203 Modify the StringStatistics to trim the minimu...

2017-09-21 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/169 I don't understand why you need to trim the strings. Protobuf doesn't support strings over 1024 characters? ---

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-20 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 Squash commit. Thanks @ajayyadava @majetideepak ---

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-15 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 @majetideepak how do I do that? :) ---

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-13 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 @majetideepak pls take another look. Thx! ---

[GitHub] orc issue #134: Orc 17

2017-09-13 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/134 LGTM ---

[GitHub] orc pull request #134: Orc 17

2017-09-13 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r138531251 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -0,0 +1,172 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-11 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r138235902 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,18 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #134: Orc 17

2017-09-11 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r138018152 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -0,0 +1,172 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #134: Orc 17

2017-09-11 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r138019713 --- Diff: c++/include/orc/OrcFile.hh --- @@ -103,12 +103,18 @@ namespace orc { }; /** - * Create a stream to a local file

[GitHub] orc issue #159: ORC-175: ORC-232: add jmh-generator-annprocess in pom.xml. i...

2017-09-11 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/159 @iamhumanbeing did you compare it with zstd? Based on my experience, zstd is way better than igzip. I would expect a similar result with ISA-L. It doesn't seem to be adding a lot of value if we plan

[GitHub] orc issue #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-04 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/151 @omalley please take another look. thanks. ---

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-09-02 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r136697977 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r134897076 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-15 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r10343 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-14 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r133008869 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #151: ORC-226 Support getWriterId in c++ reader interface

2017-08-10 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/151#discussion_r132589551 --- Diff: c++/include/orc/Reader.hh --- @@ -288,6 +288,17 @@ namespace orc { virtual uint64_t getCompressionSize() const = 0

[GitHub] orc pull request #152: ORC-227: [C++] Fix docker failure due to ExternalProj...

2017-08-10 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/152#discussion_r132531200 --- Diff: c++/src/ByteRLE.cc --- @@ -26,9 +26,9 @@ namespace orc { - const size_t MINIMUM_REPEAT = 3; - const size_t MAXIMUM_REPEAT

[GitHub] orc pull request #152: ORC-227: [C++] Fix docker failure due to ExternalProj...

2017-08-10 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/152#discussion_r132522783 --- Diff: c++/src/ByteRLE.cc --- @@ -26,9 +26,9 @@ namespace orc { - const size_t MINIMUM_REPEAT = 3; - const size_t MAXIMUM_REPEAT

[GitHub] orc issue #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-08-07 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/128 @omalley Update change again, please take a look. I also reduce the initial stream memory capacity from 4M to 1M, and greatly reduce the pre-allocated memory for footers and ps. This should alleviate

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-08-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r131780372 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,228 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #142: [ORC-218] Cache timezone information in the library.

2017-07-31 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/142#discussion_r130473949 --- Diff: c++/src/CMakeLists.txt --- @@ -125,6 +125,83 @@ include_directories ( ${LZ4_INCLUDE_DIRS} ) +# To avoid reading the Timezone

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-07-20 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r128647874 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-07-19 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r128392669 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc issue #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-07-05 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/128 Hi @omalley , please see my replies and the new commit. Let me know if you have further questions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] orc pull request #134: Orc 17

2017-07-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r125729396 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -34,15 +34,13 @@ #include "common/hdfs_configuration.h" #include "common/configu

[GitHub] orc pull request #134: Orc 17

2017-07-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r125728966 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -123,17 +123,21 @@ namespace orc { uint64_t length, uint64_t offset) override

[GitHub] orc pull request #134: Orc 17

2017-07-05 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r125728350 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -66,22 +64,22 @@ namespace orc { options = config->GetOptions(); } hdfs::IoServ

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124720147 --- Diff: c++/src/ColumnWriter.cc --- @@ -0,0 +1,507 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124720134 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124718827 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124718523 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-29 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124718336 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-28 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r124715175 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #134: Orc 17

2017-06-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123852932 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -121,39 +116,24 @@ namespace orc { } uint64_t getNaturalReadSize() const override

[GitHub] orc pull request #134: Orc 17

2017-06-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123853053 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -66,22 +64,22 @@ namespace orc { options = config->GetOptions(); } hdfs::IoServ

[GitHub] orc pull request #134: Orc 17

2017-06-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123853204 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -34,15 +34,13 @@ #include "common/hdfs_configuration.h" #include "common/configu

[GitHub] orc pull request #134: Orc 17

2017-06-22 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/134#discussion_r123646956 --- Diff: c++/src/OrcHdfsFile.cc --- @@ -0,0 +1,170 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc issue #132: ORC-202. Add writer implementation enum to file format

2017-06-21 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/132 That makes sense. I add the writer code in c++ writer. We might need a similar change for c++ reader. Will do it in a separate PR. --- If your project is set up for it, you can reply to this email

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120799267 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120751420 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120739743 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120708078 --- Diff: c++/include/orc/Writer.hh --- @@ -0,0 +1,294 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120706989 --- Diff: c++/src/Writer.cc --- @@ -0,0 +1,659 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120704852 --- Diff: c++/src/ColumnWriter.cc --- @@ -0,0 +1,507 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120701079 --- Diff: c++/src/ColumnWriter.cc --- @@ -0,0 +1,507 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-07 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/128#discussion_r120699358 --- Diff: c++/include/orc/OrcFile.hh --- @@ -119,6 +120,17 @@ namespace orc { * @param path the name of the file in the local file system

[GitHub] orc issue #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-06-01 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/128 @majetideepak have you got a chance to take a look? Sorry for the big diff. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] orc pull request #128: ORC-178 Implement Basic C++ Writer and Writer Option

2017-05-24 Thread xndai
GitHub user xndai opened a pull request: https://github.com/apache/orc/pull/128 ORC-178 Implement Basic C++ Writer and Writer Option 1. Add basic Writer and WriterOption 2. Add StructColumnWriter and IntegerColumnWriter. With them, we will be able to write a complete ORC

[GitHub] orc issue #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/126 Thanks @majetideepak for reviewing. @prasanthj @omalley please help review and/or accept the PR. Thx! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118057983 --- Diff: c++/test/TestByteRLEEncoder.cc --- @@ -0,0 +1,231 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118057495 --- Diff: c++/test/TestByteRLEEncoder.cc --- @@ -0,0 +1,231 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048655 --- Diff: c++/src/RLEv1.hh --- @@ -26,6 +26,59 @@ namespace orc { +class RleEncoderV1 : public RleEncoder { +public

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048627 --- Diff: c++/src/RLEv1.cc --- @@ -26,8 +26,173 @@ namespace orc { const uint64_t MINIMUM_REPEAT = 3; +const uint64_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048556 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048038 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118048019 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118046740 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc pull request #126: ORC-191 Implement RLE v1 encoder

2017-05-23 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/126#discussion_r118044090 --- Diff: c++/src/ByteRLE.cc --- @@ -27,6 +27,272 @@ namespace orc { const size_t MINIMUM_REPEAT = 3; + const size_t MAXIMUM_REPEAT = 127

[GitHub] orc issue #122: ORC-192 Implement zlib compresion stream

2017-05-22 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/122 Hi @majetideepak , what happens next after you approved? are you able to accept this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-22 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117822601 --- Diff: c++/src/Compression.cc --- @@ -636,6 +884,33 @@ DIAGNOSTIC_POP return static_cast(result); } + std::unique_ptr

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-20 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117622782 --- Diff: c++/src/Compression.cc --- @@ -636,6 +884,33 @@ DIAGNOSTIC_POP return static_cast(result); } + std::unique_ptr

[GitHub] orc issue #126: ORC-191 Implement RLE v1 encoder

2017-05-19 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/126 @omalley @majetideepak Can you also take a look at this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] orc issue #122: ORC-192 Implement zlib compresion stream

2017-05-19 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/122 @majetideepak @omalley please take another look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117381049 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117380313 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117379895 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117378369 --- Diff: c++/src/Compression.cc --- @@ -33,6 +33,254 @@ namespace orc { + class CompressionStreamBase: public BufferedOutputStream

[GitHub] orc pull request #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on a diff in the pull request: https://github.com/apache/orc/pull/122#discussion_r117363862 --- Diff: c++/src/Compression.cc --- @@ -636,6 +884,33 @@ DIAGNOSTIC_POP return static_cast(result); } + std::unique_ptr

[GitHub] orc issue #122: ORC-192 Implement zlib compresion stream

2017-05-18 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/122 @omalley can you please take a look at this? Thx. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   >