[arrow] branch master updated: ARROW-9681: [Java] Fix test failures of Arrow Memory - Core on big-endian platform

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new cf1c749  ARROW-9681: [Java] Fix test failures of Arrow Memory - Core 
on big-endian platform
cf1c749 is described below

commit cf1c749098d49a1f1f2f4c7dc1ea645f3102824c
Author: Kazuaki Ishizaki 
AuthorDate: Thu Aug 13 21:23:29 2020 -0700

ARROW-9681: [Java] Fix test failures of Arrow Memory - Core on big-endian 
platform

This PR fixes two test failures on BE platform by taking care of big-endian 
in `setBytes()` and `memcpy()`.

```
% mvn -B -Drat.skip=true 
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
 -Dflatc.download.skip=true -rf :arrow-memory-core test
...
[ERROR]   TestArrowBuf.testSetBytesBigEndian:145 arrays first differed at 
element [0]; expected:<0> but was:<7>
[ERROR]   TestByteFunctionHelpers.testStringCompare Multiple Failures (2 
failures)
expected:<-1> but was:<1>
...
[ERROR] Tests run: 31, Failures: 2, Errors: 0, Skipped: 0
```

Closes #7923 from kiszk/ARROW-9681

Authored-by: Kazuaki Ishizaki 
Signed-off-by: Micah Kornfield 
---
 .../java/org/apache/arrow/memory/ArrowBuf.java |  4 +--
 .../arrow/memory/util/ByteFunctionHelpers.java | 34 ++
 2 files changed, 31 insertions(+), 7 deletions(-)

diff --git 
a/java/memory/memory-core/src/main/java/org/apache/arrow/memory/ArrowBuf.java 
b/java/memory/memory-core/src/main/java/org/apache/arrow/memory/ArrowBuf.java
index e81a92f..ea5e29f 100644
--- 
a/java/memory/memory-core/src/main/java/org/apache/arrow/memory/ArrowBuf.java
+++ 
b/java/memory/memory-core/src/main/java/org/apache/arrow/memory/ArrowBuf.java
@@ -163,7 +163,7 @@ public final class ArrowBuf implements AutoCloseable {
* Returns the byte order of elements in this buffer.
*/
   public ByteOrder order() {
-return ByteOrder.LITTLE_ENDIAN;
+return ByteOrder.nativeOrder();
   }
 
   /**
@@ -846,7 +846,7 @@ public final class ArrowBuf implements AutoCloseable {
 src.position(src.position() + length);
   } else {
 final ByteOrder originalByteOrder = src.order();
-src.order(ByteOrder.LITTLE_ENDIAN);
+src.order(order());
 try {
   // copy word at a time
   while (length - 128 >= LONG_SIZE) {
diff --git 
a/java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/ByteFunctionHelpers.java
 
b/java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/ByteFunctionHelpers.java
index df57128..9579245 100644
--- 
a/java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/ByteFunctionHelpers.java
+++ 
b/java/memory/memory-core/src/main/java/org/apache/arrow/memory/util/ByteFunctionHelpers.java
@@ -17,6 +17,8 @@
 
 package org.apache.arrow.memory.util;
 
+import java.nio.ByteOrder;
+
 import org.apache.arrow.memory.ArrowBuf;
 import org.apache.arrow.memory.BoundsChecking;
 import org.apache.arrow.memory.util.hash.ArrowBufHasher;
@@ -28,6 +30,8 @@ import org.apache.arrow.memory.util.hash.SimpleHasher;
 public class ByteFunctionHelpers {
   static final org.slf4j.Logger logger = 
org.slf4j.LoggerFactory.getLogger(ByteFunctionHelpers.class);
 
+  private static final boolean LITTLE_ENDIAN = ByteOrder.nativeOrder() == 
ByteOrder.LITTLE_ENDIAN;
+
   private ByteFunctionHelpers() {}
 
   /**
@@ -152,7 +156,11 @@ public class ByteFunctionHelpers {
 long leftLong = MemoryUtil.UNSAFE.getLong(lPos);
 long rightLong = MemoryUtil.UNSAFE.getLong(rPos);
 if (leftLong != rightLong) {
-  return unsignedLongCompare(Long.reverseBytes(leftLong), 
Long.reverseBytes(rightLong));
+  if (LITTLE_ENDIAN) {
+return unsignedLongCompare(Long.reverseBytes(leftLong), 
Long.reverseBytes(rightLong));
+  } else {
+return unsignedLongCompare(leftLong, rightLong);
+  }
 }
 lPos += 8;
 rPos += 8;
@@ -164,7 +172,11 @@ public class ByteFunctionHelpers {
   long leftLong = MemoryUtil.UNSAFE.getLong(lPos);
   long rightLong = MemoryUtil.UNSAFE.getLong(rPos);
   if (leftLong != rightLong) {
-return unsignedLongCompare(Long.reverseBytes(leftLong), 
Long.reverseBytes(rightLong));
+if (LITTLE_ENDIAN) {
+  return unsignedLongCompare(Long.reverseBytes(leftLong), 
Long.reverseBytes(rightLong));
+} else {
+  return unsignedLongCompare(leftLong, rightLong);
+}
   }
   lPos += 8;
   rPos += 8;
@@ -175,7 +187,11 @@ public class ByteFunctionHelpers {
   int leftInt = MemoryUtil.UNSAFE.getInt(lPos);
   int rightInt = MemoryUtil.UNSAFE.getInt(rPos);
   if (leftInt != rightInt) {
-return unsignedIntCompare(Integer.reverseBytes(left

[arrow] branch master updated (b2788c5 -> d23f0a6)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from b2788c5  ARROW-9725: [Rust] [DataFusion] SortExec and LimitExec re-use 
MergeExec
 add d23f0a6  ARROW-9706: [Java] Tests of TestLargeListVector correctly 
read offset

No new revisions were added by this update.

Summary of changes:
 .../apache/arrow/vector/TestLargeListVector.java   | 50 +++---
 1 file changed, 25 insertions(+), 25 deletions(-)

[arrow] branch master updated: ARROW-9671: [C++] Fix a bug in BasicDecimal128 constructor that interprets uint64_t integers with highest bit set as negative.

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 3d0a9d5  ARROW-9671: [C++] Fix a bug in BasicDecimal128 constructor 
that interprets uint64_t integers with highest bit set as negative.
3d0a9d5 is described below

commit 3d0a9d58b6fe29dcb208c3fa244c789449517988
Author: Mingyu Zhong 
AuthorDate: Fri Aug 7 19:56:01 2020 -0700

ARROW-9671: [C++] Fix a bug in BasicDecimal128 constructor that interprets 
uint64_t integers with highest bit set as negative.

Closes #7915 from MingyuZhong/bn

Lead-authored-by: Mingyu Zhong 
Co-authored-by: Micah Kornfield 
Signed-off-by: Micah Kornfield 
---
 cpp/src/arrow/util/basic_decimal.h |  7 ---
 cpp/src/arrow/util/decimal_test.cc | 22 --
 2 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/cpp/src/arrow/util/basic_decimal.h 
b/cpp/src/arrow/util/basic_decimal.h
index 01feeac..23c38bb 100644
--- a/cpp/src/arrow/util/basic_decimal.h
+++ b/cpp/src/arrow/util/basic_decimal.h
@@ -51,10 +51,11 @@ class ARROW_EXPORT BasicDecimal128 {
 
   /// \brief Convert any integer value into a BasicDecimal128.
   template ::value, 
T>::type>
+typename = typename std::enable_if<
+std::is_integral::value && (sizeof(T) <= sizeof(uint64_t)), 
T>::type>
   constexpr BasicDecimal128(T value) noexcept
-  : BasicDecimal128(static_cast(value) >= 0 ? 0 : -1,
-static_cast(value)) {}
+  : BasicDecimal128(value >= T{0} ? 0 : -1, static_cast(value)) 
{  // NOLINT
+  }
 
   /// \brief Create a BasicDecimal128 from an array of bytes. Bytes are 
assumed to be in
   /// native-endian byte order.
diff --git a/cpp/src/arrow/util/decimal_test.cc 
b/cpp/src/arrow/util/decimal_test.cc
index b62992c..856f10e 100644
--- a/cpp/src/arrow/util/decimal_test.cc
+++ b/cpp/src/arrow/util/decimal_test.cc
@@ -218,8 +218,7 @@ TEST(DecimalZerosTest, NoLeadingZerosDecimalPoint) {
 template 
 class Decimal128Test : public ::testing::Test {
  public:
-  Decimal128Test() : value_(42) {}
-  const T value_;
+  Decimal128Test() {}
 };
 
 using Decimal128Types =
@@ -231,18 +230,29 @@ using Decimal128Types =
 TYPED_TEST_SUITE(Decimal128Test, Decimal128Types);
 
 TYPED_TEST(Decimal128Test, ConstructibleFromAnyIntegerType) {
-  Decimal128 value(this->value_);
-  ASSERT_EQ(42, value.low_bits());
+  Decimal128 value(TypeParam{42});
+  EXPECT_EQ(42, value.low_bits());
+  EXPECT_EQ(0, value.high_bits());
+
+  Decimal128 max_value(std::numeric_limits::max());
+  EXPECT_EQ(std::numeric_limits::max(), max_value.low_bits());
+  EXPECT_EQ(0, max_value.high_bits());
+
+  Decimal128 min_value(std::numeric_limits::min());
+  EXPECT_EQ(std::numeric_limits::min(), min_value.low_bits());
+  EXPECT_EQ((std::is_signed::value ? -1 : 0), 
min_value.high_bits());
 }
 
 TEST(Decimal128TestTrue, ConstructibleFromBool) {
   Decimal128 value(true);
-  ASSERT_EQ(1, value.low_bits());
+  EXPECT_EQ(1, value.low_bits());
+  EXPECT_EQ(0, value.high_bits());
 }
 
 TEST(Decimal128TestFalse, ConstructibleFromBool) {
   Decimal128 value(false);
-  ASSERT_EQ(0, value.low_bits());
+  EXPECT_EQ(0, value.low_bits());
+  EXPECT_EQ(0, value.high_bits());
 }
 
 TEST(Decimal128Test, Division) {

[arrow] branch master updated: ARROW-9536: [Java] Miss parameters in PlasmaOutOfMemoryException.java

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new f40e287  ARROW-9536: [Java] Miss parameters in 
PlasmaOutOfMemoryException.java
f40e287 is described below

commit f40e2871919b577995a8e9e2419af2e494f574cb
Author: offthewall123 
AuthorDate: Fri Aug 7 19:51:43 2020 -0700

ARROW-9536: [Java] Miss parameters in PlasmaOutOfMemoryException.java

Miss parameters in PlasmaOutOfMemoryException.java

Closes #7815 from offthewall123/miss_parameter_bug_fix

Authored-by: offthewall123 
Signed-off-by: Micah Kornfield 
---
 .../plasma/exceptions/PlasmaOutOfMemoryException.java   |  8 
 .../java/org/apache/arrow/plasma/PlasmaClientTest.java  | 17 -
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git 
a/java/plasma/src/main/java/org/apache/arrow/plasma/exceptions/PlasmaOutOfMemoryException.java
 
b/java/plasma/src/main/java/org/apache/arrow/plasma/exceptions/PlasmaOutOfMemoryException.java
index fd323fa..ffc4177 100644
--- 
a/java/plasma/src/main/java/org/apache/arrow/plasma/exceptions/PlasmaOutOfMemoryException.java
+++ 
b/java/plasma/src/main/java/org/apache/arrow/plasma/exceptions/PlasmaOutOfMemoryException.java
@@ -22,6 +22,14 @@ package org.apache.arrow.plasma.exceptions;
  */
 public class PlasmaOutOfMemoryException extends RuntimeException {
 
+  public PlasmaOutOfMemoryException(String message) {
+super("The plasma store ran out of memory." + message);
+  }
+
+  public PlasmaOutOfMemoryException(String message, Throwable t) {
+super("The plasma store ran out of memory." + message, t);
+  }
+
   public PlasmaOutOfMemoryException() {
 super("The plasma store ran out of memory.");
   }
diff --git 
a/java/plasma/src/test/java/org/apache/arrow/plasma/PlasmaClientTest.java 
b/java/plasma/src/test/java/org/apache/arrow/plasma/PlasmaClientTest.java
index d3aebea..e02ee51 100644
--- a/java/plasma/src/test/java/org/apache/arrow/plasma/PlasmaClientTest.java
+++ b/java/plasma/src/test/java/org/apache/arrow/plasma/PlasmaClientTest.java
@@ -26,6 +26,7 @@ import java.util.stream.Collectors;
 
 import org.apache.arrow.plasma.exceptions.DuplicateObjectException;
 import org.apache.arrow.plasma.exceptions.PlasmaClientException;
+import org.apache.arrow.plasma.exceptions.PlasmaOutOfMemoryException;
 import org.junit.Assert;
 
 public class PlasmaClientTest {
@@ -277,6 +278,20 @@ public class PlasmaClientTest {
 client.release(id);
   }
 
+  public void doPlasmaOutOfMemoryExceptionTest() {
+System.out.println("Start PlasmaOutOfMemoryException test.");
+PlasmaClient client = (PlasmaClient) pLink;
+byte[] objectId = new byte[20];
+Arrays.fill(objectId, (byte) 1);
+try {
+  ByteBuffer byteBuffer = client.create(objectId, 2, null);
+  Assert.fail("Fail to create an object, The plasma store ran out of 
memory.");
+} catch (PlasmaOutOfMemoryException e) {
+  System.out.println(String.format("Expected PlasmaOutOfMemoryException: 
%s", e));
+  System.out.println("PlasmaOutOfMemoryException test success.");
+}
+  }
+
   private byte[] getArrayFilledWithValue(int arrayLength, byte val) {
 byte[] arr = new byte[arrayLength];
 Arrays.fill(arr, val);
@@ -290,9 +305,9 @@ public class PlasmaClientTest {
   public static void main(String[] args) throws Exception {
 
 PlasmaClientTest plasmaClientTest = new PlasmaClientTest();
+plasmaClientTest.doPlasmaOutOfMemoryExceptionTest();
 plasmaClientTest.doByteBufferTest();
 plasmaClientTest.doTest();
-
   }
 
 }

[arrow] branch decimal256 created (now 7525286)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch decimal256
in repository https://gitbox.apache.org/repos/asf/arrow.git.


  at 7525286  ARROW-9573: [Python][Dataset] Provide 
read_table(ignore_prefixes=)

No new revisions were added by this update.

[arrow] branch master updated: ARROW-9308: [Format] Add Feature enum for forward compatibility.

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new b990efe  ARROW-9308: [Format] Add Feature enum for forward 
compatibility.
b990efe is described below

commit b990efe80605d7757aeee2bcc2376ddeb7f7aaf2
Author: Micah Kornfield 
AuthorDate: Thu Jul 2 20:33:45 2020 -0700

ARROW-9308: [Format] Add Feature enum for forward compatibility.

Will start a discussion on the mailing list shortly.

Closes #7502 from emkornfield/feature_flag

Authored-by: Micah Kornfield 
Signed-off-by: Micah Kornfield 
---
 format/Schema.fbs | 32 
 1 file changed, 32 insertions(+)

diff --git a/format/Schema.fbs b/format/Schema.fbs
index 2624730..09b0730 100644
--- a/format/Schema.fbs
+++ b/format/Schema.fbs
@@ -40,6 +40,35 @@ enum MetadataVersion:short {
   V5,
 }
 
+/// Represents Arrow Features that might not have full support
+/// within implementations. This is intended to be used in
+/// two scenarios:
+///  1.  A mechanism for readers of Arrow Streams
+///  and files to understand that the stream or file makes
+///  use of a feature that isn't supported or unknown to
+///  the implementation (and therefore can meet the Arrow
+///  forward compatibility guarantees).
+///  2.  A means of negotiating between a client and server
+///  what features a stream is allowed to use. The enums
+///  values here are intented to represent higher level
+///  features, additional details maybe negotiated
+///  with key-value pairs specific to the protocol.
+///
+/// Enums added to this list should be assigned power-of-two values
+/// to facilitate exchanging and comparing bitmaps for supported
+/// features.
+enum Feature : long {
+  /// Needed to make flatbuffers happy.
+  UNUSED = 0,
+  /// The stream makes use of multiple full dictionaries with the
+  /// same ID and assumes clients implement dictionary replacement
+  /// correctly.
+  DICTIONARY_REPLACEMENT = 1,
+  /// The stream makes use of compressed bodies as described
+  /// in Message.fbs.
+  COMPRESSED_BODY = 2
+}
+
 /// These are stored in the flatbuffer in the Type union below
 
 table Null {
@@ -369,6 +398,9 @@ table Schema {
   fields: [Field];
   // User-defined metadata
   custom_metadata: [ KeyValue ];
+
+  /// Features used in the stream/file.
+  features : [ Feature ];
 }
 
 root_type Schema;

[arrow] branch master updated (b344b3b -> e78aa4c)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from b344b3b  ARROW-9090: [C++] Bump versions of bundled libraries
 add e78aa4c  ARROW-7955: [Java] Support large buffer for file/stream IPC

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/memory/ArrowBuf.java |   4 +-
 .../apache/arrow/vector/BaseFixedWidthVector.java  |  12 +-
 .../arrow/vector/BaseVariableWidthVector.java  |  78 -
 .../org/apache/arrow/vector/VarBinaryVector.java   |   2 +-
 .../org/apache/arrow/vector/VarCharVector.java |   2 +-
 .../org/apache/arrow/vector/ipc/ArrowReader.java   |  12 +-
 .../apache/arrow/vector/ipc/ArrowStreamReader.java |   2 +-
 .../org/apache/arrow/vector/ipc/ArrowWriter.java   |  12 +-
 .../org/apache/arrow/vector/ipc/ReadChannel.java   |   2 +-
 .../arrow/vector/ipc/message/ArrowRecordBatch.java |  12 +-
 .../vector/ipc/message/MessageChannelReader.java   |   2 +-
 .../org/apache/arrow/vector/ITTestLargeVector.java |  39 +
 .../vector/ipc/ITTestIPCWithLargeArrowBuffers.java | 187 +
 13 files changed, 294 insertions(+), 72 deletions(-)
 create mode 100644 
java/vector/src/test/java/org/apache/arrow/vector/ipc/ITTestIPCWithLargeArrowBuffers.java

[arrow] branch master updated (47f31d1 -> f79a381)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 47f31d1  ARROW-9000: [Java] Update errorprone to 2.4.0
 add f79a381  ARROW-8972: [Java] Support range value comparison for large 
varchar/varbinary vectors

No new revisions were added by this update.

Summary of changes:
 .../arrow/vector/compare/RangeEqualsVisitor.java   | 41 --
 .../vector/compare/TestRangeEqualsVisitor.java | 21 +++
 2 files changed, 59 insertions(+), 3 deletions(-)

[arrow] branch master updated (7cd7f3b -> 47f31d1)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 7cd7f3b  ARROW-8844: [C++] Transfer bitmap in words
 add 47f31d1  ARROW-9000: [Java] Update errorprone to 2.4.0

No new revisions were added by this update.

Summary of changes:
 java/pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[arrow] branch master updated: ARROW-8696: [Java] Convert tests to maven failsafe

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 93ba086  ARROW-8696: [Java] Convert tests to maven failsafe
93ba086 is described below

commit 93ba086820f5336d67177acd0c35cb11e7c66ece
Author: Ryan Murray 
AuthorDate: Thu May 21 20:49:08 2020 -0700

ARROW-8696: [Java] Convert tests to maven failsafe

Some tests are run via main() and can be run as integration tests
instead. This makes running as part of an automated job easier.

Closes #7100 from rymurr/ARROW-8696

Authored-by: Ryan Murray 
Signed-off-by: Micah Kornfield 
---
 java/README.md | 13 
 java/memory/pom.xml| 32 
 ...LargeArrowBuf.java => ITTestLargeArrowBuf.java} | 38 ++
 java/vector/pom.xml| 36 -
 ...TestLargeVector.java => ITTestLargeVector.java} | 85 +++---
 5 files changed, 145 insertions(+), 59 deletions(-)

diff --git a/java/README.md b/java/README.md
index 9851308..773d2ba 100644
--- a/java/README.md
+++ b/java/README.md
@@ -112,6 +112,19 @@ mvn -Dlogback.configurationFile=file:
 
 See [Logback Configuration][1] for more details.
 
+## Integration Tests
+
+Integration tests which require more time or more memory can be run by 
activating 
+the `integration-tests` profile. This activates the [maven failsafe][4] plugin
+and any class prefixed with `IT` will be run during the testing phase. The 
integration
+tests currently require a larger amount of memory (>4GB) and time to complete. 
To activate 
+the profile:
+
+```bash
+mvn -Pintegration-tests 
+```
+
 [1]: https://logback.qos.ch/manual/configuration.html
 [2]: https://github.com/apache/arrow/blob/master/cpp/README.md
 [3]: http://google.github.io/styleguide/javaguide.html
+[4]: https://maven.apache.org/surefire/maven-failsafe-plugin/
\ No newline at end of file
diff --git a/java/memory/pom.xml b/java/memory/pom.xml
index 45d1c98..b3418e6 100644
--- a/java/memory/pom.xml
+++ b/java/memory/pom.xml
@@ -46,4 +46,36 @@
   
   
 
+  
+
+  
+  integration-tests
+  
+
+  
+org.apache.maven.plugins
+maven-failsafe-plugin
+
+  
+${project.build.directory}
+
true
+UTC
+  
+  
+
+
+  
+
+  integration-test
+  verify
+
+  
+
+  
+
+  
+
+  
 
diff --git 
a/java/memory/src/test/java/org/apache/arrow/memory/TestLargeArrowBuf.java 
b/java/memory/src/test/java/org/apache/arrow/memory/ITTestLargeArrowBuf.java
similarity index 65%
rename from 
java/memory/src/test/java/org/apache/arrow/memory/TestLargeArrowBuf.java
rename to 
java/memory/src/test/java/org/apache/arrow/memory/ITTestLargeArrowBuf.java
index 7d85187..fa8d510 100644
--- a/java/memory/src/test/java/org/apache/arrow/memory/TestLargeArrowBuf.java
+++ b/java/memory/src/test/java/org/apache/arrow/memory/ITTestLargeArrowBuf.java
@@ -19,46 +19,54 @@ package org.apache.arrow.memory;
 
 import static org.junit.Assert.assertEquals;
 
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
 /**
  * Integration test for large (more than 2GB) {@link 
org.apache.arrow.memory.ArrowBuf}.
  * To run this test, please make sure there is at least 4GB memory in the 
system.
- * 
- *   Please note that this is not a standard test case, so please run it by 
manually invoking the
- *   main method.
- * 
  */
-public class TestLargeArrowBuf {
+public class ITTestLargeArrowBuf {
+  private static final Logger logger = 
LoggerFactory.getLogger(ITTestLargeArrowBuf.class);
 
-  private static void testLargeArrowBuf(long bufSize) {
+  private void run(long bufSize) {
 try (BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
  ArrowBuf largeBuf = allocator.buffer(bufSize)) {
   assertEquals(bufSize, largeBuf.capacity());
-  System.out.println("Successfully allocated a buffer with capacity " + 
largeBuf.capacity());
+  logger.trace("Successfully allocated a buffer with capacity {}", 
largeBuf.capacity());
 
   for (long i = 0; i < bufSize / 8; i++) {
 largeBuf.setLong(i * 8, i);
 
 if ((i + 1) % 1 == 0) {
-  System.out.println("Successfully written " + (i + 1) + " long 
words");
+  logger.trace("Successfully written {} long words", i + 1);
 }
   }
-  System.out.println("Successfully written " + (bufSize / 8) + " long 
words");
+  logger.trace("Successfully written {} long wo

[arrow] branch master updated: ARROW-8121: [Java] Enhance code style checking for Java code (add spaces after commas, semi-colons and type casts)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 39cc743  ARROW-8121: [Java] Enhance code style checking for Java code 
(add spaces after commas, semi-colons and type casts)
39cc743 is described below

commit 39cc7434479ddce565e0a0fe35f416a9cd992700
Author: liyafan82 
AuthorDate: Fri May 15 21:58:59 2020 -0700

ARROW-8121: [Java] Enhance code style checking for Java code (add spaces 
after commas, semi-colons and type casts)

This is in response to a discussion in 
https://github.com/apache/arrow/pull/6039#discussion_r375161992

We found the current style checking for Java code is not sufficient. So we 
want to enhace it in a series of "small" steps, in order to avoid having to 
change too many files at once.

In this issue, we add spaces after commas, semi-colons and type casts.

Closes #6622 from liyafan82/fly_0313_style

Authored-by: liyafan82 
Signed-off-by: Micah Kornfield 
---
 .../java/org/apache/arrow/AvroToArrowUtils.java|  2 +-
 .../java/org/apache/arrow/AvroSkipFieldTest.java   | 30 ++--
 .../arrow/adapter/jdbc/JdbcToArrowUtils.java   |  2 +-
 .../apache/arrow/adapter/orc/OrcStripeReader.java  |  2 +-
 .../apache/arrow/adapter/orc/OrcReaderTest.java|  4 +-
 java/dev/checkstyle/checkstyle.xml |  3 ++
 .../java/org/apache/arrow/flight/ArrowMessage.java |  2 +-
 .../gandiva/evaluator/ExpressionRegistry.java  |  2 +-
 .../apache/arrow/gandiva/evaluator/Projector.java  |  2 +-
 .../gandiva/evaluator/SelectionVectorInt16.java|  2 +-
 .../apache/arrow/gandiva/expression/InNode.java|  8 ++--
 .../gandiva/evaluator/ProjectorDecimalTest.java| 10 ++--
 .../arrow/gandiva/evaluator/ProjectorTest.java |  4 +-
 .../main/java/io/netty/buffer/NettyArrowBuf.java   | 22 -
 .../org/apache/arrow/memory/AllocationManager.java |  8 ++--
 .../arrow/memory/AllocationOutcomeDetails.java |  2 +-
 .../java/org/apache/arrow/memory/ArrowBuf.java |  6 +--
 .../java/org/apache/arrow/memory/BufferLedger.java |  8 ++--
 .../arrow/memory/LowCostIdentityHashMap.java   | 16 +++
 .../apache/arrow/memory/util/LargeMemoryUtil.java  |  2 +-
 .../arrow/memory/util/hash/MurmurHasher.java   |  2 +-
 .../java/io/netty/buffer/TestNettyArrowBuf.java|  2 +-
 .../arrow/memory/TestLowCostIdentityHashMap.java   | 10 ++--
 .../arrow/vector/BitVectorHelperBenchmarks.java|  2 +-
 .../org/apache/arrow/vector/Float8Benchmarks.java  |  2 +-
 .../org/apache/arrow/vector/VarCharBenchmarks.java |  2 +-
 .../org/apache/arrow/plasma/PlasmaClientTest.java  | 24 +-
 .../java/org/apache/arrow/util/Collections2.java   |  4 +-
 .../arrow/vector/BaseVariableWidthVector.java  |  2 +-
 .../java/org/apache/arrow/vector/BitVector.java|  2 +-
 .../org/apache/arrow/vector/DecimalVector.java |  2 +-
 .../org/apache/arrow/vector/DurationVector.java|  2 +-
 .../java/org/apache/arrow/vector/UInt1Vector.java  |  2 +-
 .../vector/complex/BaseRepeatedValueVector.java|  6 +--
 .../arrow/vector/complex/EmptyValuePopulator.java  |  2 +-
 .../org/apache/arrow/vector/complex/MapVector.java |  2 +-
 .../vector/complex/NonNullableStructVector.java|  2 +-
 .../vector/dictionary/ListSubfieldEncoder.java |  2 +-
 .../apache/arrow/vector/ipc/JsonFileReader.java|  2 +-
 .../org/apache/arrow/vector/ipc/WriteChannel.java  |  2 +-
 .../arrow/vector/ipc/message/ArrowRecordBatch.java |  2 +-
 .../apache/arrow/vector/util/DecimalUtility.java   |  4 +-
 .../org/apache/arrow/vector/TestBitVector.java |  6 +--
 .../apache/arrow/vector/TestBitVectorHelper.java   |  6 +--
 .../org/apache/arrow/vector/TestDecimalVector.java |  2 +-
 .../apache/arrow/vector/TestDictionaryVector.java  | 38 +++
 .../org/apache/arrow/vector/TestListVector.java| 12 ++---
 .../org/apache/arrow/vector/TestValueVector.java   | 54 +++---
 .../apache/arrow/vector/TestVarCharListVector.java |  2 +-
 .../apache/arrow/vector/TestVectorUnloadLoad.java  |  4 +-
 .../vector/complex/writer/TestComplexWriter.java   | 22 -
 .../org/apache/arrow/vector/ipc/BaseFileTest.java  | 20 
 .../apache/arrow/vector/ipc/TestArrowStream.java   |  2 +-
 .../org/apache/arrow/vector/ipc/TestJSONFile.java  |  2 +-
 .../vector/testing/TestValueVectorPopulator.java   | 12 ++---
 .../vector/validate/TestValidateVectorVisitor.java |  4 +-
 56 files changed, 204 insertions(+), 201 deletions(-)

diff --git 
a/java/adapter/avro/src/main/java/org/apache/arrow/AvroToArrowUtils.java 
b/java/adapter/avro/src/main/java/org/apache/arrow/AvroToArrowUtils.java
index 1d9a06c..1e54f1f 100644
--- a/java/adapter/avro/src/main/java/org/apache/arrow/AvroToArrowUtils.java
+++ b/java/adapter/avro/src/main/java/org/ap

[arrow] branch master updated: ARROW-8811: [Java] Fix CI

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new f94e6ff  ARROW-8811: [Java] Fix CI
f94e6ff is described below

commit f94e6ff61ee305b4d35bfcba1d6b325952f8088d
Author: Micah Kornfield 
AuthorDate: Thu May 14 23:07:24 2020 -0700

ARROW-8811: [Java] Fix CI

Closes #7190 from emkornfield/fix_java

Authored-by: Micah Kornfield 
Signed-off-by: Micah Kornfield 
---
 .../src/main/java/org/apache/arrow/vector/util/ValueVectorUtility.java | 3 +--
 .../java/org/apache/arrow/vector/validate/ValidateVectorVisitor.java   | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git 
a/java/vector/src/main/java/org/apache/arrow/vector/util/ValueVectorUtility.java
 
b/java/vector/src/main/java/org/apache/arrow/vector/util/ValueVectorUtility.java
index f54e208..7c2de7f 100644
--- 
a/java/vector/src/main/java/org/apache/arrow/vector/util/ValueVectorUtility.java
+++ 
b/java/vector/src/main/java/org/apache/arrow/vector/util/ValueVectorUtility.java
@@ -17,6 +17,7 @@
 
 package org.apache.arrow.vector.util;
 
+import org.apache.arrow.memory.ArrowBuf;
 import org.apache.arrow.util.Preconditions;
 import org.apache.arrow.vector.BufferLayout;
 import org.apache.arrow.vector.FieldVector;
@@ -25,8 +26,6 @@ import org.apache.arrow.vector.ValueVector;
 import org.apache.arrow.vector.types.pojo.ArrowType;
 import org.apache.arrow.vector.validate.ValidateVectorVisitor;
 
-import io.netty.buffer.ArrowBuf;
-
 /**
  * Utility methods for {@link ValueVector}.
  */
diff --git 
a/java/vector/src/main/java/org/apache/arrow/vector/validate/ValidateVectorVisitor.java
 
b/java/vector/src/main/java/org/apache/arrow/vector/validate/ValidateVectorVisitor.java
index 1965afd..a6d10c2 100644
--- 
a/java/vector/src/main/java/org/apache/arrow/vector/validate/ValidateVectorVisitor.java
+++ 
b/java/vector/src/main/java/org/apache/arrow/vector/validate/ValidateVectorVisitor.java
@@ -19,6 +19,7 @@ package org.apache.arrow.vector.validate;
 
 import java.util.List;
 
+import org.apache.arrow.memory.ArrowBuf;
 import org.apache.arrow.vector.BaseFixedWidthVector;
 import org.apache.arrow.vector.BaseVariableWidthVector;
 import org.apache.arrow.vector.FieldVector;
@@ -33,8 +34,6 @@ import org.apache.arrow.vector.complex.UnionVector;
 import org.apache.arrow.vector.types.pojo.Field;
 import org.apache.arrow.vector.util.ValueVectorUtility;
 
-import io.netty.buffer.ArrowBuf;
-
 /**
  * Visitor to validate vector (without validating data).
  * This visitor could be used for {@link ValueVector#accept(VectorVisitor, 
Object)} API,

[arrow] branch master updated: ARROW-8169: [Java] Improve the performance of JDBC adapter by allocating memory proactively

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 80dc959  ARROW-8169: [Java] Improve the performance of JDBC adapter by 
allocating memory proactively
80dc959 is described below

commit 80dc95928310c9068b5a1b4bbac9870ac0ba096c
Author: liyafan82 
AuthorDate: Thu May 14 21:36:32 2020 -0700

ARROW-8169: [Java] Improve the performance of JDBC adapter by allocating 
memory proactively

The current implementation use setSafe methods to dynamically allocate 
memory if necessary. For fixed width vectors (which are frequently used in 
JDBC), however, we can allocate memory proactively, since the vector size is 
known as a configuration parameter. So for fixed width vectors, we can use set 
methods instead.

This change leads to two benefits:
1. When processing each value, we no longer have to check vector capacity 
and reallocate memroy if needed. This leads to better performance.
2. If we allow the memory to expand automatically (each time by 2x), the 
amount of memory usually ends up being more than necessary. By allocating 
memory by the configuration parameter, we allocate no more, or no less.

Benchmark results show notable performance improvements:

Before:

Benchmark Mode Cnt Score Error Units
JdbcAdapterBenchmarks.consumeBenchmark avgt 5 521.700 ± 4.837 us/op

After:

Benchmark Mode Cnt Score Error Units
JdbcAdapterBenchmarks.consumeBenchmark avgt 5 430.523 ± 9.932 us/op

Closes #6672 from liyafan82/fly_0319_jdbc

Authored-by: liyafan82 
Signed-off-by: Micah Kornfield 
---
 .../arrow/adapter/jdbc/ArrowVectorIterator.java| 40 +++---
 .../org/apache/arrow/adapter/jdbc/JdbcToArrow.java |  3 ++
 .../adapter/jdbc/JdbcToArrowConfigBuilder.java |  3 ++
 .../arrow/adapter/jdbc/JdbcToArrowUtils.java   | 16 +++--
 .../arrow/adapter/jdbc/consumer/ArrayConsumer.java | 16 +
 .../adapter/jdbc/consumer/BigIntConsumer.java  |  8 +++--
 .../arrow/adapter/jdbc/consumer/BitConsumer.java   |  8 +++--
 .../arrow/adapter/jdbc/consumer/DateConsumer.java  |  8 +++--
 .../adapter/jdbc/consumer/DecimalConsumer.java |  8 +++--
 .../adapter/jdbc/consumer/DoubleConsumer.java  |  8 +++--
 .../arrow/adapter/jdbc/consumer/FloatConsumer.java |  8 +++--
 .../arrow/adapter/jdbc/consumer/IntConsumer.java   |  8 +++--
 .../adapter/jdbc/consumer/SmallIntConsumer.java|  8 +++--
 .../arrow/adapter/jdbc/consumer/TimeConsumer.java  |  8 +++--
 .../adapter/jdbc/consumer/TimestampConsumer.java   |  8 +++--
 .../adapter/jdbc/consumer/TinyIntConsumer.java |  8 +++--
 .../jdbc/h2/JdbcToArrowVectorIteratorTest.java | 12 +++
 .../arrow/adapter/jdbc/JdbcAdapterBenchmarks.java  |  2 +-
 18 files changed, 150 insertions(+), 30 deletions(-)

diff --git 
a/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java
 
b/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java
index 39d92ff..973bdf1 100644
--- 
a/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java
+++ 
b/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java
@@ -25,6 +25,8 @@ import java.util.Iterator;
 import org.apache.arrow.adapter.jdbc.consumer.CompositeJdbcConsumer;
 import org.apache.arrow.adapter.jdbc.consumer.JdbcConsumer;
 import org.apache.arrow.util.Preconditions;
+import org.apache.arrow.vector.BaseFixedWidthVector;
+import org.apache.arrow.vector.ValueVector;
 import org.apache.arrow.vector.VectorSchemaRoot;
 import org.apache.arrow.vector.types.pojo.Schema;
 
@@ -92,12 +94,20 @@ public class ArrowVectorIterator implements 
Iterator, AutoClos
 // consume data
 try {
   int readRowCount = 0;
-  while ((targetBatchSize == JdbcToArrowConfig.NO_LIMIT_BATCH_SIZE || 
readRowCount < targetBatchSize) &&
-  resultSet.next()) {
-compositeConsumer.consume(resultSet);
-readRowCount++;
+  if (targetBatchSize == JdbcToArrowConfig.NO_LIMIT_BATCH_SIZE) {
+while (resultSet.next()) {
+  ensureCapacity(root, readRowCount + 1);
+  compositeConsumer.consume(resultSet);
+  readRowCount++;
+}
+  } else {
+while (readRowCount < targetBatchSize && resultSet.next()) {
+  compositeConsumer.consume(resultSet);
+  readRowCount++;
+}
   }
 
+
   root.setRowCount(readRowCount);
 } catch (Exception e) {
   compositeConsumer.close();
@@ -109,6 +119,9 @@ public class ArrowVectorIterator implements 
Iterator, AutoClos
 VectorSchemaRoot root = null;
 try {
   root = VectorSchemaRoot.create(schema, config.getAllocator());
+  if (con

[arrow] branch master updated: ARROW-8481: [Java] Provide an allocation manager based on Unsafe API

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 3aa3339  ARROW-8481: [Java] Provide an allocation manager based on 
Unsafe API
3aa3339 is described below

commit 3aa33392e7ca11c48c09a963d165275b26280157
Author: liyafan82 
AuthorDate: Thu May 14 21:24:48 2020 -0700

ARROW-8481: [Java] Provide an allocation manager based on Unsafe API

This is in response to the discussion in 
https://github.com/apache/arrow/pull/6323#issuecomment-614195070

In this issue, we provide an allocation manager that is capable of 
allocation large (> 2GB) buffers. In addition, it does not depend on the netty 
library, which is aligning with the general trend of removing netty 
dependencies. In the future, we are going to make it the default allocation 
manager.

Closes #6956 from liyafan82/fly_0416_unsf

Lead-authored-by: liyafan82 
Co-authored-by: emkornfield 
Signed-off-by: Micah Kornfield 
---
 .../org/apache/arrow/memory/BaseAllocator.java |  2 +-
 .../memory/DefaultAllocationManagerOption.java | 98 ++
 .../arrow/memory/UnsafeAllocationManager.java  | 65 ++
 .../apache/arrow/memory/TestAllocationManager.java | 52 
 .../arrow/memory/TestUnsafeAllocationManager.java  | 70 
 5 files changed, 286 insertions(+), 1 deletion(-)

diff --git 
a/java/memory/src/main/java/org/apache/arrow/memory/BaseAllocator.java 
b/java/memory/src/main/java/org/apache/arrow/memory/BaseAllocator.java
index 57ee58b..1b9b6b1 100644
--- a/java/memory/src/main/java/org/apache/arrow/memory/BaseAllocator.java
+++ b/java/memory/src/main/java/org/apache/arrow/memory/BaseAllocator.java
@@ -763,7 +763,7 @@ public abstract class BaseAllocator extends Accountant 
implements BufferAllocato
  */
 @Value.Default
 AllocationManager.Factory getAllocationManagerFactory() {
-  return NettyAllocationManager.FACTORY;
+  return DefaultAllocationManagerOption.DEFAULT_ALLOCATION_MANAGER_FACTORY;
 }
 
 /**
diff --git 
a/java/memory/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java
 
b/java/memory/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java
new file mode 100644
index 000..66f0ec6
--- /dev/null
+++ 
b/java/memory/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.memory;
+
+/**
+ * A class for choosing the default allocation manager.
+ */
+public class DefaultAllocationManagerOption {
+
+  /**
+   * The environmental variable to set the default allocation manager type.
+   */
+  public static final String ALLOCATION_MANAGER_TYPE_ENV_NAME = 
"ARROW_ALLOCATION_MANAGER_TYPE";
+
+  /**
+   * The system property to set the default allocation manager type.
+   */
+  public static final String ALLOCATION_MANAGER_TYPE_PROPERTY_NAME = 
"arrow.allocation.manager.type";
+
+  static final org.slf4j.Logger LOGGER = 
org.slf4j.LoggerFactory.getLogger(DefaultAllocationManagerOption.class);
+
+  /**
+   * The default allocation manager factory.
+   */
+  public static final AllocationManager.Factory 
DEFAULT_ALLOCATION_MANAGER_FACTORY =
+  getDefaultAllocationManagerFactory();
+
+  /**
+   * The allocation manager type.
+   */
+  public enum AllocationManagerType {
+/**
+ * Netty based allocation manager.
+ */
+Netty,
+
+/**
+ * Unsafe based allocation manager.
+ */
+Unsafe,
+
+/**
+ * Unknown type.
+ */
+Unknown,
+  }
+
+  static AllocationManagerType getDefaultAllocationManagerType() {
+AllocationManagerType ret = AllocationManagerType.Unknown;
+
+try {
+  String envValue = System.getenv(ALLOCATION_MANAGER_TYPE_ENV_NAME);
+  ret = AllocationManagerType.valueOf(envValue);
+} catch (IllegalArgumentException | NullPointerException e) {
+  // ignore the exception, and make the allocation manager type remain 

[arrow] branch master updated (1dcf433 -> 799ea4f)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 1dcf433  ARROW-7699: [Java] Support concating dense union vectors in 
batch
 add 799ea4f  ARROW-8108: [Java] Extract a common interface for dictionary 
encoders

No new revisions were added by this update.

Summary of changes:
 .../algorithm/dictionary/DictionaryEncoder.java}   | 30 ++
 .../dictionary/HashTableDictionaryEncoder.java |  4 ++-
 .../dictionary/LinearDictionaryEncoder.java|  4 ++-
 .../dictionary/SearchDictionaryEncoder.java|  4 ++-
 4 files changed, 22 insertions(+), 20 deletions(-)
 copy 
java/{adapter/avro/src/main/java/org/apache/arrow/consumers/AvroEnumConsumer.java
 => 
algorithm/src/main/java/org/apache/arrow/algorithm/dictionary/DictionaryEncoder.java}
 (51%)

[arrow] branch master updated (a551617 -> 8a0c24d)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from a551617  ARROW-8664: [Java] Add flag to skip null check
 add 8a0c24d  ARROW-8020: [Java] Implement vector validate functionality

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/UnionVector.java|   1 -
 .../arrow/vector/util/ValueVectorUtility.java  |  47 +
 .../vector/validate/ValidateVectorVisitor.java | 221 +++
 .../vector/testing/ValueVectorDataPopulator.java   |   6 +-
 .../vector/validate/TestValidateVectorVisitor.java | 235 +
 5 files changed, 507 insertions(+), 3 deletions(-)
 create mode 100644 
java/vector/src/main/java/org/apache/arrow/vector/validate/ValidateVectorVisitor.java
 create mode 100644 
java/vector/src/test/java/org/apache/arrow/vector/validate/TestValidateVectorVisitor.java

[arrow] branch master updated (8a0c24d -> 1dcf433)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 8a0c24d  ARROW-8020: [Java] Implement vector validate functionality
 add 1dcf433  ARROW-7699: [Java] Support concating dense union vectors in 
batch

No new revisions were added by this update.

Summary of changes:
 .../main/codegen/templates/DenseUnionVector.java   |  22 ++-
 .../java/org/apache/arrow/vector/types/Types.java  |  28 ++--
 .../apache/arrow/vector/util/VectorAppender.java   |  94 ++--
 .../apache/arrow/vector/TestDenseUnionVector.java  |  78 +-
 .../arrow/vector/util/TestVectorAppender.java  | 160 +
 5 files changed, 355 insertions(+), 27 deletions(-)

[arrow] branch master updated (d8b8cc7 -> a551617)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d8b8cc7  ARROW-8555: [FlightRPC][Java] implement DoExchange
 add a551617  ARROW-8664: [Java] Add flag to skip null check

No new revisions were added by this update.

Summary of changes:
 java/vector/src/main/java/org/apache/arrow/vector/DurationVector.java | 3 ++-
 .../src/main/java/org/apache/arrow/vector/FixedSizeBinaryVector.java  | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampMicroTZVector.java | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampMicroVector.java   | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampMilliTZVector.java | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampMilliVector.java   | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampNanoTZVector.java  | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampNanoVector.java| 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampSecTZVector.java   | 4 +++-
 .../src/main/java/org/apache/arrow/vector/TimeStampSecVector.java | 4 +++-
 .../vector/src/main/java/org/apache/arrow/vector/VarBinaryVector.java | 4 +++-
 java/vector/src/main/java/org/apache/arrow/vector/VarCharVector.java  | 4 +++-
 12 files changed, 35 insertions(+), 12 deletions(-)

[arrow] branch master updated: ARROW-8555: [FlightRPC][Java] implement DoExchange

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new d8b8cc7  ARROW-8555: [FlightRPC][Java] implement DoExchange
d8b8cc7 is described below

commit d8b8cc7d714a6e7eae4eb4ebc4edc3649de4bef2
Author: David Li 
AuthorDate: Thu May 14 20:48:23 2020 -0700

ARROW-8555: [FlightRPC][Java] implement DoExchange

This is a complete implementation of DoExchange for Java. It is not tested 
against the C++ implementation yet, however, it still passes integration tests, 
so the internal refactoring should not have broken compatibility with existing 
clients/servers.

In this PR, I've refactored DoGet/DoPut/DoExchange on the client and server 
to share their implementation as much as possible. DoGet/DoPut retain their 
behavior of "eagerly" reading/writing schemas, but DoExchange allows the 
client/server to delay writing the schema until ready. This is checked in the 
unit tests.

I also ran into some test flakes and tried to address them, by making sure 
we clean up things in the right order, and adding missing `close()` calls in 
some existing tests.

Closes #7012 from lidavidm/doexchange-java

Authored-by: David Li 
Signed-off-by: Micah Kornfield 
---
 java/flight/flight-core/pom.xml|   8 +
 .../java/org/apache/arrow/flight/ArrowMessage.java |  35 +-
 .../java/org/apache/arrow/flight/CallStatus.java   |   2 +-
 .../apache/arrow/flight/FlightBindingService.java  |  35 +-
 .../java/org/apache/arrow/flight/FlightClient.java | 197 ++
 .../java/org/apache/arrow/flight/FlightMethod.java |   3 +
 .../org/apache/arrow/flight/FlightProducer.java|  50 +--
 .../java/org/apache/arrow/flight/FlightServer.java |   6 +-
 .../org/apache/arrow/flight/FlightService.java | 175 +
 .../java/org/apache/arrow/flight/FlightStream.java | 105 --
 .../arrow/flight/OutboundStreamListener.java   |  82 +
 .../arrow/flight/OutboundStreamListenerImpl.java   | 119 +++
 .../apache/arrow/flight/TestBasicOperation.java|  19 +-
 .../org/apache/arrow/flight/TestDoExchange.java| 395 +
 .../org/apache/arrow/flight/TestErrorMetadata.java |  10 +-
 .../org/apache/arrow/flight/TestServerOptions.java |  17 +-
 16 files changed, 1018 insertions(+), 240 deletions(-)

diff --git a/java/flight/flight-core/pom.xml b/java/flight/flight-core/pom.xml
index 8301c71..43ac6cc 100644
--- a/java/flight/flight-core/pom.xml
+++ b/java/flight/flight-core/pom.xml
@@ -132,6 +132,14 @@
   1.12.0
   test
 
+
+  org.apache.arrow
+  arrow-vector
+  ${project.version}
+  tests
+  test-jar
+  test
+
   
   
 
diff --git 
a/java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
 
b/java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
index fd59dd5..1758215 100644
--- 
a/java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
+++ 
b/java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java
@@ -154,6 +154,24 @@ class ArrowMessage implements AutoCloseable {
 this.appMetadata = null;
   }
 
+  /**
+   * Create an ArrowMessage containing only application metadata.
+   * @param appMetadata The application-provided metadata buffer.
+   */
+  public ArrowMessage(ArrowBuf appMetadata) {
+this.message = null;
+this.bufs = ImmutableList.of();
+this.descriptor = null;
+this.appMetadata = appMetadata;
+  }
+
+  public ArrowMessage(FlightDescriptor descriptor) {
+this.message = null;
+this.bufs = ImmutableList.of();
+this.descriptor = descriptor;
+this.appMetadata = null;
+  }
+
   private ArrowMessage(FlightDescriptor descriptor, MessageMetadataResult 
message, ArrowBuf appMetadata,
ArrowBuf buf) {
 this.message = message;
@@ -171,6 +189,10 @@ class ArrowMessage implements AutoCloseable {
   }
 
   public HeaderType getMessageType() {
+if (message == null) {
+  // Null message occurs for metadata-only messages (in DoExchange)
+  return HeaderType.NONE;
+}
 return HeaderType.getHeader(message.headerType());
   }
 
@@ -271,8 +293,19 @@ class ArrowMessage implements AutoCloseable {
* @return InputStream
*/
   private InputStream asInputStream(BufferAllocator allocator) {
-try {
+if (message == null) {
+  // If we have no IPC message, it's a pure-metadata message
+  final FlightData.Builder builder = FlightData.newBuilder();
+  if (descriptor != null) {
+builder.setFlightDescriptor(descriptor);
+  }
+  if (appMetadata != null) {
+builder.setAppMetadata(ByteString.copyFrom(appMetadata.nioBuffer()));
+  }
+  return NO_BODY_MARSHALLER.stream(builder.build());
+}

[arrow] branch master updated: ARROW-8634: [Java] Add Getting Started section to Java README

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 1f175e1  ARROW-8634: [Java] Add Getting Started section to Java README
1f175e1 is described below

commit 1f175e12a101960fd5b293a2dc366855eaaad141
Author: Andy Grove 
AuthorDate: Thu Apr 30 21:42:51 2020 -0700

ARROW-8634: [Java] Add Getting Started section to Java README

Currently, it is difficult to get started with Arrow Java because there are 
no examples, and no "getting started" guide.

This PR adds a couple of trivial examples as a starting point.

Closes #7066 from andygrove/ARROW-8634

Authored-by: Andy Grove 
Signed-off-by: Micah Kornfield 
---
 java/README.md | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/java/README.md b/java/README.md
index f312636..9851308 100644
--- a/java/README.md
+++ b/java/README.md
@@ -19,6 +19,16 @@
 
 # Arrow Java
 
+## Getting Started
+
+The following guides explain the fundamental data structures used in the Java 
implementation of Apache Arrow.
+
+- [ValueVector](https://arrow.apache.org/docs/java/vector.html) is an 
abstraction that is used to store a sequence of values having the same type in 
an individual column.
+- 
[VectorSchemaRoot](https://arrow.apache.org/docs/java/vector_schema_root.html) 
is a container that can hold multiple vectors based on a schema. 
+- The [Reading/Writing IPC 
formats](https://arrow.apache.org/docs/java/ipc.html) guide explains how to 
stream record batches as well as serializing record batches to files.
+
+Generated javadoc documentation is available 
[here](https://arrow.apache.org/docs/java/).
+
 ## Setup Build Environment
 
 install:

[arrow] branch master updated (dfc14ef -> 194dc15)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from dfc14ef  ARROW-2260: [C++][Plasma] Use Gflags for command-line parsing
 add 194dc15  ARROW-8392: [Java] Fix overflow related corner cases for 
vector value comparison

No new revisions were added by this update.

Summary of changes:
 .../algorithm/sort/DefaultVectorComparators.java   |  10 +-
 .../sort/TestDefaultVectorComparator.java  | 142 +++--
 2 files changed, 138 insertions(+), 14 deletions(-)

[arrow] branch master updated (283e188 -> dfc14ef)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 283e188  ARROW-3861: [Python] ParquetDataset.read() respect specified 
columns and not include partition columns
 add dfc14ef  ARROW-2260: [C++][Plasma] Use Gflags for command-line parsing

No new revisions were added by this update.

Summary of changes:
 cpp/src/arrow/util/config.h.cmake |   2 +
 cpp/src/plasma/CMakeLists.txt |   1 +
 cpp/src/plasma/store.cc   | 122 --
 3 files changed, 80 insertions(+), 45 deletions(-)

[arrow] branch master updated: ARROW-8478: [Java] Revert "ARROW-7534

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new f7fb49c  ARROW-8478: [Java] Revert "ARROW-7534
f7fb49c is described below

commit f7fb49cfa19fe2d39dd54a426b1288d33342faf5
Author: Micah Kornfield 
AuthorDate: Wed Apr 15 23:11:07 2020 -0700

ARROW-8478: [Java] Revert "ARROW-7534

This reverts commit 5a94b5d6f6c170a02ac1da2dcf98c5dbc8ea1ccb.

I don't think this will unblock the release candidate but we should wait 
until the release is cut.

Closes #6955 from emkornfield/revert_contrib

Authored-by: Micah Kornfield 
Signed-off-by: Micah Kornfield 
---
 ci/scripts/java_build.sh   |  2 +-
 ci/scripts/java_test.sh|  2 +-
 cpp/src/jni/orc/CMakeLists.txt |  2 +-
 java/{contrib => }/adapter/avro/pom.xml|  2 +-
 .../avro/src/main/java/org/apache/arrow/AvroToArrow.java   |  0
 .../avro/src/main/java/org/apache/arrow/AvroToArrowConfig.java |  0
 .../main/java/org/apache/arrow/AvroToArrowConfigBuilder.java   |  0
 .../avro/src/main/java/org/apache/arrow/AvroToArrowUtils.java  |  0
 .../main/java/org/apache/arrow/AvroToArrowVectorIterator.java  |  0
 .../java/org/apache/arrow/consumers/AvroArraysConsumer.java|  0
 .../java/org/apache/arrow/consumers/AvroBooleanConsumer.java   |  0
 .../java/org/apache/arrow/consumers/AvroBytesConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroDoubleConsumer.java|  0
 .../main/java/org/apache/arrow/consumers/AvroEnumConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroFixedConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroFloatConsumer.java |  0
 .../main/java/org/apache/arrow/consumers/AvroIntConsumer.java  |  0
 .../main/java/org/apache/arrow/consumers/AvroLongConsumer.java |  0
 .../main/java/org/apache/arrow/consumers/AvroMapConsumer.java  |  0
 .../main/java/org/apache/arrow/consumers/AvroNullConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroStringConsumer.java|  0
 .../java/org/apache/arrow/consumers/AvroStructConsumer.java|  0
 .../java/org/apache/arrow/consumers/AvroUnionsConsumer.java|  0
 .../main/java/org/apache/arrow/consumers/BaseAvroConsumer.java |  0
 .../java/org/apache/arrow/consumers/CompositeAvroConsumer.java |  0
 .../src/main/java/org/apache/arrow/consumers/Consumer.java |  0
 .../src/main/java/org/apache/arrow/consumers/SkipConsumer.java |  0
 .../src/main/java/org/apache/arrow/consumers/SkipFunction.java |  0
 .../org/apache/arrow/consumers/logical/AvroDateConsumer.java   |  0
 .../apache/arrow/consumers/logical/AvroDecimalConsumer.java|  0
 .../apache/arrow/consumers/logical/AvroTimeMicroConsumer.java  |  0
 .../apache/arrow/consumers/logical/AvroTimeMillisConsumer.java |  0
 .../arrow/consumers/logical/AvroTimestampMicrosConsumer.java   |  0
 .../arrow/consumers/logical/AvroTimestampMillisConsumer.java   |  0
 .../src/test/java/org/apache/arrow/AvroLogicalTypesTest.java   |  0
 .../avro/src/test/java/org/apache/arrow/AvroSkipFieldTest.java |  0
 .../avro/src/test/java/org/apache/arrow/AvroTestBase.java  |  0
 .../test/java/org/apache/arrow/AvroToArrowIteratorTest.java|  0
 .../avro/src/test/java/org/apache/arrow/AvroToArrowTest.java   |  0
 .../test/java/org/apache/arrow/TestWriteReadAvroRecord.java|  0
 .../avro/src/test/resources/schema/attrs/test_enum_attrs.avsc  |  0
 .../avro/src/test/resources/schema/attrs/test_fixed_attr.avsc  |  0
 .../src/test/resources/schema/attrs/test_record_attrs.avsc |  0
 .../avro/src/test/resources/schema/logical/test_date.avsc  |  0
 .../test/resources/schema/logical/test_decimal_invalid1.avsc   |  0
 .../test/resources/schema/logical/test_decimal_invalid2.avsc   |  0
 .../test/resources/schema/logical/test_decimal_invalid3.avsc   |  0
 .../test/resources/schema/logical/test_decimal_invalid4.avsc   |  0
 .../schema/logical/test_decimal_with_original_bytes.avsc   |  0
 .../schema/logical/test_decimal_with_original_fixed.avsc   |  0
 .../src/test/resources/schema/logical/test_time_micros.avsc|  0
 .../src/test/resources/schema/logical/test_time_millis.avsc|  0
 .../test/resources/schema/logical/test_timestamp_micros.avsc   |  0
 .../test/resources/schema/logical/test_timestamp_millis.avsc   |  0
 .../src/test/resources/schema/skip/test_skip_array_before.avsc |  0
 .../test/resources/schema/skip/test_skip_array_expected.avsc   |  0
 .../avro/src/test/resources/schema/skip/test_skip_base1.avsc   |  0
 .../avro/src/test/resources/schema/skip/test_skip_base2.avsc   |  0
 .../test/resources/schema/skip/test_skip_boolean_expected.avsc |  0
 .../test/resources/schema/skip/test_skip_bytes_expected.avsc   |  0
 .../test/resource

[arrow] branch master updated (83462a1 -> 5a94b5d)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 83462a1  ARROW-590: [Integration][C++] Implement union types
 add 5a94b5d  ARROW-7534: [Java] Create a new java/contrib module

No new revisions were added by this update.

Summary of changes:
 ci/scripts/java_build.sh   |  2 +-
 ci/scripts/java_test.sh|  2 +-
 cpp/src/jni/orc/CMakeLists.txt |  2 +-
 java/{ => contrib}/adapter/avro/pom.xml|  2 +-
 .../avro/src/main/java/org/apache/arrow/AvroToArrow.java   |  0
 .../avro/src/main/java/org/apache/arrow/AvroToArrowConfig.java |  0
 .../main/java/org/apache/arrow/AvroToArrowConfigBuilder.java   |  0
 .../avro/src/main/java/org/apache/arrow/AvroToArrowUtils.java  |  0
 .../main/java/org/apache/arrow/AvroToArrowVectorIterator.java  |  0
 .../java/org/apache/arrow/consumers/AvroArraysConsumer.java|  0
 .../java/org/apache/arrow/consumers/AvroBooleanConsumer.java   |  0
 .../java/org/apache/arrow/consumers/AvroBytesConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroDoubleConsumer.java|  0
 .../main/java/org/apache/arrow/consumers/AvroEnumConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroFixedConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroFloatConsumer.java |  0
 .../main/java/org/apache/arrow/consumers/AvroIntConsumer.java  |  0
 .../main/java/org/apache/arrow/consumers/AvroLongConsumer.java |  0
 .../main/java/org/apache/arrow/consumers/AvroMapConsumer.java  |  0
 .../main/java/org/apache/arrow/consumers/AvroNullConsumer.java |  0
 .../java/org/apache/arrow/consumers/AvroStringConsumer.java|  0
 .../java/org/apache/arrow/consumers/AvroStructConsumer.java|  0
 .../java/org/apache/arrow/consumers/AvroUnionsConsumer.java|  0
 .../main/java/org/apache/arrow/consumers/BaseAvroConsumer.java |  0
 .../java/org/apache/arrow/consumers/CompositeAvroConsumer.java |  0
 .../src/main/java/org/apache/arrow/consumers/Consumer.java |  0
 .../src/main/java/org/apache/arrow/consumers/SkipConsumer.java |  0
 .../src/main/java/org/apache/arrow/consumers/SkipFunction.java |  0
 .../org/apache/arrow/consumers/logical/AvroDateConsumer.java   |  0
 .../apache/arrow/consumers/logical/AvroDecimalConsumer.java|  0
 .../apache/arrow/consumers/logical/AvroTimeMicroConsumer.java  |  0
 .../apache/arrow/consumers/logical/AvroTimeMillisConsumer.java |  0
 .../arrow/consumers/logical/AvroTimestampMicrosConsumer.java   |  0
 .../arrow/consumers/logical/AvroTimestampMillisConsumer.java   |  0
 .../src/test/java/org/apache/arrow/AvroLogicalTypesTest.java   |  0
 .../avro/src/test/java/org/apache/arrow/AvroSkipFieldTest.java |  0
 .../avro/src/test/java/org/apache/arrow/AvroTestBase.java  |  0
 .../test/java/org/apache/arrow/AvroToArrowIteratorTest.java|  0
 .../avro/src/test/java/org/apache/arrow/AvroToArrowTest.java   |  0
 .../test/java/org/apache/arrow/TestWriteReadAvroRecord.java|  0
 .../avro/src/test/resources/schema/attrs/test_enum_attrs.avsc  |  0
 .../avro/src/test/resources/schema/attrs/test_fixed_attr.avsc  |  0
 .../src/test/resources/schema/attrs/test_record_attrs.avsc |  0
 .../avro/src/test/resources/schema/logical/test_date.avsc  |  0
 .../test/resources/schema/logical/test_decimal_invalid1.avsc   |  0
 .../test/resources/schema/logical/test_decimal_invalid2.avsc   |  0
 .../test/resources/schema/logical/test_decimal_invalid3.avsc   |  0
 .../test/resources/schema/logical/test_decimal_invalid4.avsc   |  0
 .../schema/logical/test_decimal_with_original_bytes.avsc   |  0
 .../schema/logical/test_decimal_with_original_fixed.avsc   |  0
 .../src/test/resources/schema/logical/test_time_micros.avsc|  0
 .../src/test/resources/schema/logical/test_time_millis.avsc|  0
 .../test/resources/schema/logical/test_timestamp_micros.avsc   |  0
 .../test/resources/schema/logical/test_timestamp_millis.avsc   |  0
 .../src/test/resources/schema/skip/test_skip_array_before.avsc |  0
 .../test/resources/schema/skip/test_skip_array_expected.avsc   |  0
 .../avro/src/test/resources/schema/skip/test_skip_base1.avsc   |  0
 .../avro/src/test/resources/schema/skip/test_skip_base2.avsc   |  0
 .../test/resources/schema/skip/test_skip_boolean_expected.avsc |  0
 .../test/resources/schema/skip/test_skip_bytes_expected.avsc   |  0
 .../test/resources/schema/skip/test_skip_double_expected.avsc  |  0
 .../test/resources/schema/skip/test_skip_enum_expected.avsc|  0
 .../test/resources/schema/skip/test_skip_fixed_expected.avsc   |  0
 .../test/resources/schema/skip/test_skip_float_expected.avsc   |  0
 .../src/test/resources/schema/skip/test_skip_int_expected.avsc |  0
 .../test/resources/schema/skip/test_skip_long_expected.avsc|  0
 .../src/test/resources/schema/s

[arrow] branch master updated: ARROW-8158: [Java] Getting length of data buffer and base variable width vector

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new a2376a2  ARROW-8158: [Java] Getting length of data buffer and base 
variable width vector
a2376a2 is described below

commit a2376a22b3bad46da5a800bfeaac48977e718099
Author: tianchen 
AuthorDate: Fri Apr 10 22:27:45 2020 -0700

ARROW-8158: [Java] Getting length of data buffer and base variable width 
vector

For string data buffer and base variable width vector can we have a way to 
get length of the data?

For instance, in ArrowColumnVector in StringAccessor we use 
stringResult.start and stringResult.end, instead we would like to get length of 
the data through an exposed function.

Now we have getStartOffset in BaseVariableVector and 
getElementStartIndex/getElementEndIndex in BaseListVector. To be consistent, we 
add getEndOffset for BaseVariableVector.

Closes #6684 from tianchen92/ARROW-8158

Authored-by: tianchen 
Signed-off-by: Micah Kornfield 
---
 .../arrow/vector/BaseVariableWidthVector.java  |  9 +++-
 .../org/apache/arrow/vector/TestValueVector.java   | 24 ++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git 
a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthVector.java
 
b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthVector.java
index 5a595cb..0b8cbb9 100644
--- 
a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthVector.java
+++ 
b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthVector.java
@@ -1215,7 +1215,7 @@ public abstract class BaseVariableWidthVector extends 
BaseValueVector
 return getStartOffset(index);
   }
 
-  protected final int getStartOffset(int index) {
+  public final int getStartOffset(int index) {
 return offsetBuffer.getInt(index * OFFSET_WIDTH);
   }
 
@@ -1393,4 +1393,11 @@ public abstract class BaseVariableWidthVector extends 
BaseValueVector
   public  OUT accept(VectorVisitor visitor, IN value) {
 return visitor.visit(this, value);
   }
+
+  /**
+   * Gets the ending offset of a record, given its index.
+   */
+  public final int getEndOffset(int index) {
+return offsetBuffer.getInt((index + 1) * OFFSET_WIDTH);
+  }
 }
diff --git 
a/java/vector/src/test/java/org/apache/arrow/vector/TestValueVector.java 
b/java/vector/src/test/java/org/apache/arrow/vector/TestValueVector.java
index 65bc963..61cf853 100644
--- a/java/vector/src/test/java/org/apache/arrow/vector/TestValueVector.java
+++ b/java/vector/src/test/java/org/apache/arrow/vector/TestValueVector.java
@@ -2701,4 +2701,28 @@ public class TestValueVector {
 }
 writer.endList();
   }
+
+  @Test
+  public void testVariableVectorGetEndOffset() {
+try (final VarCharVector vector1 = new VarCharVector("v1", allocator);
+ final VarBinaryVector vector2 = new VarBinaryVector("v2", allocator)) 
{
+
+  setVector(vector1, STR1, null, STR2);
+  setVector(vector2, STR1, STR2, STR3);
+
+  assertEquals(0, vector1.getStartOffset(0));
+  assertEquals(STR1.length, vector1.getEndOffset(0));
+  assertEquals(STR1.length, vector1.getStartOffset(1));
+  assertEquals(STR1.length, vector1.getEndOffset(1));
+  assertEquals(STR1.length, vector1.getStartOffset(2));
+  assertEquals(STR1.length + STR2.length, vector1.getEndOffset(2));
+
+  assertEquals(0, vector2.getStartOffset(0));
+  assertEquals(STR1.length, vector2.getEndOffset(0));
+  assertEquals(STR1.length, vector2.getStartOffset(1));
+  assertEquals(STR1.length + STR2.length, vector2.getEndOffset(1));
+  assertEquals(STR1.length + STR2.length, vector2.getStartOffset(2));
+  assertEquals(STR1.length + STR2.length + STR3.length, 
vector2.getEndOffset(2));
+}
+  }
 }

[arrow] branch master updated (e686f71 -> af17e44)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from e686f71  ARROW-7048: [Java] Support for combining multiple vectors 
under VectorSchemaRoot
 add af17e44  ARROW-7935: [Java] Remove Netty dependency for 
BufferAllocator and ReferenceManager

No new revisions were added by this update.

Summary of changes:
 .../main/java/io/netty/buffer/PooledByteBufAllocatorL.java  | 13 +++--
 .../java/org/apache/arrow/memory/ArrowByteBufAllocator.java |  3 +++
 .../main/java/org/apache/arrow/memory/BaseAllocator.java| 11 ---
 .../main/java/org/apache/arrow/memory/BufferAllocator.java  |  3 +++
 .../src/main/java/org/apache/arrow/memory/BufferLedger.java |  3 ++-
 5 files changed, 19 insertions(+), 14 deletions(-)

[arrow] branch master updated (a68b68d -> e686f71)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from a68b68d  ARROW-7806: [Python] Support LargeListArray and 
list conversion to pandas.
 add e686f71  ARROW-7048: [Java] Support for combining multiple vectors 
under VectorSchemaRoot

No new revisions were added by this update.

Summary of changes:
 .../vector/util/VectorSchemaRootAppender.java  |  83 +++
 .../vector/util/TestVectorSchemaRootAppender.java  | 161 +
 2 files changed, 244 insertions(+)
 create mode 100644 
java/vector/src/main/java/org/apache/arrow/vector/util/VectorSchemaRootAppender.java
 create mode 100644 
java/vector/src/test/java/org/apache/arrow/vector/util/TestVectorSchemaRootAppender.java

[arrow] branch master updated (e92416f -> 0f36697)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from e92416f  ARROW-8009: [Java] Fix the hash code methods for BitVector
 add 0f36697  ARROW-7837 [JAVA] copyFromSafe fails due to a bug in 
handleSafe

No new revisions were added by this update.

Summary of changes:
 .../vector/VariableWidthVectorBenchmarks.java  | 40 ++
 .../arrow/vector/BaseVariableWidthVector.java  | 12 +++
 .../org/apache/arrow/vector/VarBinaryVector.java   | 11 +++---
 .../org/apache/arrow/vector/VarCharVector.java | 12 +++
 .../java/org/apache/arrow/vector/TestCopyFrom.java | 32 +
 5 files changed, 89 insertions(+), 18 deletions(-)

[arrow] branch master updated (5ffbf0a -> e92416f)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 5ffbf0a  ARROW-8016: [Developer] Fix jira-python deprecation warning 
in merge_arrow_pr.py
 add e92416f  ARROW-8009: [Java] Fix the hash code methods for BitVector

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/vector/BitVector.java| 24 +++
 .../org/apache/arrow/vector/TestBitVector.java | 35 ++
 2 files changed, 59 insertions(+)

[arrow] branch master updated (946beaa -> 7e0802b)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 946beaa  ARROW-7916: [C++] Project IPC batches to materialized fields 
only
 add 7e0802b  ARROW-7958: [Java] Update Avro to version 1.9.2

No new revisions were added by this update.

Summary of changes:
 java/adapter/avro/pom.xml | 2 +-
 java/performance/pom.xml  | 2 +-
 java/pom.xml  | 1 +
 3 files changed, 3 insertions(+), 2 deletions(-)

[arrow] branch master updated (38504e3 -> 45a7047)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 38504e3  ARROW-7400: [Java] Avoid the worst case for quick sort
 add 45a7047  ARROW-7546: [Java] Use new implementation to concat vectors 
values in batch

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/vector/ipc/ArrowReader.java | 19 +++
 1 file changed, 3 insertions(+), 16 deletions(-)

[arrow] branch master updated (389d38b -> 38504e3)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 389d38b  ARROW-7725: [C++] Add infrastructure for unity builds and 
precompiled headers
 add 38504e3  ARROW-7400: [Java] Avoid the worst case for quick sort

No new revisions were added by this update.

Summary of changes:
 java/algorithm/pom.xml |   6 ++
 .../sort/FixedWidthInPlaceVectorSorter.java|  68 +++-
 .../apache/arrow/algorithm/sort/IndexSorter.java   |  59 ++-
 .../arrow/algorithm/sort/InsertionSorter.java  |  74 +
 .../sort/TestFixedWidthInPlaceVectorSorter.java|  95 +
 .../arrow/algorithm/sort/TestIndexSorter.java  | 109 +--
 .../arrow/algorithm/sort/TestInsertionSorter.java  | 117 +
 7 files changed, 513 insertions(+), 15 deletions(-)
 create mode 100644 
java/algorithm/src/main/java/org/apache/arrow/algorithm/sort/InsertionSorter.java
 create mode 100644 
java/algorithm/src/test/java/org/apache/arrow/algorithm/sort/TestInsertionSorter.java

[arrow] branch master updated (4b54997 -> bc261d1)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 4b54997  ARROW-7726: [CI] [C++] Use boost binaries on Windows GHA build
 add bc261d1  ARROW-7073: [Java] Support concating vectors values in batch

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/UnionVector.java|  12 +-
 .../apache/arrow/vector/util/VectorAppender.java   | 326 +++
 ...{TransferPair.java => VectorBatchAppender.java} |  22 +-
 .../vector/testing/ValueVectorDataPopulator.java   |  73 -
 .../arrow/vector/util/TestVectorAppender.java  | 346 +
 .../vector/util/TestVectorBatchAppender.java}  |  41 +--
 6 files changed, 789 insertions(+), 31 deletions(-)
 create mode 100644 
java/vector/src/main/java/org/apache/arrow/vector/util/VectorAppender.java
 copy java/vector/src/main/java/org/apache/arrow/vector/util/{TransferPair.java 
=> VectorBatchAppender.java} (64%)
 create mode 100644 
java/vector/src/test/java/org/apache/arrow/vector/util/TestVectorAppender.java
 copy 
java/{algorithm/src/test/java/org/apache/arrow/algorithm/sort/TestOffHeapIntStack.java
 => 
vector/src/test/java/org/apache/arrow/vector/util/TestVectorBatchAppender.java} 
(53%)

[arrow] branch master updated (85996e6 -> 4c7bfc7)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 85996e6  ARROW-6871: [Java] Enhance TransferPair related parameters 
check and tests
 add 4c7bfc7  ARROW-7301: [Java] Sql type DATE should correspond to 
DateDayVector

No new revisions were added by this update.

Summary of changes:
 .../arrow/adapter/jdbc/JdbcToArrowUtils.java   |  8 ++--
 .../arrow/adapter/jdbc/consumer/DateConsumer.java  | 46 --
 .../arrow/adapter/jdbc/JdbcToArrowTestHelper.java  | 12 +++---
 .../adapter/jdbc/h2/JdbcToArrowDataTypesTest.java  |  6 +--
 .../arrow/adapter/jdbc/h2/JdbcToArrowNullTest.java | 10 ++---
 .../arrow/adapter/jdbc/h2/JdbcToArrowTest.java |  6 +--
 .../adapter/jdbc/h2/JdbcToArrowTimeZoneTest.java   |  6 +--
 .../jdbc/h2/JdbcToArrowVectorIteratorTest.java | 14 ---
 .../test/resources/h2/test1_all_datatypes_h2.yml   |  2 +-
 .../test1_all_datatypes_selected_null_rows_h2.yml  |  2 +-
 .../jdbc/src/test/resources/h2/test1_date_h2.yml   | 20 +-
 .../src/test/resources/h2/test1_est_date_h2.yml| 20 +-
 .../src/test/resources/h2/test1_gmt_date_h2.yml| 20 +-
 .../src/test/resources/h2/test1_pst_date_h2.yml| 20 +-
 .../org/apache/arrow/vector/DateDayVector.java |  1 +
 15 files changed, 110 insertions(+), 83 deletions(-)

[arrow] branch master updated (af24bb7 -> 85996e6)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from af24bb7  ARROW-6724: [C++] Allow simpler BufferOutputStream creation
 add 85996e6  ARROW-6871: [Java] Enhance TransferPair related parameters 
check and tests

No new revisions were added by this update.

Summary of changes:
 .../sort/FixedWidthInPlaceVectorSorter.java|   1 +
 .../TransferPairBenchmarks.java}   |  75 +-
 .../src/main/codegen/templates/UnionVector.java|   5 +-
 .../apache/arrow/vector/BaseFixedWidthVector.java  |   5 +-
 .../arrow/vector/BaseVariableWidthVector.java  |   5 +-
 .../java/org/apache/arrow/vector/BitVector.java|   5 +-
 .../arrow/vector/complex/FixedSizeListVector.java  |   5 +-
 .../apache/arrow/vector/complex/ListVector.java|   5 +-
 .../apache/arrow/vector/complex/StructVector.java  |   5 +-
 .../apache/arrow/vector/TestSplitAndTransfer.java  | 166 +++--
 10 files changed, 221 insertions(+), 56 deletions(-)
 copy 
java/performance/src/test/java/org/apache/arrow/vector/{Float8Benchmarks.java 
=> util/TransferPairBenchmarks.java} (59%)

[arrow] branch master updated (49aada2 -> af24bb7)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 49aada2  ARROW-7734: [C++] check status details for nullptr in equality
 add af24bb7  ARROW-6724: [C++] Allow simpler BufferOutputStream creation

No new revisions were added by this update.

Summary of changes:
 cpp/src/arrow/dataset/file_ipc_test.cc | 7 ++-
 cpp/src/arrow/extension_type_test.cc   | 4 ++--
 cpp/src/arrow/io/compressed_test.cc| 2 +-
 cpp/src/arrow/io/memory.h  | 2 +-
 cpp/src/arrow/ipc/feather_test.cc  | 4 ++--
 cpp/src/parquet/reader_test.cc | 2 +-
 6 files changed, 9 insertions(+), 12 deletions(-)

[arrow] branch master updated (942a4d0 -> 49aada2)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 942a4d0  ARROW-6738: [Java] Fix problems with current union comparison 
logic
 add 49aada2  ARROW-7734: [C++] check status details for nullptr in equality

No new revisions were added by this update.

Summary of changes:
 cpp/src/arrow/status.h   |  8 ++--
 cpp/src/arrow/status_test.cc | 13 +
 2 files changed, 19 insertions(+), 2 deletions(-)

[arrow] branch master updated (bd08d0e -> 942a4d0)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from bd08d0e  ARROW-7712: [CI] [Crossbow] Delete fuzzit jobs
 add 942a4d0  ARROW-6738: [Java] Fix problems with current union comparison 
logic

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/UnionVector.java|  2 +-
 .../arrow/vector/compare/ApproxEqualsVisitor.java  |  6 ++-
 .../arrow/vector/compare/RangeEqualsVisitor.java   | 46 ++-
 .../vector/compare/TestRangeEqualsVisitor.java | 51 ++
 4 files changed, 82 insertions(+), 23 deletions(-)

[arrow] branch master updated (9e0f76a -> 24a39dc)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 9e0f76a  ARROW-7666: [Packaging][deb] Always use Ninja to reduce build 
time
 add 24a39dc  ARROW-6352: [Java] Add implementation of DenseUnionVector

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/BaseReader.java |   3 +
 .../{UnionReader.java => DenseUnionReader.java}| 111 +--
 .../main/codegen/templates/DenseUnionVector.java   | 909 +
 .../main/codegen/templates/DenseUnionWriter.java   | 254 ++
 .../arrow/vector/compare/RangeEqualsVisitor.java   |   7 +
 .../arrow/vector/compare/TypeEqualsVisitor.java|   6 +
 .../apache/arrow/vector/compare/VectorVisitor.java |   4 +
 .../vector/complex/impl/AbstractBaseReader.java|  17 +
 .../{UnionHolder.java => DenseUnionHolder.java}|   9 +-
 .../java/org/apache/arrow/vector/types/Types.java  |  11 +-
 ...tUnionVector.java => TestDenseUnionVector.java} | 280 +--
 11 files changed, 1480 insertions(+), 131 deletions(-)
 copy java/vector/src/main/codegen/templates/{UnionReader.java => 
DenseUnionReader.java} (61%)
 create mode 100644 java/vector/src/main/codegen/templates/DenseUnionVector.java
 create mode 100644 java/vector/src/main/codegen/templates/DenseUnionWriter.java
 copy 
java/vector/src/main/java/org/apache/arrow/vector/holders/{UnionHolder.java => 
DenseUnionHolder.java} (83%)
 copy java/vector/src/test/java/org/apache/arrow/vector/{TestUnionVector.java 
=> TestDenseUnionVector.java} (55%)

[arrow] branch master updated (b852a57 -> cba234a)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from b852a57  ARROW-7667: [Packaging][deb] Add ubuntu-eoan to nightly jobs
 add cba234a  ARROW-6929: [C++] Remove first offset==0 check from Validate()

No new revisions were added by this update.

Summary of changes:
 cpp/src/arrow/array/validate.cc| 4 +---
 cpp/src/arrow/array_binary_test.cc | 2 --
 cpp/src/arrow/array_list_test.cc   | 2 --
 3 files changed, 1 insertion(+), 7 deletions(-)

[arrow] branch master updated (be4b245 -> e3b68dd)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from be4b245  ARROW-7567: [Java] Bump Checkstyle from 6.19 to 8.19
 add e3b68dd  ARROW-7567: [Java] Fix races in checkstyle upgdae

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/algorithm/search/TestParallelSearcher.java| 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[arrow] branch master updated (0f30b70 -> be4b245)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 0f30b70  ARROW-7092: [R] Add vignette for dplyr and datasets
 add be4b245  ARROW-7567: [Java] Bump Checkstyle from 6.19 to 8.19

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/AvroToArrowUtils.java|  3 +-
 .../org/apache/arrow/TestWriteReadAvroRecord.java  |  1 -
 .../deduplicate/TestVectorRunDeduplicator.java |  7 +-
 .../TestHashTableBasedDictionaryBuilder.java   |  1 -
 .../dictionary/TestSearchDictionaryEncoder.java|  1 -
 .../TestSearchTreeBasedDictionaryBuilder.java  |  1 -
 .../arrow/algorithm/rank/TestVectorRank.java   |  1 -
 .../algorithm/search/TestVectorRangeSearcher.java  |  3 +-
 .../arrow/algorithm/search/TestVectorSearcher.java |  1 -
 .../sort/TestDefaultVectorComparator.java  |  1 -
 .../sort/TestFixedWidthOutOfPlaceVectorSorter.java |  1 -
 java/dev/checkstyle/checkstyle.xml | 32 -
 .../java/org/apache/arrow/flight/FlightClient.java |  1 -
 .../java/org/apache/arrow/flight/FlightInfo.java   |  1 -
 .../java/org/apache/arrow/flight/FlightServer.java |  1 -
 .../arrow/flight/auth/BasicServerAuthHandler.java  |  1 -
 .../arrow/flight/auth/ServerAuthWrapper.java   |  1 -
 .../arrow/flight/TestApplicationMetadata.java  |  2 -
 .../java/org/apache/arrow/flight/TestAuth.java |  1 -
 .../apache/arrow/flight/TestBasicOperation.java|  1 -
 .../test/java/org/apache/arrow/flight/TestTls.java |  1 -
 .../apache/arrow/flight/auth/TestBasicAuth.java|  1 -
 .../arrow/gandiva/evaluator/ProjectorTest.java |  1 -
 .../src/main/java/io/netty/buffer/ArrowBuf.java|  3 +-
 .../apache/arrow/memory/AllocationReservation.java |  2 +-
 .../org/apache/arrow/memory/BaseAllocator.java |  3 +-
 .../java/org/apache/arrow/memory/BufferLedger.java |  2 +
 .../org/apache/arrow/memory/ReferenceManager.java  |  2 +-
 .../org/apache/arrow/memory/TestBaseAllocator.java | 12 ++--
 .../arrow/memory/util/TestArrowBufPointer.java |  3 +-
 .../arrow/memory/util/hash/TestArrowBufHasher.java |  2 -
 .../arrow/adapter/AvroAdapterBenchmarks.java   |  2 +-
 .../arrow/adapter/jdbc/JdbcAdapterBenchmarks.java  |  2 +-
 .../org/apache/arrow/vector/IntBenchmarks.java |  1 -
 java/pom.xml   |  4 +-
 .../org/apache/arrow/vector/AllocationHelper.java  |  2 +-
 .../apache/arrow/vector/BaseFixedWidthVector.java  |  6 +-
 .../org/apache/arrow/vector/BaseValueVector.java   |  3 +-
 .../java/org/apache/arrow/vector/BitVector.java|  5 +-
 .../org/apache/arrow/vector/BitVectorHelper.java   |  2 +
 .../org/apache/arrow/vector/DecimalVector.java |  3 +-
 .../org/apache/arrow/vector/VectorSchemaRoot.java  |  2 +-
 .../arrow/vector/complex/FixedSizeListVector.java  |  2 +
 .../apache/arrow/vector/complex/ListVector.java|  4 ++
 .../vector/complex/NonNullableStructVector.java|  2 +
 .../apache/arrow/vector/complex/StructVector.java  |  4 ++
 .../org/apache/arrow/vector/types/pojo/Field.java  |  5 +-
 .../arrow/vector/util/ByteFunctionHelpers.java | 12 ++--
 .../apache/arrow/vector/TestBitVectorHelper.java   |  2 +-
 .../org/apache/arrow/vector/TestDecimalVector.java | 10 +--
 .../apache/arrow/vector/TestSplitAndTransfer.java  |  2 -
 .../org/apache/arrow/vector/TestStructVector.java  |  1 -
 .../org/apache/arrow/vector/TestValueVector.java   |  2 +-
 .../org/apache/arrow/vector/TestVectorReset.java   |  5 +-
 .../apache/arrow/vector/TestVectorSchemaRoot.java  |  1 -
 .../apache/arrow/vector/TestVectorUnloadLoad.java  | 10 +--
 .../org/apache/arrow/vector/ipc/TestArrowFile.java |  2 +-
 .../org/apache/arrow/vector/ipc/TestJSONFile.java  | 76 --
 .../vector/testing/ValueVectorDataPopulator.java   |  2 +-
 .../arrow/vector/types/pojo/TestExtensionType.java |  1 -
 .../util/TestElementAddressableVectorIterator.java |  1 -
 61 files changed, 117 insertions(+), 153 deletions(-)

[arrow] branch master updated (1953fac -> cb840e6)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 1953fac  ARROW-7284: [Java] ensure java implementation meets clarified 
dictionary spec
 add cb840e6  ARROW-7599: [Java] Fix build break due to change in 
RangeEqualsVisitor

No new revisions were added by this update.

Summary of changes:
 .../main/java/org/apache/arrow/algorithm/search/ParallelSearcher.java | 2 +-
 .../java/org/apache/arrow/vector/complex/impl/TestComplexCopier.java  | 2 +-
 .../test/java/org/apache/arrow/vector/ipc/TestArrowReaderWriter.java  | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

[arrow] branch master updated (1b567e6 -> 1953fac)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 1b567e6  ARROW-6836: [Format] add a custom_metadata:[KeyValue] field 
to the Footer table in File.fbs
 add 1953fac  ARROW-7284: [Java] ensure java implementation meets clarified 
dictionary spec

No new revisions were added by this update.

Summary of changes:
 .../org/apache/arrow/vector/ipc/ArrowReader.java   |  27 +++
 .../vector/ipc/message/ArrowDictionaryBatch.java   |  15 ++
 .../vector/ipc/message/MessageSerializer.java  |   4 +-
 .../arrow/vector/ipc/TestArrowReaderWriter.java| 246 ++---
 4 files changed, 259 insertions(+), 33 deletions(-)

[arrow] branch master updated (2d43110 -> 1b567e6)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 2d43110  ARROW-7264: [Java] RangeEqualsVisitor type check is not 
correct
 add 1b567e6  ARROW-6836: [Format] add a custom_metadata:[KeyValue] field 
to the Footer table in File.fbs

No new revisions were added by this update.

Summary of changes:
 format/File.fbs | 3 +++
 1 file changed, 3 insertions(+)

[arrow] branch master updated (d6b81e0 -> 2d43110)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d6b81e0  ARROW-7590: [C++] Don't ignore managed files in thirdparty
 add 2d43110  ARROW-7264: [Java] RangeEqualsVisitor type check is not 
correct

No new revisions were added by this update.

Summary of changes:
 .../algorithm/deduplicate/DeduplicationUtils.java  |  2 +-
 .../dictionary/LinearDictionaryEncoder.java|  2 +-
 .../arrow/vector/compare/ApproxEqualsVisitor.java  | 24 ---
 .../arrow/vector/compare/RangeEqualsVisitor.java   | 46 +-
 .../arrow/vector/compare/VectorEqualsVisitor.java  | 18 ++---
 .../vector/dictionary/DictionaryHashTable.java |  2 +-
 .../vector/compare/TestRangeEqualsVisitor.java | 30 +++---
 .../vector/complex/impl/TestComplexCopier.java | 10 -
 8 files changed, 94 insertions(+), 40 deletions(-)

[arrow] branch master updated (910ae42 -> bdffecf)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 910ae42  ARROW-7568: [Java] Bump Apache Avro from 1.9.0 to 1.9.1
 add bdffecf  ARROW-7538: [Java] Clarify actual and desired size in 
AllocationManager

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/memory/AllocationManager.java | 10 --
 .../org/apache/arrow/memory/NettyAllocationManager.java | 17 +++--
 .../java/org/apache/arrow/memory/TestBaseAllocator.java | 11 ---
 3 files changed, 23 insertions(+), 15 deletions(-)

[arrow] branch master updated (f4fc7c4 -> 910ae42)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from f4fc7c4  ARROW-7570: [Java] Fix high severity issues
 add 910ae42  ARROW-7568: [Java] Bump Apache Avro from 1.9.0 to 1.9.1

No new revisions were added by this update.

Summary of changes:
 java/adapter/avro/pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[arrow] branch master updated (e55c22c -> f4fc7c4)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from e55c22c  ARROW-7572: [Java] Enforce Maven 3.3+ as mentioned in README
 add f4fc7c4  ARROW-7570: [Java] Fix high severity issues

No new revisions were added by this update.

Summary of changes:
 .../src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java  | 5 -
 .../java/org/apache/arrow/vector/complex/AbstractStructVector.java   | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

[arrow] branch master updated (64d96d9 -> e55c22c)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 64d96d9  ARROW-7595: [R][CI] R appveyor job fails due to pacman 
compression change
 add e55c22c  ARROW-7572: [Java] Enforce Maven 3.3+ as mentioned in README

No new revisions were added by this update.

Summary of changes:
 java/pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[arrow] branch master updated (b178e15 -> bdfdc79)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from b178e15  ARROW-7277: [Java] [Doc] Add discussion about vector lifecycle
 add bdfdc79  ARROW-7490: [Java] Avro converter should convert attributes 
and props to FieldType metadata

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/AvroToArrowUtils.java| 173 ++---
 .../java/org/apache/arrow/AvroToArrowTest.java |  72 +
 .../test_enum_attrs.avsc}  |   5 +-
 .../test_fixed_attr.avsc}  |   5 +-
 .../test_record_attrs.avsc}|  11 +-
 5 files changed, 199 insertions(+), 67 deletions(-)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_enum.avsc => 
attrs/test_enum_attrs.avsc} (92%)
 copy java/adapter/avro/src/test/resources/schema/{test_fixed.avsc => 
attrs/test_fixed_attr.avsc} (91%)
 copy 
java/adapter/avro/src/test/resources/schema/{skip/test_skip_record_before.avsc 
=> attrs/test_record_attrs.avsc} (79%)

[arrow] branch master updated (da0e218 -> b178e15)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from da0e218  ARROW-7259: [Java] Support subfield encoder use different 
hasher
 add b178e15  ARROW-7277: [Java] [Doc] Add discussion about vector lifecycle

No new revisions were added by this update.

Summary of changes:
 docs/source/java/ipc.rst|  24 +++--
 docs/source/java/vector.rst | 169 +---
 docs/source/java/vector_schema_root.rst |  12 ++-
 3 files changed, 159 insertions(+), 46 deletions(-)

[arrow] branch master updated (6a40bc6 -> da0e218)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 6a40bc6  ARROW-7406: [Java] NonNullableStructVector#hashCode should 
pass hasher to child vectors
 add da0e218  ARROW-7259: [Java] Support subfield encoder use different 
hasher

No new revisions were added by this update.

Summary of changes:
 .../arrow/vector/dictionary/ListSubfieldEncoder.java   | 10 --
 .../arrow/vector/dictionary/StructSubfieldEncoder.java | 14 --
 2 files changed, 20 insertions(+), 4 deletions(-)

[arrow] branch master updated (1216505 -> 6a40bc6)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 1216505  ARROW-7497: [Python] Stop relying on (deprecated) 
pandas.util.testing, move to pandas.testing
 add 6a40bc6  ARROW-7406: [Java] NonNullableStructVector#hashCode should 
pass hasher to child vectors

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/vector/complex/NonNullableStructVector.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[arrow] branch master updated (9742007 -> 1216505)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 9742007  ARROW-6112: [Java] Support int64 buffer  lengths in Java
 add 1216505  ARROW-7497: [Python] Stop relying on (deprecated) 
pandas.util.testing, move to pandas.testing

No new revisions were added by this update.

Summary of changes:
 python/pyarrow/tests/test_feather.py |  2 +-
 python/pyarrow/tests/test_hdfs.py|  6 --
 python/pyarrow/tests/test_ipc.py |  3 +--
 python/pyarrow/tests/test_orc.py |  2 +-
 python/pyarrow/tests/test_pandas.py  | 14 +++---
 python/pyarrow/tests/test_parquet.py | 12 ++--
 python/pyarrow/tests/test_table.py   |  2 +-
 python/pyarrow/tests/util.py | 21 +
 8 files changed, 42 insertions(+), 20 deletions(-)

[arrow] branch master updated (884b96c -> 9742007)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 884b96c  ARROW-7542: [CI][C++] Use $(sysctl -n hw.ncpu) instead of 
$(nproc) on macOS
 add 9742007  ARROW-6112: [Java] Support int64 buffer  lengths in Java

No new revisions were added by this update.

Summary of changes:
 cpp/src/gandiva/jni/jni_common.cc  |  17 +--
 .../arrow/adapter/orc/OrcReferenceManager.java |   8 +-
 .../java/org/apache/arrow/flight/ArrowMessage.java |   2 +-
 .../example/integration/IntegrationTestClient.java |   4 +-
 .../arrow/gandiva/evaluator/SelectionVector.java   |   4 +-
 .../arrow/gandiva/evaluator/VectorExpander.java|   6 +-
 .../src/main/java/io/netty/buffer/ArrowBuf.java| 147 +++--
 .../main/java/io/netty/buffer/NettyArrowBuf.java   |  26 +++-
 .../io/netty/buffer/PooledByteBufAllocatorL.java   |   5 +-
 .../org/apache/arrow/memory/AllocationManager.java |   8 +-
 .../org/apache/arrow/memory/BaseAllocator.java |  16 +--
 .../org/apache/arrow/memory/BufferAllocator.java   |   4 +-
 .../java/org/apache/arrow/memory/BufferLedger.java |  12 +-
 .../org/apache/arrow/memory/BufferManager.java |   4 +-
 .../arrow/memory/NettyAllocationManager.java   |   8 +-
 .../org/apache/arrow/memory/ReferenceManager.java  |  16 +--
 .../memory/rounding/DefaultRoundingPolicy.java |   2 +-
 .../arrow/memory/rounding/RoundingPolicy.java  |   2 +-
 .../memory/rounding/SegmentRoundingPolicy.java |   2 +-
 .../apache/arrow/memory/util/ArrowBufPointer.java  |  14 +-
 .../arrow/memory/util/ByteFunctionHelpers.java |  64 ++---
 .../apache/arrow/memory/util/LargeMemoryUtil.java} |  40 +++---
 .../arrow/memory/util/hash/ArrowBufHasher.java |   4 +-
 .../arrow/memory/util/hash/MurmurHasher.java   |  12 +-
 .../arrow/memory/util/hash/SimpleHasher.java   |   5 +-
 .../java/org/apache/arrow/util/Preconditions.java  |   6 +-
 .../arrow/memory/util/TestArrowBufPointer.java |   4 +-
 .../ipc/message/ArrowRecordBatchBenchmarks.java|   2 +-
 .../src/main/codegen/templates/UnionVector.java|  14 +-
 .../apache/arrow/vector/BaseFixedWidthVector.java  |   6 +-
 .../org/apache/arrow/vector/BaseValueVector.java   |   2 +-
 .../arrow/vector/BaseVariableWidthVector.java  |  31 ++---
 .../java/org/apache/arrow/vector/BitVector.java|   3 +-
 .../apache/arrow/vector/VariableWidthVector.java   |   2 +-
 .../vector/complex/BaseRepeatedValueVector.java|  19 +--
 .../arrow/vector/complex/FixedSizeListVector.java  |   8 +-
 .../apache/arrow/vector/complex/ListVector.java|   8 +-
 .../apache/arrow/vector/complex/StructVector.java  |   7 +-
 .../org/apache/arrow/vector/ipc/ReadChannel.java   |  17 ++-
 .../org/apache/arrow/vector/ipc/WriteChannel.java  |  21 ++-
 .../vector/ipc/message/ArrowDictionaryBatch.java   |   2 +-
 .../arrow/vector/ipc/message/ArrowFieldNode.java   |  10 +-
 .../arrow/vector/ipc/message/ArrowMessage.java |   2 +-
 .../arrow/vector/ipc/message/ArrowRecordBatch.java |  16 ++-
 .../vector/ipc/message/MessageSerializer.java  |  48 +++
 .../apache/arrow/vector/DirtyRootAllocator.java|   4 +-
 .../TestOversizedAllocationForValueVector.java |   7 +-
 .../org/apache/arrow/vector/TestStructVector.java  |   2 +-
 .../org/apache/arrow/vector/TestValueVector.java   |   2 +-
 .../org/apache/arrow/vector/TestVectorReAlloc.java |   2 +-
 .../org/apache/arrow/vector/TestVectorReset.java   |   2 +-
 .../apache/arrow/vector/TestVectorUnloadLoad.java  |   4 +-
 .../arrow/vector/ipc/MessageSerializerTest.java|  16 +--
 .../arrow/vector/ipc/TestArrowReaderWriter.java|   5 +-
 54 files changed, 392 insertions(+), 312 deletions(-)
 copy java/{vector/src/main/java/org/apache/arrow/vector/AddOrGetResult.java => 
memory/src/main/java/org/apache/arrow/memory/util/LargeMemoryUtil.java} (52%)

[arrow] branch master updated (dff612e -> e237726)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from dff612e  ARROW-7329: [Java] AllocationManager: Allow managing 
different types …
 add e237726  ARROW-7506: [Java] JMH benchmarks should be called from main 
methods

No new revisions were added by this update.

Summary of changes:
 .../performance/src/test/java/io/netty/buffer/ArrowBufBenchmarks.java | 4 +---
 .../src/test/java/org/apache/arrow/adapter/AvroAdapterBenchmarks.java | 4 +---
 .../java/org/apache/arrow/adapter/jdbc/JdbcAdapterBenchmarks.java | 4 +---
 .../src/test/java/org/apache/arrow/memory/AllocatorBenchmarks.java| 4 +---
 .../org/apache/arrow/memory/util/ByteFunctionHelpersBenchmarks.java   | 4 +---
 .../test/java/org/apache/arrow/vector/BaseValueVectorBenchmarks.java  | 4 +---
 .../test/java/org/apache/arrow/vector/BitVectorHelperBenchmarks.java  | 1 -
 .../test/java/org/apache/arrow/vector/DecimalVectorBenchmarks.java| 4 +---
 .../src/test/java/org/apache/arrow/vector/Float8Benchmarks.java   | 4 +---
 .../src/test/java/org/apache/arrow/vector/VarCharBenchmarks.java  | 4 +---
 .../java/org/apache/arrow/vector/VariableWidthVectorBenchmarks.java   | 4 +---
 .../apache/arrow/vector/dictionary/DictionaryEncoderBenchmarks.java   | 4 +---
 .../apache/arrow/vector/ipc/message/ArrowRecordBatchBenchmarks.java   | 4 +---
 13 files changed, 12 insertions(+), 37 deletions(-)

[arrow] branch master updated (7f905a6 -> 17bdb5a)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 7f905a6  ARROW-7387: [C#] Support ListType Serialization
 add 17bdb5a  ARROW-7429: [Java] Enhance code style checking for Java code 
(remove consecutive spaces)

No new revisions were added by this update.

Summary of changes:
 .../org/apache/arrow/AvroToArrowConfigBuilder.java |  2 +-
 .../java/org/apache/arrow/AvroToArrowUtils.java| 48 +++---
 .../org/apache/arrow/adapter/orc/OrcReader.java|  4 +-
 .../algorithm/deduplicate/DeduplicationUtils.java  |  2 +-
 .../algorithm/sort/CompositeVectorComparator.java  |  2 +-
 .../arrow/algorithm/sort/OffHeapIntStack.java  |  2 +-
 .../dictionary/TestHashTableDictionaryEncoder.java |  4 +-
 java/dev/checkstyle/checkstyle.xml |  3 ++
 .../java/org/apache/arrow/flight/ArrowMessage.java |  2 +-
 .../apache/arrow/flight/example/FlightHolder.java  |  2 +-
 .../apache/arrow/gandiva/evaluator/JniLoader.java  |  2 +-
 .../apache/arrow/gandiva/expression/InNode.java|  8 ++--
 .../gandiva/evaluator/DecimalTypeUtilTest.java |  2 +-
 .../arrow/gandiva/evaluator/FilterProjectTest.java |  4 +-
 .../arrow/gandiva/evaluator/ProjectorTest.java |  2 +-
 .../main/java/io/netty/buffer/NettyArrowBuf.java   |  6 +--
 .../org/apache/arrow/memory/BaseAllocator.java |  2 +-
 .../java/org/apache/arrow/memory/BufferLedger.java |  2 +-
 .../memory/rounding/SegmentRoundingPolicy.java |  2 +-
 .../java/io/netty/buffer/TestNettyArrowBuf.java|  2 +-
 .../org/apache/arrow/memory/TestAccountant.java|  8 ++--
 .../org/apache/arrow/memory/TestBaseAllocator.java |  6 +--
 .../arrow/vector/FloatingPointBenchmarks.java  |  2 +-
 .../org/apache/arrow/plasma/PlasmaClientTest.java  | 27 ++--
 .../arrow/vector/BaseVariableWidthVector.java  |  2 +-
 .../org/apache/arrow/vector/DurationVector.java|  4 +-
 .../java/org/apache/arrow/vector/TypeLayout.java   |  2 +-
 .../java/org/apache/arrow/vector/UInt1Vector.java  |  2 +-
 .../java/org/apache/arrow/vector/UInt4Vector.java  |  2 +-
 .../java/org/apache/arrow/vector/UInt8Vector.java  |  2 +-
 .../arrow/vector/compare/ApproxEqualsVisitor.java  |  8 ++--
 .../apache/arrow/vector/ipc/ArrowStreamReader.java |  2 +-
 .../apache/arrow/vector/ipc/JsonFileWriter.java|  2 +-
 .../java/org/apache/arrow/vector/util/Text.java|  2 +-
 .../apache/arrow/vector/TestBitVectorHelper.java   |  4 +-
 .../org/apache/arrow/vector/TestListVector.java| 26 ++--
 .../org/apache/arrow/vector/TestMapVector.java | 10 ++---
 .../org/apache/arrow/vector/TestUnionVector.java   |  2 +-
 .../org/apache/arrow/vector/TestValueVector.java   |  4 +-
 .../apache/arrow/vector/TestVectorSchemaRoot.java  |  2 +-
 .../vector/complex/writer/TestComplexWriter.java   | 10 ++---
 .../org/apache/arrow/vector/ipc/BaseFileTest.java  | 16 
 .../org/apache/arrow/vector/ipc/TestArrowFile.java |  2 +-
 .../arrow/vector/ipc/TestArrowReaderWriter.java|  8 ++--
 44 files changed, 131 insertions(+), 127 deletions(-)

[arrow] branch master updated (0ddc1f4 -> 41c9b18)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 0ddc1f4  ARROW-6473: Dictionary encoding format clarifications/future 
proofing
 add 41c9b18  ARROW-7026: [Java] Remove assertions in 
MessageSerializer/vector/writer/reader

No new revisions were added by this update.

Summary of changes:
 java/vector/src/main/codegen/templates/StructWriters.java|  2 +-
 .../org/apache/arrow/vector/BaseVariableWidthVector.java |  2 +-
 .../java/org/apache/arrow/vector/complex/StructVector.java   |  2 +-
 .../java/org/apache/arrow/vector/ipc/JsonFileReader.java |  3 ++-
 .../java/org/apache/arrow/vector/ipc/JsonFileWriter.java |  5 +++--
 .../apache/arrow/vector/ipc/message/MessageSerializer.java   | 12 ++--
 6 files changed, 14 insertions(+), 12 deletions(-)

[arrow] branch master updated (5c2bb6f -> 0ddc1f4)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 5c2bb6f  PARQUET-1692: [C++] LogicalType::FromThrift error on Centos 7 
RPM
 add 0ddc1f4  ARROW-6473: Dictionary encoding format clarifications/future 
proofing

No new revisions were added by this update.

Summary of changes:
 docs/source/format/Columnar.rst | 41 -
 format/Message.fbs  |  3 ++-
 format/Schema.fbs   |  7 ++-
 3 files changed, 48 insertions(+), 3 deletions(-)

[arrow] branch master updated (93ea67e -> dbc5681)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 93ea67e  ARROW-7166: [Java] Remove redundant code for Jdbc adapters
 add dbc5681  ARROW-6889: [Java] ComplexCopier enable FixedSizeList type & 
fix RangeEqualsVisitor StackOverFlow

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/ComplexCopier.java  |  12 +-
 .../arrow/vector/compare/RangeEqualsVisitor.java   |   2 +-
 .../vector/compare/TestRangeEqualsVisitor.java |  43 ++
 .../vector/complex/impl/TestComplexCopier.java | 155 +
 4 files changed, 208 insertions(+), 4 deletions(-)
 create mode 100644 
java/vector/src/test/java/org/apache/arrow/vector/complex/impl/TestComplexCopier.java

[arrow] branch master updated (2929d2c -> 93ea67e)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 2929d2c  ARROW-7255: [CI] Re-enable source release test on pull request
 add 93ea67e  ARROW-7166: [Java] Remove redundant code for Jdbc adapters

No new revisions were added by this update.

Summary of changes:
 .../arrow/adapter/jdbc/consumer/ArrayConsumer.java | 15 +-
 .../{NullConsumer.java => BaseConsumer.java}   | 32 +++--
 .../adapter/jdbc/consumer/BigIntConsumer.java  | 32 ++---
 .../adapter/jdbc/consumer/BinaryConsumer.java  | 16 +--
 .../arrow/adapter/jdbc/consumer/BitConsumer.java   | 32 ++---
 .../arrow/adapter/jdbc/consumer/BlobConsumer.java  |  6 +--
 .../arrow/adapter/jdbc/consumer/ClobConsumer.java  | 15 +-
 .../arrow/adapter/jdbc/consumer/DateConsumer.java  | 55 ++
 .../adapter/jdbc/consumer/DecimalConsumer.java | 32 ++---
 .../adapter/jdbc/consumer/DoubleConsumer.java  | 32 ++---
 .../arrow/adapter/jdbc/consumer/FloatConsumer.java | 32 ++---
 .../arrow/adapter/jdbc/consumer/IntConsumer.java   | 32 ++---
 .../arrow/adapter/jdbc/consumer/NullConsumer.java  | 16 +--
 .../adapter/jdbc/consumer/SmallIntConsumer.java| 32 ++---
 .../arrow/adapter/jdbc/consumer/TimeConsumer.java  | 55 ++
 .../adapter/jdbc/consumer/TimestampConsumer.java   | 55 ++
 .../adapter/jdbc/consumer/TinyIntConsumer.java | 32 ++---
 .../adapter/jdbc/consumer/VarCharConsumer.java | 32 ++---
 18 files changed, 106 insertions(+), 447 deletions(-)
 copy 
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/consumer/{NullConsumer.java
 => BaseConsumer.java} (63%)

[arrow] branch master updated (95119a5 -> 74fa956)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 95119a5  ARROW-7216: [Java] Improve the performance of 
setting/clearing individual bits
 add 74fa956  ARROW-7072: [Java] Support concating validity bits efficiently

No new revisions were added by this update.

Summary of changes:
 .../org/apache/arrow/vector/BitVectorHelper.java   | 81 +
 .../apache/arrow/vector/TestBitVectorHelper.java   | 82 ++
 2 files changed, 163 insertions(+)

[arrow] branch master updated (ee3f10a -> 95119a5)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from ee3f10a  ARROW-7225: [C++] Fix `*std::move(Result)` for move-only T
 add 95119a5  ARROW-7216: [Java] Improve the performance of 
setting/clearing individual bits

No new revisions were added by this update.

Summary of changes:
 .../adapter/jdbc/consumer/BinaryConsumer.java  |  2 +-
 .../arrow/adapter/jdbc/consumer/ClobConsumer.java  |  4 +-
 .../algorithm/deduplicate/DeduplicationUtils.java  |  4 +-
 .../sort/FixedWidthOutOfPlaceVectorSorter.java |  4 +-
 .../sort/VariableWidthOutOfPlaceVectorSorter.java  |  4 +-
 .../arrow/algorithm/search/TestVectorSearcher.java |  4 +-
 .../arrow/vector/BitVectorHelperBenchmarks.java| 63 --
 .../apache/arrow/vector/BaseFixedWidthVector.java  |  8 +--
 .../arrow/vector/BaseVariableWidthVector.java  | 28 +-
 .../java/org/apache/arrow/vector/BigIntVector.java | 10 ++--
 .../java/org/apache/arrow/vector/BitVector.java| 30 +--
 .../org/apache/arrow/vector/BitVectorHelper.java   | 53 +++---
 .../org/apache/arrow/vector/DateDayVector.java | 10 ++--
 .../org/apache/arrow/vector/DateMilliVector.java   | 10 ++--
 .../org/apache/arrow/vector/DecimalVector.java | 22 
 .../org/apache/arrow/vector/DurationVector.java|  8 +--
 .../apache/arrow/vector/FixedSizeBinaryVector.java | 10 ++--
 .../java/org/apache/arrow/vector/Float4Vector.java | 10 ++--
 .../java/org/apache/arrow/vector/Float8Vector.java | 10 ++--
 .../java/org/apache/arrow/vector/IntVector.java| 10 ++--
 .../org/apache/arrow/vector/IntervalDayVector.java |  8 +--
 .../apache/arrow/vector/IntervalYearVector.java| 10 ++--
 .../org/apache/arrow/vector/SmallIntVector.java| 12 ++---
 .../org/apache/arrow/vector/TimeMicroVector.java   | 10 ++--
 .../org/apache/arrow/vector/TimeMilliVector.java   | 10 ++--
 .../org/apache/arrow/vector/TimeNanoVector.java| 10 ++--
 .../org/apache/arrow/vector/TimeSecVector.java | 10 ++--
 .../arrow/vector/TimeStampMicroTZVector.java   |  6 +--
 .../apache/arrow/vector/TimeStampMicroVector.java  |  6 +--
 .../arrow/vector/TimeStampMilliTZVector.java   |  6 +--
 .../apache/arrow/vector/TimeStampMilliVector.java  |  6 +--
 .../apache/arrow/vector/TimeStampNanoTZVector.java |  6 +--
 .../apache/arrow/vector/TimeStampNanoVector.java   |  6 +--
 .../apache/arrow/vector/TimeStampSecTZVector.java  |  6 +--
 .../apache/arrow/vector/TimeStampSecVector.java|  6 +--
 .../org/apache/arrow/vector/TimeStampVector.java   |  4 +-
 .../org/apache/arrow/vector/TinyIntVector.java | 12 ++---
 .../java/org/apache/arrow/vector/UInt1Vector.java  | 12 ++---
 .../java/org/apache/arrow/vector/UInt2Vector.java  | 12 ++---
 .../java/org/apache/arrow/vector/UInt4Vector.java  | 10 ++--
 .../java/org/apache/arrow/vector/UInt8Vector.java  | 10 ++--
 .../org/apache/arrow/vector/VarBinaryVector.java   |  4 +-
 .../org/apache/arrow/vector/VarCharVector.java |  4 +-
 .../arrow/vector/complex/FixedSizeListVector.java  |  6 +--
 .../apache/arrow/vector/complex/ListVector.java|  4 +-
 .../apache/arrow/vector/complex/StructVector.java  |  4 +-
 .../apache/arrow/vector/TestBitVectorHelper.java   | 24 -
 .../org/apache/arrow/vector/TestListVector.java|  6 +--
 .../org/apache/arrow/vector/TestValueVector.java   |  2 +-
 .../apache/arrow/vector/TestVectorUnloadLoad.java  |  4 +-
 50 files changed, 317 insertions(+), 223 deletions(-)

[arrow] branch master updated (c20aaa0 -> ee3f10a)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from c20aaa0  ARROW-7161: [C++] Migrate filesystem APIs from Status to 
Result
 add ee3f10a  ARROW-7225: [C++] Fix `*std::move(Result)` for move-only T

No new revisions were added by this update.

Summary of changes:
 cpp/src/arrow/result.h   |  2 +-
 cpp/src/arrow/result_test.cc | 20 
 2 files changed, 17 insertions(+), 5 deletions(-)

[arrow] branch master updated (9f5e4b4 -> 03da904)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 9f5e4b4  ARROW-6867: [FlightRPC][Java] clean up default executor
 add 03da904  ARROW-7021: [Java] UnionFixedSizeListWriter decimal type 
should check writer index

No new revisions were added by this update.

Summary of changes:
 .../templates/UnionFixedSizeListWriter.java|  9 
 .../arrow/vector/TestFixedSizeListVector.java  | 51 ++
 2 files changed, 60 insertions(+)

[arrow] branch master updated (d2ca1f1 -> 9f5e4b4)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d2ca1f1  ARROW-6720: [C++] Add HDFS implementation to filesystem layer
 add 9f5e4b4  ARROW-6867: [FlightRPC][Java] clean up default executor

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/flight/FlightServer.java | 31 +--
 .../org/apache/arrow/flight/TestServerOptions.java | 44 ++
 2 files changed, 71 insertions(+), 4 deletions(-)

[arrow] branch master updated (bdedaf8 -> 2186578)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from bdedaf8  ARROW-6820: [Format] Update Map type child to "entries"
 add 2186578  ARROW-7176: [C++] Fix arrow::ipc compiler warning

No new revisions were added by this update.

Summary of changes:
 cpp/src/arrow/ipc/reader.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[arrow] branch master updated (d47d3ba -> eae2c1b)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d47d3ba  ARROW-6888: [Java] Support copy operation for vector value 
comparators
 add eae2c1b  ARROW-7152: [Java] Delete useless class DiffFunction

No new revisions were added by this update.

Summary of changes:
 .../apache/arrow/vector/compare/DiffFunction.java  | 27 --
 1 file changed, 27 deletions(-)
 delete mode 100644 
java/vector/src/main/java/org/apache/arrow/vector/compare/DiffFunction.java

[arrow] branch master updated (c46c5e6 -> d47d3ba)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from c46c5e6  ARROW-6707: [Java] Improve the performance of JDBC adapters 
by using nullable information
 add d47d3ba  ARROW-6888: [Java] Support copy operation for vector value 
comparators

No new revisions were added by this update.

Summary of changes:
 .../algorithm/sort/CompositeVectorComparator.java  |  9 
 .../algorithm/sort/DefaultVectorComparators.java   | 61 ++
 .../algorithm/sort/StableVectorComparator.java |  5 +-
 .../algorithm/sort/VectorValueComparator.java  |  6 +++
 .../sort/TestDefaultVectorComparator.java  | 20 +++
 .../algorithm/sort/TestStableVectorComparator.java |  5 ++
 6 files changed, 105 insertions(+), 1 deletion(-)

[arrow] branch master updated (b0f3845 -> c46c5e6)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from b0f3845  ARROW-7164: [CI] Dev cron github action is failing every 15 
minutes
 add c46c5e6  ARROW-6707: [Java] Improve the performance of JDBC adapters 
by using nullable information

No new revisions were added by this update.

Summary of changes:
 .../arrow/adapter/jdbc/ArrowVectorIterator.java|   2 +-
 .../arrow/adapter/jdbc/JdbcToArrowUtils.java   |  38 ++-
 .../arrow/adapter/jdbc/consumer/ArrayConsumer.java |  94 --
 .../adapter/jdbc/consumer/BigIntConsumer.java  |  70 +++-
 .../adapter/jdbc/consumer/BinaryConsumer.java  |  73 -
 .../arrow/adapter/jdbc/consumer/BitConsumer.java   |  70 +++-
 .../arrow/adapter/jdbc/consumer/BlobConsumer.java  |  15 +-
 .../arrow/adapter/jdbc/consumer/ClobConsumer.java  | 111 +--
 .../jdbc/consumer/CompositeJdbcConsumer.java   |  15 +-
 .../arrow/adapter/jdbc/consumer/DateConsumer.java  |  92 +-
 .../adapter/jdbc/consumer/DecimalConsumer.java |  70 +++-
 .../adapter/jdbc/consumer/DoubleConsumer.java  |  70 +++-
 .../arrow/adapter/jdbc/consumer/FloatConsumer.java |  70 +++-
 .../arrow/adapter/jdbc/consumer/IntConsumer.java   |  70 +++-
 .../arrow/adapter/jdbc/consumer/JdbcConsumer.java  |   1 -
 .../adapter/jdbc/consumer/SmallIntConsumer.java|  70 +++-
 .../arrow/adapter/jdbc/consumer/TimeConsumer.java  |  90 -
 .../adapter/jdbc/consumer/TimestampConsumer.java   |  90 -
 .../adapter/jdbc/consumer/TinyIntConsumer.java |  70 +++-
 .../adapter/jdbc/consumer/VarCharConsumer.java |  74 -
 .../arrow/adapter/JdbcAdapterBenchmarks.java   | 144 
 .../arrow/adapter/jdbc/JdbcAdapterBenchmarks.java  | 361 +
 22 files changed, 1374 insertions(+), 386 deletions(-)
 delete mode 100644 
java/performance/src/test/java/org/apache/arrow/adapter/JdbcAdapterBenchmarks.java
 create mode 100644 
java/performance/src/test/java/org/apache/arrow/adapter/jdbc/JdbcAdapterBenchmarks.java

[arrow] branch master updated (d130910 -> 497163f)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from d130910  ARROW-7098: [Java] Improve the performance of comparing two 
memory blocks
 add 497163f  ARROW-6594: [Java] Support logical type encodings from Avro

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/AvroToArrowUtils.java| 150 ---
 .../AvroDateConsumer.java} |  15 +-
 .../consumers/logical/AvroDecimalConsumer.java |  88 +
 .../AvroTimeMicroConsumer.java}|  15 +-
 .../AvroTimeMillisConsumer.java}   |  15 +-
 .../AvroTimestampMicrosConsumer.java}  |  15 +-
 .../AvroTimestampMillisConsumer.java}  |  15 +-
 .../org/apache/arrow/AvroLogicalTypesTest.java | 201 +
 .../java/org/apache/arrow/AvroSkipFieldTest.java   |  26 ---
 .../test/java/org/apache/arrow/AvroTestBase.java   |  25 +++
 .../org/apache/arrow/AvroToArrowIteratorTest.java  |  10 +-
 .../java/org/apache/arrow/AvroToArrowTest.java |  23 ---
 .../test_date.avsc}|   3 +-
 .../test_decimal_invalid1.avsc}|   5 +-
 .../test_decimal_invalid2.avsc}|   5 +-
 .../test_decimal_invalid3.avsc}|   5 +-
 .../test_decimal_invalid4.avsc}|   7 +-
 .../test_decimal_with_original_bytes.avsc} |   5 +-
 .../test_decimal_with_original_fixed.avsc} |   7 +-
 .../test_time_micros.avsc} |   3 +-
 .../test_time_millis.avsc} |   3 +-
 .../test_timestamp_micros.avsc}|   3 +-
 .../test_timestamp_millis.avsc}|   3 +-
 23 files changed, 521 insertions(+), 126 deletions(-)
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroIntConsumer.java
 => logical/AvroDateConsumer.java} (71%)
 create mode 100644 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/logical/AvroDecimalConsumer.java
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroLongConsumer.java
 => logical/AvroTimeMicroConsumer.java} (70%)
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroIntConsumer.java
 => logical/AvroTimeMillisConsumer.java} (70%)
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroLongConsumer.java
 => logical/AvroTimestampMicrosConsumer.java} (68%)
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroLongConsumer.java
 => logical/AvroTimestampMillisConsumer.java} (68%)
 create mode 100644 
java/adapter/avro/src/test/java/org/apache/arrow/AvroLogicalTypesTest.java
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_int.avsc => 
logical/test_date.avsc} (95%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_bytes.avsc => 
logical/test_decimal_invalid1.avsc} (92%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_bytes.avsc => 
logical/test_decimal_invalid2.avsc} (91%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_bytes.avsc => 
logical/test_decimal_invalid3.avsc} (91%)
 copy java/adapter/avro/src/test/resources/schema/{test_fixed.avsc => 
logical/test_decimal_invalid4.avsc} (90%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_bytes.avsc => 
logical/test_decimal_with_original_bytes.avsc} (92%)
 copy java/adapter/avro/src/test/resources/schema/{test_fixed.avsc => 
logical/test_decimal_with_original_fixed.avsc} (90%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_long.avsc => 
logical/test_time_micros.avsc} (94%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_int.avsc => 
logical/test_time_millis.avsc} (94%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_long.avsc => 
logical/test_timestamp_micros.avsc} (94%)
 copy java/adapter/avro/src/test/resources/schema/{test_primitive_long.avsc => 
logical/test_timestamp_millis.avsc} (94%)

[arrow] branch master updated (088ecac -> d130910)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 088ecac  ARROW-6887: [Java] Create prose documentation for using 
ValueVectors
 add d130910  ARROW-7098: [Java] Improve the performance of comparing two 
memory blocks

No new revisions were added by this update.

Summary of changes:
 .../main/java/org/apache/arrow/memory/util/ByteFunctionHelpers.java | 6 +++---
 .../main/java/org/apache/arrow/memory/util/hash/SimpleHasher.java   | 2 +-
 .../src/main/java/org/apache/arrow/vector/BitVectorHelper.java  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

[arrow] branch master updated (8219a8b -> 088ecac)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 8219a8b  ARROW-7103: [R] Various minor cleanups
 add 088ecac  ARROW-6887: [Java] Create prose documentation for using 
ValueVectors

No new revisions were added by this update.

Summary of changes:
 docs/source/index.rst  |   1 +
 docs/source/{cpp => java}/index.rst|  18 +--
 docs/source/java/ipc.rst   | 175 +
 docs/source/java/vector.rst| 121 ++
 docs/source/java/vector_schema_root.rst|  68 
 .../org/apache/arrow/vector/VectorSchemaRoot.java  |   9 ++
 6 files changed, 382 insertions(+), 10 deletions(-)
 copy docs/source/{cpp => java}/index.rst (76%)
 create mode 100644 docs/source/java/ipc.rst
 create mode 100644 docs/source/java/vector.rst
 create mode 100644 docs/source/java/vector_schema_root.rst

[arrow] branch master updated (3fbbcda -> 5d33018)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 3fbbcda  ARROW-7020: [Java] Fix the bugs when calculating vector hash 
code
 add 5d33018  ARROW-6911: [Java] Provide composite comparator

No new revisions were added by this update.

Summary of changes:
 .../algorithm/sort/CompositeVectorComparator.java  |  62 
 .../sort/TestCompositeVectorComparator.java| 112 +
 2 files changed, 174 insertions(+)
 create mode 100644 
java/algorithm/src/main/java/org/apache/arrow/algorithm/sort/CompositeVectorComparator.java
 create mode 100644 
java/algorithm/src/test/java/org/apache/arrow/algorithm/sort/TestCompositeVectorComparator.java

[arrow] branch master updated (660358d -> 3fbbcda)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 660358d  ARROW-7082: [Packaging][deb] Add apache-arrow-archive-keyring 
package
 add 3fbbcda  ARROW-7020: [Java] Fix the bugs when calculating vector hash 
code

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/UnionVector.java| 13 +--
 .../apache/arrow/vector/BaseFixedWidthVector.java  |  3 ++
 .../arrow/vector/BaseVariableWidthVector.java  |  3 ++
 .../java/org/apache/arrow/vector/ZeroVector.java   |  3 +-
 .../arrow/vector/complex/FixedSizeListVector.java  |  3 +-
 .../apache/arrow/vector/complex/ListVector.java|  3 +-
 .../apache/arrow/vector/complex/StructVector.java  |  3 +-
 .../org/apache/arrow/vector/TestUnionVector.java   | 22 
 .../org/apache/arrow/vector/TestValueVector.java   | 42 ++
 9 files changed, 89 insertions(+), 6 deletions(-)

[arrow] branch master updated (44e8d9f -> 76c9c39)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 44e8d9f  ARROW-7007: [C++] Add use_mmap option to LocalFS
 add 76c9c39  ARROW-6912: [Java] Extract a common base class for avro 
converter consumers

No new revisions were added by this update.

Summary of changes:
 .../apache/arrow/consumers/AvroArraysConsumer.java | 29 +++--
 .../arrow/consumers/AvroBooleanConsumer.java   | 35 ++---
 .../apache/arrow/consumers/AvroBytesConsumer.java  | 35 ++---
 .../apache/arrow/consumers/AvroDoubleConsumer.java | 36 ++
 .../apache/arrow/consumers/AvroEnumConsumer.java   | 36 ++
 .../apache/arrow/consumers/AvroFixedConsumer.java  | 35 ++---
 .../apache/arrow/consumers/AvroFloatConsumer.java  | 36 ++
 .../apache/arrow/consumers/AvroIntConsumer.java| 36 ++
 .../apache/arrow/consumers/AvroLongConsumer.java   | 36 ++
 .../apache/arrow/consumers/AvroMapConsumer.java| 29 +++--
 .../apache/arrow/consumers/AvroNullConsumer.java   | 33 +++-
 .../apache/arrow/consumers/AvroStringConsumer.java | 34 ++--
 .../apache/arrow/consumers/AvroStructConsumer.java | 30 +++---
 .../apache/arrow/consumers/AvroUnionsConsumer.java | 30 +++---
 ...{AvroIntConsumer.java => BaseAvroConsumer.java} | 29 ++---
 .../arrow/consumers/CompositeAvroConsumer.java |  3 +-
 16 files changed, 49 insertions(+), 453 deletions(-)
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroIntConsumer.java
 => BaseAvroConsumer.java} (67%)

[arrow] branch master updated (c1090a3 -> 558263f)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from c1090a3  ARROW-6286: [GLib] Add support for LargeList type
 add 558263f  ARROW-6592: [Java] Add support for skipping decoding of 
columns/field in Avro converter

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/AvroToArrowConfig.java   |  24 +-
 .../org/apache/arrow/AvroToArrowConfigBuilder.java |  74 +++
 .../java/org/apache/arrow/AvroToArrowUtils.java| 209 +--
 .../apache/arrow/AvroToArrowVectorIterator.java|   6 +-
 .../apache/arrow/consumers/AvroArraysConsumer.java |   3 +-
 .../arrow/consumers/AvroBooleanConsumer.java   |   4 +-
 .../apache/arrow/consumers/AvroBytesConsumer.java  |   3 +-
 .../apache/arrow/consumers/AvroDoubleConsumer.java |   3 +-
 .../apache/arrow/consumers/AvroEnumConsumer.java   |   3 +-
 .../apache/arrow/consumers/AvroFixedConsumer.java  |   3 +-
 .../apache/arrow/consumers/AvroFloatConsumer.java  |   3 +-
 .../apache/arrow/consumers/AvroIntConsumer.java|   3 +-
 .../apache/arrow/consumers/AvroLongConsumer.java   |   3 +-
 .../apache/arrow/consumers/AvroMapConsumer.java|   5 +-
 .../apache/arrow/consumers/AvroNullConsumer.java   |   3 +-
 .../apache/arrow/consumers/AvroStringConsumer.java |   3 +-
 .../apache/arrow/consumers/AvroStructConsumer.java |   6 +-
 .../apache/arrow/consumers/AvroUnionsConsumer.java |   5 +-
 .../arrow/consumers/CompositeAvroConsumer.java |  12 +-
 .../java/org/apache/arrow/consumers/Consumer.java  |  13 +-
 .../arrow/consumers/NullableTypeConsumer.java  |  79 ---
 .../{AvroIntConsumer.java => SkipConsumer.java}|  34 +-
 .../org/apache/arrow/consumers/SkipFunction.java}  |  13 +-
 .../java/org/apache/arrow/AvroSkipFieldTest.java   | 652 +
 .../test/java/org/apache/arrow/AvroTestBase.java   |   4 +-
 .../org/apache/arrow/AvroToArrowIteratorTest.java  |   6 +-
 .../java/org/apache/arrow/AvroToArrowTest.java |   2 +-
 .../test_skip_array_before.avsc}   |   4 +-
 .../test_skip_array_expected.avsc} |   3 +-
 .../test_skip_base1.avsc}  |   7 +-
 .../test_skip_base2.avsc}  |   6 +-
 .../test_skip_boolean_expected.avsc}   |   5 +-
 .../test_skip_bytes_expected.avsc} |   6 +-
 .../test_skip_double_expected.avsc}|   5 +-
 .../test_skip_enum_expected.avsc}  |   6 +-
 .../test_skip_fixed_expected.avsc} |   6 +-
 .../test_skip_float_expected.avsc} |   5 +-
 .../test_skip_int_expected.avsc}   |   7 +-
 .../test_skip_long_expected.avsc}  |   5 +-
 .../test_skip_map_before.avsc} |   4 +-
 .../test_skip_map_expected.avsc}   |   3 +-
 .../test_skip_multi_fields_expected.avsc}  |   4 +-
 .../test_skip_record_before.avsc}  |  11 +-
 .../test_skip_record_expected.avsc}|   6 +-
 .../test_skip_second_level_expected.avsc}  |   3 +-
 .../test_skip_single_field_expected.avsc}  |   3 +-
 .../test_skip_string_expected.avsc}|   6 +-
 .../test_skip_third_level_expected.avsc}   |  20 +-
 .../test_skip_union_before.avsc}   |   7 +-
 .../test_skip_union_multi_fields_expected.avsc}|   6 +-
 .../test_skip_union_nullable_field_expected.avsc}  |   6 +-
 .../test_skip_union_one_field_expected.avsc}   |   6 +-
 .../test/resources/schema/test_nested_record.avsc  |   2 +-
 .../arrow/adapter/AvroAdapterBenchmarks.java   |   3 +-
 54 files changed, 1064 insertions(+), 269 deletions(-)
 create mode 100644 
java/adapter/avro/src/main/java/org/apache/arrow/AvroToArrowConfigBuilder.java
 delete mode 100644 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/NullableTypeConsumer.java
 copy 
java/adapter/avro/src/main/java/org/apache/arrow/consumers/{AvroIntConsumer.java
 => SkipConsumer.java} (66%)
 copy 
java/{vector/src/main/java/org/apache/arrow/vector/compare/DiffFunction.java => 
adapter/avro/src/main/java/org/apache/arrow/consumers/SkipFunction.java} (77%)
 create mode 100644 
java/adapter/avro/src/test/java/org/apache/arrow/AvroSkipFieldTest.java
 copy java/adapter/avro/src/test/resources/schema/{test_record.avsc => 
skip/test_skip_array_before.avsc} (91%)
 copy java/adapter/avro/src/test/resources/schema/{test_record.avsc => 
skip/test_skip_array_expected.avsc} (94%)
 copy java/adapter/avro/src/test/resources/schema/{test_record.avsc => 
skip/test_skip_base1.avsc} (78%)
 copy java/adapter/avro/src/test/resources/schema/{test_record.avsc => 
skip/test_skip_base2.avsc} (85%)
 copy java/adapter/avro/src/test/resources/schema/{test_record.avsc => 
skip/test_skip_boolean_expected.avsc} (89%)
 copy java/adapter/avro/src/test/resources/

[arrow] branch master updated (54a54c5 -> 43191b7)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 54a54c5  ARROW-6969: [C++][Dataset] ParquetScanTask defer memory usage
 add 43191b7  ARROW-6598: [Java] Sort the code for ApproxEqualsVisitor and 
provide an interface for custom vector equality

No new revisions were added by this update.

Summary of changes:
 .../apache/arrow/vector/FloatingPointVector.java   |   2 +-
 .../arrow/vector/compare/ApproxEqualsVisitor.java  |  87 +++-
 .../vector/compare/VectorValueEqualizer.java}  |  27 ++--
 .../compare/util/ValueEpsilonEqualizers.java   | 149 +
 4 files changed, 186 insertions(+), 79 deletions(-)
 copy 
java/{algorithm/src/main/java/org/apache/arrow/algorithm/sort/OutOfPlaceVectorSorter.java
 => 
vector/src/main/java/org/apache/arrow/vector/compare/VectorValueEqualizer.java} 
(53%)
 create mode 100644 
java/vector/src/main/java/org/apache/arrow/vector/compare/util/ValueEpsilonEqualizers.java

[arrow] branch master updated (ee5b04a -> ac99ca0)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from ee5b04a  ARROW-6866: [Java] Improve the performance of calculating 
hash code for struct vector
 add ac99ca0  ARROW-6933: [Java] Suppor linear dictionary encoder

No new revisions were added by this update.

Summary of changes:
 ...ryEncoder.java => LinearDictionaryEncoder.java} | 60 +-
 .../arrow/algorithm/search/VectorSearcher.java |  2 +-
 ...coder.java => TestLinearDictionaryEncoder.java} | 36 +
 3 files changed, 51 insertions(+), 47 deletions(-)
 copy 
java/algorithm/src/main/java/org/apache/arrow/algorithm/dictionary/{SearchDictionaryEncoder.java
 => LinearDictionaryEncoder.java} (60%)
 copy 
java/algorithm/src/test/java/org/apache/arrow/algorithm/dictionary/{TestSearchDictionaryEncoder.java
 => TestLinearDictionaryEncoder.java} (89%)

[arrow] branch master updated (7fc4a37 -> ee5b04a)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 7fc4a37  ARROW-6672: [Java] Extract a common interface for dictionary 
builders
 add ee5b04a  ARROW-6866: [Java] Improve the performance of calculating 
hash code for struct vector

No new revisions were added by this update.

Summary of changes:
 .../arrow/vector/complex/NonNullableStructVector.java | 15 +++
 1 file changed, 7 insertions(+), 8 deletions(-)

[arrow] branch master updated (776165c -> 7fc4a37)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 776165c  ARROW-6394 [Java] Support conversions between delta vector 
and partial sum vector
 add 7fc4a37  ARROW-6672: [Java] Extract a common interface for dictionary 
builders

No new revisions were added by this update.

Summary of changes:
 .../algorithm/dictionary/DictionaryBuilder.java| 72 ++
 .../HashTableBasedDictionaryBuilder.java   | 29 ++---
 .../SearchTreeBasedDictionaryBuilder.java  | 34 +++---
 ...va => TestHashTableBasedDictionaryBuilder.java} |  2 +-
 4 files changed, 84 insertions(+), 53 deletions(-)
 create mode 100644 
java/algorithm/src/main/java/org/apache/arrow/algorithm/dictionary/DictionaryBuilder.java
 rename 
java/algorithm/src/test/java/org/apache/arrow/algorithm/dictionary/{TestHashTableBasedDictionaryEncoder.java
 => TestHashTableBasedDictionaryBuilder.java} (99%)

[arrow] branch master updated (89080e4 -> 776165c)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 89080e4  ARROW-6966: [Go] Set a default memset for when the platform 
doesn't set one
 add 776165c  ARROW-6394 [Java] Support conversions between delta vector 
and partial sum vector

No new revisions were added by this update.

Summary of changes:
 .../arrow/algorithm/misc/PartialSumUtils.java  | 119 ++
 .../arrow/algorithm/misc/TestPartialSumUtils.java  | 138 +
 2 files changed, 257 insertions(+)
 create mode 100644 
java/algorithm/src/main/java/org/apache/arrow/algorithm/misc/PartialSumUtils.java
 create mode 100644 
java/algorithm/src/test/java/org/apache/arrow/algorithm/misc/TestPartialSumUtils.java

[arrow] branch master updated (32a1e5c -> d923462)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 32a1e5c  ARROW-6938: [Packaging][Python] Disable bz2 in Windows wheels 
and build ZSTD in bundled mode to triage linking issues
 add d923462  ARROW-6932: [JAVA] incorrect log on known extension type

No new revisions were added by this update.

Summary of changes:
 .../src/main/java/org/apache/arrow/vector/types/pojo/Field.java  | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

[arrow] branch master updated (4d6b68e -> 3207ac9)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 4d6b68e  ARROW-6898: [Java][hotfix] fix ArrowWriter memory leak
 add 3207ac9  ARROW-6916: [Developer] Sort tasks by name in Crossbow e-mail 
report

No new revisions were added by this update.

Summary of changes:
 dev/tasks/crossbow.py   | 9 +
 docs/source/developers/crossbow.rst | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

[arrow] branch master updated (0475455 -> 4d6b68e)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 0475455  ARROW-6728: [C#] Support reading and writing Date32 and 
Date64 arrays
 add 4d6b68e  ARROW-6898: [Java][hotfix] fix ArrowWriter memory leak

No new revisions were added by this update.

Summary of changes:
 .../main/java/org/apache/arrow/vector/ipc/ArrowWriter.java   | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

[arrow] branch master updated (3675073 -> 0475455)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 3675073  ARROW-6795: [C#] Fix for reading large (2GB+) files
 add 0475455  ARROW-6728: [C#] Support reading and writing Date32 and 
Date64 arrays

No new revisions were added by this update.

Summary of changes:
 csharp/src/Apache.Arrow/Arrays/ArrowArrayFactory.cs | 2 ++
 csharp/src/Apache.Arrow/Ipc/ArrowStreamWriter.cs| 8 ++--
 csharp/test/Apache.Arrow.Tests/TestData.cs  | 8 ++--
 3 files changed, 10 insertions(+), 8 deletions(-)

[arrow] branch master updated (a81db80 -> 3675073)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from a81db80  ARROW-6813: [Ruby] Arrow::Table.load with headers=true leads 
to exception in Arrow 0.15
 add 3675073  ARROW-6795: [C#] Fix for reading large (2GB+) files

No new revisions were added by this update.

Summary of changes:
 .../src/Apache.Arrow/Ipc/ArrowFileReaderImplementation.cs   | 13 -
 1 file changed, 4 insertions(+), 9 deletions(-)

[arrow] branch master updated (1766eb9 -> 9f7cef1)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 1766eb9  ARROW-6898: [Java] Fix potential memory leak in ArrowWriter 
and several test classes
 add 9f7cef1  ARROW-6850: [Java] Jdbc converter support Null type

No new revisions were added by this update.

Summary of changes:
 .../arrow/adapter/jdbc/JdbcToArrowUtils.java   |  7 
 .../{BitConsumer.java => NullConsumer.java}| 27 -
 .../adapter/jdbc/AbstractJdbcToArrowTest.java  |  1 +
 .../arrow/adapter/jdbc/JdbcToArrowTestHelper.java  |  5 +++
 .../adapter/jdbc/h2/JdbcToArrowDataTypesTest.java  |  9 -
 .../arrow/adapter/jdbc/h2/JdbcToArrowTest.java |  4 ++
 .../test/resources/h2/test1_all_datatypes_h2.yml   | 25 ++--
 .../h2/{test1_bit_h2.yml => test1_null_h2.yml} | 44 +-
 8 files changed, 62 insertions(+), 60 deletions(-)
 copy 
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/consumer/{BitConsumer.java
 => NullConsumer.java} (62%)
 copy java/adapter/jdbc/src/test/resources/h2/{test1_bit_h2.yml => 
test1_null_h2.yml} (58%)

[arrow] branch master updated (edae4a1 -> 1766eb9)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from edae4a1  ARROW-6671: [C++][Python] Use more consistent names for 
sparse tensor items
 add 1766eb9  ARROW-6898: [Java] Fix potential memory leak in ArrowWriter 
and several test classes

No new revisions were added by this update.

Summary of changes:
 .../org/apache/arrow/vector/ipc/ArrowWriter.java   |6 +-
 .../arrow/vector/TestBufferOwnershipTransfer.java  |   16 +
 .../vector/complex/writer/TestComplexWriter.java   | 1036 ++--
 .../arrow/vector/ipc/TestArrowReaderWriter.java|  133 ++-
 4 files changed, 642 insertions(+), 549 deletions(-)

[arrow] branch master updated (bd488e7 -> 0c4ec7b)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from bd488e7  ARROW-6659: [Rust] [DataFusion] Refactor of HashAggregateExec 
to support custom merge
 add 0c4ec7b  ARROW-6853: [Java] Support vector and dictionary encoder use 
different hasher for calculating hashCode

No new revisions were added by this update.

Summary of changes:
 .../apache/arrow/memory/util/ByteFunctionHelpers.java  | 12 +++-
 .../vector/src/main/codegen/templates/UnionVector.java |  9 -
 .../org/apache/arrow/vector/BaseFixedWidthVector.java  |  8 +++-
 .../apache/arrow/vector/BaseVariableWidthVector.java   |  8 +++-
 .../main/java/org/apache/arrow/vector/NullVector.java  |  6 ++
 .../main/java/org/apache/arrow/vector/ValueVector.java |  8 +++-
 .../main/java/org/apache/arrow/vector/ZeroVector.java  |  6 ++
 .../arrow/vector/complex/FixedSizeListVector.java  |  8 +++-
 .../org/apache/arrow/vector/complex/ListVector.java|  8 +++-
 .../arrow/vector/complex/NonNullableStructVector.java  |  8 +++-
 .../org/apache/arrow/vector/complex/StructVector.java  |  8 +++-
 .../arrow/vector/dictionary/DictionaryEncoder.java | 13 ++---
 .../arrow/vector/dictionary/DictionaryHashTable.java   | 18 ++
 .../arrow/vector/types/pojo/TestExtensionType.java |  8 +++-
 14 files changed, 111 insertions(+), 17 deletions(-)

[arrow] branch master updated (08d6343 -> a75e1b7)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 08d6343  ARROW-6184: [Java] Provide hash table based dictionary encoder
 add a75e1b7  ARROW-6452: [Java] Override ValueVector toString() method

No new revisions were added by this update.

Summary of changes:
 .../src/main/codegen/templates/UnionVector.java|  7 ++
 .../org/apache/arrow/vector/BaseValueVector.java   |  9 ++-
 .../arrow/vector/complex/AbstractStructVector.java |  6 ++
 .../arrow/vector/util/ValueVectorUtility.java  | 85 ++
 .../org/apache/arrow/vector/TestValueVector.java   | 58 +++
 5 files changed, 164 insertions(+), 1 deletion(-)
 create mode 100644 
java/vector/src/main/java/org/apache/arrow/vector/util/ValueVectorUtility.java

[arrow] branch master updated (018e1ff -> 08d6343)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 018e1ff  ARROW-6877: [C++] Add additional Boost versions to support 
1.71 and the presumed next 2 future versions
 add 08d6343  ARROW-6184: [Java] Provide hash table based dictionary encoder

No new revisions were added by this update.

Summary of changes:
 .../dictionary/HashTableDictionaryEncoder.java | 144 +
 .../dictionary/SearchDictionaryEncoder.java|  17 +--
 ...er.java => TestHashTableDictionaryEncoder.java} |  54 
 .../dictionary/TestSearchDictionaryEncoder.java|   2 +-
 4 files changed, 173 insertions(+), 44 deletions(-)
 create mode 100644 
java/algorithm/src/main/java/org/apache/arrow/algorithm/dictionary/HashTableDictionaryEncoder.java
 copy 
java/algorithm/src/test/java/org/apache/arrow/algorithm/dictionary/{TestSearchDictionaryEncoder.java
 => TestHashTableDictionaryEncoder.java} (85%)

[arrow] branch master updated (c8bcd70 -> b9203a9)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from c8bcd70  ARROW-6732: [Java] Implement quick sort in a non-recursive 
way to avoid stack overflow
 add b9203a9  ARROW-6074: [FlightRPC][Java] Middleware

No new revisions were added by this update.

Summary of changes:
 .../org/apache/arrow/flight/FlightGrpcUtils.java   |   3 +-
 ...lightRuntimeException.java => CallHeaders.java} |  39 ++-
 .../flight/{FlightConstants.java => CallInfo.java} |  12 +-
 .../java/org/apache/arrow/flight/CallStatus.java   |   9 +
 .../java/org/apache/arrow/flight/FlightClient.java |  34 +-
 .../arrow/flight/FlightClientMiddleware.java   |  52 +++
 .../java/org/apache/arrow/flight/FlightMethod.java |  61 
 .../org/apache/arrow/flight/FlightProducer.java|  12 +
 .../arrow/flight/FlightRuntimeException.java   |   6 +
 .../java/org/apache/arrow/flight/FlightServer.java |  33 +-
 .../arrow/flight/FlightServerMiddleware.java   |  99 ++
 .../org/apache/arrow/flight/FlightService.java | 147 ++---
 .../org/apache/arrow/flight/FlightStatusCode.java  |   4 +
 .../java/org/apache/arrow/flight/FlightStream.java |   9 +
 .../java/org/apache/arrow/flight/StreamPipe.java   |  30 +-
 .../flight/grpc/ClientInterceptorAdapter.java  | 149 +
 .../grpc/ContextPropagatingExecutorService.java| 117 +++
 .../apache/arrow/flight/grpc/MetadataAdapter.java  |  72 +
 .../flight/grpc/ServerInterceptorAdapter.java  | 142 
 .../org/apache/arrow/flight/grpc/StatusUtils.java  |   4 +
 .../apache/arrow/flight/TestClientMiddleware.java  | 211 
 .../apache/arrow/flight/TestServerMiddleware.java  | 360 +
 .../org/apache/arrow/flight/perf/TestPerf.java |   7 +-
 23 files changed, 1538 insertions(+), 74 deletions(-)
 copy 
java/flight/src/main/java/org/apache/arrow/flight/{FlightRuntimeException.java 
=> CallHeaders.java} (56%)
 copy java/flight/src/main/java/org/apache/arrow/flight/{FlightConstants.java 
=> CallInfo.java} (76%)
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/FlightClientMiddleware.java
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/FlightMethod.java
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/FlightServerMiddleware.java
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/grpc/ClientInterceptorAdapter.java
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/grpc/ContextPropagatingExecutorService.java
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/grpc/MetadataAdapter.java
 create mode 100644 
java/flight/src/main/java/org/apache/arrow/flight/grpc/ServerInterceptorAdapter.java
 create mode 100644 
java/flight/src/test/java/org/apache/arrow/flight/TestClientMiddleware.java
 create mode 100644 
java/flight/src/test/java/org/apache/arrow/flight/TestServerMiddleware.java

[arrow] branch master updated (e7892a2 -> c8bcd70)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from e7892a2  ARROW-6721: [JAVA] Avro adapter benchmark only runs once in 
JMH
 add c8bcd70  ARROW-6732: [Java] Implement quick sort in a non-recursive 
way to avoid stack overflow

No new revisions were added by this update.

Summary of changes:
 .../sort/FixedWidthInPlaceVectorSorter.java| 35 +--
 .../apache/arrow/algorithm/sort/IndexSorter.java   | 36 +--
 .../arrow/algorithm/sort/OffHeapIntStack.java  | 72 ++
 .../sort/TestFixedWidthInPlaceVectorSorter.java| 31 ++
 .../arrow/algorithm/sort/TestIndexSorter.java  | 33 ++
 .../arrow/algorithm/sort/TestOffHeapIntStack.java} | 43 -
 6 files changed, 221 insertions(+), 29 deletions(-)
 create mode 100644 
java/algorithm/src/main/java/org/apache/arrow/algorithm/sort/OffHeapIntStack.java
 copy 
java/{vector/src/test/java/org/apache/arrow/vector/TestIntervalYearVector.java 
=> 
algorithm/src/test/java/org/apache/arrow/algorithm/sort/TestOffHeapIntStack.java}
 (53%)

[arrow] branch master updated (2066e59 -> e7892a2)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 2066e59  ARROW-1638: [Java] IPC roundtrip for null type
 add e7892a2  ARROW-6721: [JAVA] Avro adapter benchmark only runs once in 
JMH

No new revisions were added by this update.

Summary of changes:
 .../test/java/org/apache/arrow/adapter/AvroAdapterBenchmarks.java  | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

[arrow] branch master updated (929c9f6 -> 2066e59)

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.


from 929c9f6  ARROW-6711: [C++] Consolidate Filter and Expression
 add 2066e59  ARROW-1638: [Java] IPC roundtrip for null type

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/arrow/AvroToArrowUtils.java|   6 +-
 .../apache/arrow/consumers/AvroNullConsumer.java   |  14 +-
 .../vector/{ZeroVector.java => NullVector.java}| 105 ++
 .../java/org/apache/arrow/vector/ZeroVector.java   | 223 +++--
 .../arrow/vector/compare/RangeEqualsVisitor.java   |   4 +-
 .../arrow/vector/compare/TypeEqualsVisitor.java|   4 +-
 .../apache/arrow/vector/compare/VectorVisitor.java |   4 +-
 .../apache/arrow/vector/ipc/JsonFileReader.java|   5 +
 .../java/org/apache/arrow/vector/types/Types.java  |   4 +-
 .../org/apache/arrow/vector/ipc/BaseFileTest.java  |  21 ++
 .../arrow/vector/ipc/TestArrowReaderWriter.java|  39 
 .../org/apache/arrow/vector/ipc/TestJSONFile.java  |  33 +++
 12 files changed, 203 insertions(+), 259 deletions(-)
 copy java/vector/src/main/java/org/apache/arrow/vector/{ZeroVector.java => 
NullVector.java} (77%)

[arrow] branch master updated: ARROW-6710: [Java] Add JDBC adapter test to cover cases which contains some null values

[emkornfield]

This is an automated email from the ASF dual-hosted git repository.

emkornfield pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
 new 10838a0  ARROW-6710: [Java] Add JDBC adapter test to cover cases which 
contains some null values
10838a0 is described below

commit 10838a0efefb970f53db50a5650cfaee4c5af202
Author: tianchen 
AuthorDate: Sat Oct 5 17:18:25 2019 -0600

ARROW-6710: [Java] Add JDBC adapter test to cover cases which contains some 
null values

Related to [ARROW-6710](https://issues.apache.org/jira/browse/ARROW-6710).

The current JDBC adapter tests only cover the cases that values are all 
non-null or all null.
However, the cases that ResultSet has some null values are not covered (see 
ARROW-6709).

Closes #5521 from tianchen92/ARROW-6710 and squashes the following commits:

bc2c4588f  fix style
e24d1918a  small fix
19ebe7344  ARROW-6710:  Add JDBC adapter test to cover cases 
which contains some null values

Authored-by: tianchen 
Signed-off-by: Micah Kornfield 
---
 .../adapter/jdbc/AbstractJdbcToArrowTest.java  |  19 
 .../arrow/adapter/jdbc/JdbcToArrowTestHelper.java  | 103 -
 .../adapter/jdbc/h2/JdbcToArrowCharSetTest.java|   3 -
 .../arrow/adapter/jdbc/h2/JdbcToArrowNullTest.java |  84 -
 .../arrow/adapter/jdbc/h2/JdbcToArrowTest.java |  19 
 .../test1_all_datatypes_selected_null_rows_h2.yml  |  83 +
 6 files changed, 264 insertions(+), 47 deletions(-)

diff --git 
a/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/AbstractJdbcToArrowTest.java
 
b/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/AbstractJdbcToArrowTest.java
index 9fb37ec..44a0604 100644
--- 
a/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/AbstractJdbcToArrowTest.java
+++ 
b/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/AbstractJdbcToArrowTest.java
@@ -35,6 +35,25 @@ import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
  * Class to abstract out some common test functionality for testing JDBC to 
Arrow.
  */
 public abstract class AbstractJdbcToArrowTest {
+
+  protected static final String BIGINT = "BIGINT_FIELD5";
+  protected static final String BINARY = "BINARY_FIELD12";
+  protected static final String BIT = "BIT_FIELD17";
+  protected static final String BLOB = "BLOB_FIELD14";
+  protected static final String BOOL = "BOOL_FIELD2";
+  protected static final String CHAR = "CHAR_FIELD16";
+  protected static final String CLOB = "CLOB_FIELD15";
+  protected static final String DATE = "DATE_FIELD10";
+  protected static final String DECIMAL = "DECIMAL_FIELD6";
+  protected static final String DOUBLE = "DOUBLE_FIELD7";
+  protected static final String INT = "INT_FIELD1";
+  protected static final String REAL = "REAL_FIELD8";
+  protected static final String SMALLINT = "SMALLINT_FIELD4";
+  protected static final String TIME = "TIME_FIELD9";
+  protected static final String TIMESTAMP = "TIMESTAMP_FIELD11";
+  protected static final String TINYINT = "TINYINT_FIELD3";
+  protected static final String VARCHAR = "VARCHAR_FIELD13";
+
   protected Connection conn = null;
   protected Table table;
 
diff --git 
a/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/JdbcToArrowTestHelper.java
 
b/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/JdbcToArrowTestHelper.java
index 3e1eb09..748cfb5 100644
--- 
a/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/JdbcToArrowTestHelper.java
+++ 
b/java/adapter/jdbc/src/test/java/org/apache/arrow/adapter/jdbc/JdbcToArrowTestHelper.java
@@ -56,7 +56,11 @@ public class JdbcToArrowTestHelper {
 assertEquals(rowCount, intVector.getValueCount());
 
 for (int j = 0; j < intVector.getValueCount(); j++) {
-  assertEquals(values[j].intValue(), intVector.get(j));
+  if (values[j] == null) {
+assertTrue(intVector.isNull(j));
+  } else {
+assertEquals(values[j].intValue(), intVector.get(j));
+  }
 }
   }
 
@@ -64,7 +68,11 @@ public class JdbcToArrowTestHelper {
 assertEquals(rowCount, bitVector.getValueCount());
 
 for (int j = 0; j < bitVector.getValueCount(); j++) {
-  assertEquals(values[j].booleanValue(), bitVector.get(j) == 1);
+  if (values[j] == null) {
+assertTrue(bitVector.isNull(j));
+  } else {
+assertEquals(values[j].booleanValue(), bitVector.get(j) == 1);
+  }
 }
   }
 
@@ -72,7 +80,11 @@ public class JdbcToArrowTestHelper {
 assertEquals(rowCount, bitVector.getValueCount());
 
 for (int j = 0; j < bitVector.getValueCount(); j++) {
-  assertEquals(values[j].i

svn commit: r36093 - /dev/arrow/KEYS

[emkornfield]

Author: emkornfield
Date: Fri Sep 27 14:54:32 2019
New Revision: 36093

Log:
Add emkornfi...@gmail.com to KEYS to dev

Modified:
dev/arrow/KEYS

Modified: dev/arrow/KEYS
==
--- dev/arrow/KEYS (original)
+++ dev/arrow/KEYS Fri Sep 27 14:54:32 2019
@@ -964,3 +964,62 @@ OmhlBe/22X2+2wzVHz7Kt28PZKPg5857GglX76Av
 o+SFxkD/tGsGK97MtW8kpgrzonWP7zAAIpN3WlGMoQ==
 =TsqO
 -END PGP PUBLIC KEY BLOCK-
+pub   rsa4096 2019-09-24 [SC] [expires: 2020-09-23]
+  E47C810A90FE21FF448DA938755E743692EA1D85
+uid   [ unknown] Micah Kornfield (Apache Arrow Code Signing Key) 

+sig 3755E743692EA1D85 2019-09-24  Micah Kornfield (Apache Arrow Code 
Signing Key) 
+sub   rsa4096 2019-09-24 [E] [expires: 2020-09-23]
+sig  755E743692EA1D85 2019-09-24  Micah Kornfield (Apache Arrow Code 
Signing Key) 
+
+-BEGIN PGP PUBLIC KEY BLOCK-
+
+mQINBF2JpYwBEADVQ/Rhp2/0Smus68iXpBLCNAinGwaXvJU+Ax7vHM8C1sdgoyvh
+RF4gixaBZe2UMUf4VAgv0XjSWmfHKjmZuhA762moPzX/GM9RWqY9o0xOVhWi61Zv
+uXuVRdU2cQ0D+L7bsrLl6SUdMtwasWCe2CDJSzEcDwyyCoIrKR/BEdZ9ghLue4R4
+lqKYgb48hM1Yr0kzROE+bit/ittuyf90stja2qDyMSR5V51yDQ6d1v9fO88YUYRf
+o5PwjYVXZ3QQy+qlUbsinBAszlFsRCXphhe2tQ1+2PVD3aoAJJo/BCc6C1YbpSuF
+iRqbnyzBU2DeSfHSNbcEOwy5XYS1jmqKni4HEmFZs1P29ZeJiA9Uljm+JHz+w5pq
+9iAq/U2XAx3LSGDPSQv7dqjHRtmvZmhns0cC+C1I+V8l+n9XIzQwbRtvB0ucLrfn
+e3GTNvoabLPinTDFE2jqFEgCTcY6eJrHAEz1AAk92U7wgTpoMht3jEfbPS+7w4tS
+n3vq17GuNqvmfUIRowM+bMbrt7hcB0yJOKmnmM+lx1Fcxfxg8FOy890a54gVDuvQ
+VmKsF05Clgw3Th8e2HEty+iEYnl6anrVO47hsfOa8MxeCtzyBYI2LmX9TYix0Uc6
+Fb8lLttw05a/k+vAfWVUkfoI/WYiNOrun85IHxdcsf7E4dGmJ1Fc9rO9KQARAQAB
+tEdNaWNhaCBLb3JuZmllbGQgKEFwYWNoZSBBcnJvdyBDb2RlIFNpZ25pbmcgS2V5
+KSA8ZW1rb3JuZmllbGRAZ21haWwuY29tPokCVAQTAQoAPhYhBOR8gQqQ/iH/RI2p
+OHVedDaS6h2FBQJdiaWMAhsDBQkB4TOABQsJCAcDBRUKCQgLBRYCAwEAAh4BAheA
+AAoJEHVedDaS6h2FT0YQALxA9Vv24pXpZBJbCn5oLVh+t6Lee9BjYP0+Mv7wHg0t
+jRGDp0i3UUmYjBq5laEJHj/zAse11kSecBUCkOatqRmeWOk2wewjmxuklKvINdji
+Ir7uagqSe8m7QwCACAekRvqCMrKdYL1mJKIg/ZPHJZ/MP8QAZDn6AS3OKf1PFXye
+tbes5xNJTRC6YDt8X1RnJwbxwRJ56BSwIczgtZkNldEfPpc6+89it5wMxGRHyB9L
+HLCjQpfS6ksCNIG/TnNkjQwbh9bUVqLbUkJtnIlUGh0jgl4JRGVqJ97jyrzJnhGe
+AYq3j795P4t3lCwiT/9czoSjnZmaYsTH+Dao4aQswUZORPsBNf3UhjHv0aK3+99U
+DjeArPxsq+xo1gwFo6+hPAtuRZcv2Vurl8vvrldWXxmbdXvoPLzbbtLbRW44572R
+pSLNUQaW+paNYZdwWjRAE+4LPg1QX4MuLYcHpBy3uT2oYzBBLvsZzh8plT5jDSwd
+S2SicLrhPIwWfKBWVyBCek/Yxk8x7nksnFlranZIG3oHVU7g6xEQERRQ2jPY+UMb
+U1CS3aXlgCD3GTSSDCnxlRawaA0T//VjFxVUEjkp//FeTD4SQpGPhWZxDfYoTgKk
+anO0hKAnMmpqyD15i6y/W8hjWti2HlATvJecOR/NlTIA4vIhcPpERY6kZYNMZCQ9
+uQINBF2JpYwBEADWesN7gLqaR5otb+ezxdQUXo3pntPQNP5j7xpAqwyqgXdmnzMA
+JB73XnyIVnCxp2GOm2hH+bcptZCmoWyVQEi+C3zJJ2LyIoTidQFDeVVgRTmDOsoG
+vidLorCIKHCTPbMmkS7XiCZmzB8BZeCUtulOwNXq9z2wZyO+toiQx/9502hj1o6e
+T1So/gfY7snS5wh/Ugt2UvWDcwx0ZqpPIVwt0qvsUCvs45aoA9YnL4I++9cjxM0s
+0rUKhQ4gind1ZC9EZr/wI6YAB0ykB+Ew6nBEwcIvyR5ocO+aIj5sw8Bhn3nAw1Ka
+nwACQ8kb5979IbZ17F2EbloCPMIYpEkSEbT1d0lxWBCF1KvAcOX3WT3cIMHAqtzZ
+kenohS5skNvHFxzax7P9j6MjKa+oCI0TMCYIcaCocFNihVs5KyCGa1G8Oq7a2cw5
+yg42Pq8tPgpGtOP0HyMay1tIRRTEnf8/KqdNrL3HcV3/uZH2vd8dXJaIzzkwlFLB
+eKZOCnM8kyzyVIvIMymFiU71Bt3pxwg+c8fjJ+/0NorRYlDPRdQY8l5m0zqEzsdh
+9nZTTWSKRIhzIyfb46VaSy+R7CGPeW2TWgNVaN01nhSry3AwjJfRTi8Xzhi+9gB9
++KE+gMfdycCeSgRwdHq/ovdS28Cw9MFutQDTKrksjdc9RNCrhVJC8xciLQARAQAB
+iQI8BBgBCgAmFiEE5HyBCpD+If9Ejak4dV50NpLqHYUFAl2JpYwCGwwFCQHhM4AA
+CgkQdV50NpLqHYXn2A/+MVhG30kNc8yd3EW4MZOgfFa5mkeEo+iUavXaz7XoAiMh
+BlgqlLIGMuzFm5tpKpPoDT9uw85GuNr9x1U/iXHQU3DcyYzUcHw2Rtubm7nwXpyV
+ZPVEKPY/V/9tt73wdufas7/s8K/uhEX66rUXA3kMVqksFAUgyWSR7cWzMdLkOKrr
+bWKVKNQolpmmaPSc84x99um3REgfBTpyxZzSa/rOmR2vdNz3DSZJOgPfEFkCux9H
+1Jexqq/er5XH5SCo0zE5N+hgrdjuPER0b03TPEto3E6r2D0InV5NtA8kCxKKk6UY
+8NFstrBSECAXMRN7xrY5VuGracF9zDX192fzm0IFxtdATSGl5wll1oiYqiqBOegc
+yAgAaztTYn5700AtkpYcjsfENehbm5wr/y6RU/22I0p8ifU1zGUQMIMTbShoy8ZA
+VBJ43JMtbHt/aH4vPO0871uVnyKxGU9MQABWyVQ9g0LoGCuMkZIjNN1r8R+ZMmSR
+d7pLKbzC/jf7ggSaTLUFXipGx8Utitle2NSs6wZWUo3mWrQeOS57fuKQhTGnJDPd
+Hg3Sc5RHLw/lIC/yEtIptzcpDLgDxS1Bf2c+RF+7K6Itms/aIpJRKEfWZsNcIeRz
+pyqcs9cFy0tt3nAdWcVobS3PYhSacIDlPFPImatg3Nr+p50dx94OdWvoTzZPWTY=
+=FdOt
+-END PGP PUBLIC KEY BLOCK-

svn commit: r36077 - in /dev/arrow/apache-arrow-0.15.0-rc1: ./ apache-arrow-0.15.0.tar.gz apache-arrow-0.15.0.tar.gz.asc apache-arrow-0.15.0.tar.gz.sha256 apache-arrow-0.15.0.tar.gz.sha512

[emkornfield]

Author: emkornfield
Date: Thu Sep 26 15:49:27 2019
New Revision: 36077

Log:
Apache Arrow 0.15.0 RC1

Added:
dev/arrow/apache-arrow-0.15.0-rc1/
dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz   (with props)
dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.asc
dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha256
dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha512

Added: dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz
==
Binary file - no diff available.

Propchange: dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz
--
svn:mime-type = application/octet-stream

Added: dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.asc
==
--- dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.asc (added)
+++ dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.asc Thu Sep 26 
15:49:27 2019
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEE5HyBCpD+If9Ejak4dV50NpLqHYUFAl2M3fEACgkQdV50NpLq
+HYXAIw//ak5KaqoMFWLGLNDnZ9L9oEuNYuoCJ0YyxkDVGExDMMAt7QYGcU6Yh/5s
+VSGnldsmSStkt8sAKJE9LgnUC31ujGEpSYpe+aAhCinB7M+knxVE7IcSUozgfP1w
+AGvWfQePAhFvqKDZ8En5Wi5YbFKBzmoqGD/cpDGahUN/XjT8aTpZkMAqLT1N2bYM
+QK2D3ykgu/eg9DSOXlDXq/uKqTXCXdSxFo+oAwewh26YW+GLo7uu2MG83/RO7FZg
+tOL1xFZyaouWhQB0S1H4GuXZe28kbW/mIXpf7YvVj4qURRwbwGfAzFZQU8/ib9s/
+9XiIVFuPyDgFLB9RiYn/AtyR4OU8azwgpvdSht8IM1KSHtB4optZeqw7411ePaUg
+SM65bGd5Gl/pDhiY2iZn5CUBzkmU9icPI3he9XMGKwAddOs6KzDXYR82K4qYdQWx
+hvKLt2EIRiljF7unr5Ept8vhOV1Jdk4Z4XPW4d6wtnqUes7yr8W+v5NCak7UVJIO
+b94GfGV9a/2ZlDCiFnpqG2JV02lXZF72jEi4WXAf4Av2JdCAHij5L0JoCBfhqB7N
+8kMpWw0cU+aJO2nS2bHUKJg6t+pzwlmZjlHv+DoTpSYI8jEwFTlA8fOPRxB3qbFN
+jgUaMj5AygP2k85WuvPBRXnwZrU3DYc5aFgjeUVjnuBzArEfZmQ=
+=MsCd
+-END PGP SIGNATURE-

Added: dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha256
==
--- dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha256 (added)
+++ dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha256 Thu Sep 
26 15:49:27 2019
@@ -0,0 +1 @@
+e038672e9d1bc856aadc44dae0f655066784133caee96b77d9f74d2554138323  
apache-arrow-0.15.0.tar.gz

Added: dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha512
==
--- dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha512 (added)
+++ dev/arrow/apache-arrow-0.15.0-rc1/apache-arrow-0.15.0.tar.gz.sha512 Thu Sep 
26 15:49:27 2019
@@ -0,0 +1 @@
+9dd792ca55e4290ad7582029e0c96c04e93f4547f054d3b9a225cebcad57ecc802e8d4076c98f5f1ab0c3a854b2b76cec2c593252f0ebf239cb5bc328ca0c288
  apache-arrow-0.15.0.tar.gz