[
https://issues.apache.org/jira/browse/ARROW-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445480#comment-16445480
]
ASF GitHub Bot commented on ARROW-2471:
---------------------------------------
pitrou closed pull request #1911: ARROW-2471: [Rust] Builder zero capacity fix
URL: https://github.com/apache/arrow/pull/1911
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/rust/src/array.rs b/rust/src/array.rs
index 63d11308d..55881f550 100644
--- a/rust/src/array.rs
+++ b/rust/src/array.rs
@@ -276,6 +276,13 @@ mod tests {
}
}
+ #[test]
+ fn test_from_empty_vec() {
+ let v: Vec<i32> = vec![];
+ let a = Array::from(v);
+ assert_eq!(0, a.len());
+ }
+
#[test]
fn test_from_optional_i32() {
let a = Array::from(vec![Some(1), None, Some(2), Some(3), None]);
diff --git a/rust/src/builder.rs b/rust/src/builder.rs
index 89e00048f..69884ccac 100644
--- a/rust/src/builder.rs
+++ b/rust/src/builder.rs
@@ -83,8 +83,9 @@ impl<T> Builder<T> {
pub fn push(&mut self, v: T) {
assert!(!self.data.is_null());
if self.len == self.capacity {
- let new_capacity = self.capacity;
- self.grow(new_capacity * 2);
+ // grow capacity by 64 bytes or double the current capacity,
whichever is greater
+ let new_capacity = cmp::max(64, self.capacity * 2);
+ self.grow(new_capacity);
}
assert!(self.len < self.capacity);
unsafe {
@@ -174,6 +175,14 @@ mod tests {
assert_eq!(0, a.len());
}
+ #[test]
+ fn test_builder_i32_alloc_zero_bytes() {
+ let mut b: Builder<i32> = Builder::with_capacity(0);
+ b.push(123);
+ let a = b.finish();
+ assert_eq!(1, a.len());
+ }
+
#[test]
fn test_builder_i32() {
let mut b: Builder<i32> = Builder::with_capacity(5);
diff --git a/rust/src/list_builder.rs b/rust/src/list_builder.rs
index 96dca18e6..71547d2f0 100644
--- a/rust/src/list_builder.rs
+++ b/rust/src/list_builder.rs
@@ -55,7 +55,7 @@ mod tests {
use super::*;
#[test]
- fn test_list_u8() {
+ fn test_list_u8_default_capacity() {
let mut b: ListBuilder<u8> = ListBuilder::new();
b.push("Hello, ".as_bytes());
b.push("World!".as_bytes());
@@ -66,6 +66,17 @@ mod tests {
assert_eq!("World!".as_bytes(), buffer.slice(1));
}
+ #[test]
+ fn test_list_u8_zero_capacity() {
+ let mut b: ListBuilder<u8> = ListBuilder::with_capacity(0);
+ b.push("Hello, ".as_bytes());
+ b.push("World!".as_bytes());
+ let buffer = b.finish();
+ assert_eq!(2, buffer.len());
+ assert_eq!("Hello, ".as_bytes(), buffer.slice(0));
+ assert_eq!("World!".as_bytes(), buffer.slice(1));
+ }
+
#[test]
fn test_empty_lists() {
let mut b: ListBuilder<u8> = ListBuilder::new();
diff --git a/rust/src/memory_pool.rs b/rust/src/memory_pool.rs
index acfcc3071..6832cd205 100644
--- a/rust/src/memory_pool.rs
+++ b/rust/src/memory_pool.rs
@@ -16,8 +16,8 @@
// under the License.
use libc;
-use std::mem;
use std::cmp;
+use std::mem;
use super::error::ArrowError;
use super::error::Result;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> [Rust] Assertion when pushing value to Builder/ListBuilder with zero capacity
> -----------------------------------------------------------------------------
>
> Key: ARROW-2471
> URL: https://issues.apache.org/jira/browse/ARROW-2471
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust
> Reporter: Andy Grove
> Assignee: Andy Grove
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> If a builder is created with capacity 0 and then data is pushed to the
> builder, memory reallocation is not triggered and the push fails with an
> assertion.
> The root cause is that memory reallocation calculates the new capacity as 2x
> the current capacity which doesn't work if the current capacity is zero.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
