[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446738#comment-16446738
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-383284486
 
 
   @paddyhoran I think this is fine. Once you have done that I will then do 
another refactor to make the builders use the memory pool abstraction.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445701#comment-16445701
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-383092381
 
 
   @paddyhoran I think this is fine. Once you have done that I will then do 
another refactor to make the builders use the memory pool abstraction.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445647#comment-16445647
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-383077465
 
 
   @liurenjie1024 ok.  I'll take a look at this when I get a chance.  Right now 
though I'm going to see if I can conditionally compile this to get 
[ARROW-2474](https://issues.apache.org/jira/browse/ARROW-2474?filter=12343557) 
done.  The memory pool abstraction is not used anywhere within the codebase and 
with the fix I am proposing would not be available on windows.  I'll open a new 
JIRA to track the refactor work.
   
   @andygrove are you ok with this?  I know this is not a best practice and I 
would not want to get into the habit of doing this, I just want to get windows 
building again so I can work on windows CI.  Given that we are not sure how the 
allocator API will impact the need for a memory pool I think it's ok in this 
situation?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445536#comment-16445536
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-383041102
 
 
   @paddyhoran Can you post a PR for that? If it really blocks windows support, 
I'm certainly happy to accept the refactoring.  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445237#comment-16445237
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382962137
 
 
   @liurenjie1024 thanks.  
   
   I was looking at the C++ implementation for reference:
- I see that we basically do a reinterpret_cast before using the `*const 
u8` version.
- I thought that `usize` is really for pointers, in the C++ implementation 
they use i64 for size
   
   How do you feel about refactoring this implementation to use the functions 
in `memory.rs` once @andygrove's PR above is merged?
   
   I'm having trouble adding windows support and it's not used anywhere else in 
the codebase...


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445234#comment-16445234
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382962137
 
 
   @liurenjie1024 thanks.  
   
   I was looking at the C++ implementation for reference:
- I see that we basically do a reinterpret_cast before using the `*const 
u8` version.
- I thought that `usize` is really for pointers, in the C++ implementation 
they use i64 for size
   
   How do you feel about refactoring this implementation to use the functions 
in `memory.rs` once @andygrove's PR above is merged? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445231#comment-16445231
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382961933
 
 
   My initial plan was to create three memory pool implementations: libc, 
windows and one using allocator api (which is only available in nightly 
channel). Since we are not sure when allocator api will get stable, I think it 
still deserves the effort on it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445201#comment-16445201
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382954515
 
 
   Here is the PR:
   
   https://github.com/apache/arrow/pull/1919
   
   I haven't put a JIRA number on this yet.
   
   On Thu, Apr 19, 2018 at 8:50 PM, Andy Grove  wrote:
   
   > I commented on the JIRA but will comment here too.
   >
   > We now have two different memory implementations but only one is actually
   > used.
   >
   > Also my original code has libc calls scattered around instead of being
   > isolated in one file, which isn't good and has also meant having to deal
   > with Windows in multiple places instead of just in the memory related
   > source files.
   >
   > I'm looking at this now to try and figure out what we should do. I think
   > we will just want to use Rust's allocator API anyway once that becomes
   > stable so I'm not sure we really need a pluggable memory pool but I guess
   > we can have that discussion later.
   >
   > I'm thinking I should at least submit a PR to move all the current memory
   > handling into the memory.rs file.
   >
   >
   >
   >
   >
   > On Thu, Apr 19, 2018 at 8:07 PM, Renjie Liu 
   > wrote:
   >
   >> Hi @paddyhoran ,
   >> in fact these are conventions inherited from allocator api, and here are
   >> my thoughts:
   >>
   >>- when you allocate a new piece of memory, in most cases it's used
   >>for storing data, and you need to write to it.
   >>- usize is a type designed for size and it has different length in
   >>different platforms for the sake of efficiency. If you use i64 in 
32-bit
   >>platforms, it may be wasteful and hurt performance.
   >>
   >> —
   >> You are receiving this because you were mentioned.
   >> Reply to this email directly, view it on GitHub
   >> , or 
mute
   >> the thread
   >> 

   >> .
   >>
   >
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445192#comment-16445192
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382953309
 
 
   I commented on the JIRA but will comment here too.
   
   We now have two different memory implementations but only one is actually
   used.
   
   Also my original code has libc calls scattered around instead of being
   isolated in one file, which isn't good and has also meant having to deal
   with Windows in multiple places instead of just in the memory related
   source files.
   
   I'm looking at this now to try and figure out what we should do. I think we
   will just want to use Rust's allocator API anyway once that becomes stable
   so I'm not sure we really need a pluggable memory pool but I guess we can
   have that discussion later.
   
   I'm thinking I should at least submit a PR to move all the current memory
   handling into the memory.rs file.
   
   
   
   
   
   On Thu, Apr 19, 2018 at 8:07 PM, Renjie Liu 
   wrote:
   
   > Hi @paddyhoran ,
   > in fact these are conventions inherited from allocator api, and here are
   > my thoughts:
   >
   >- when you allocate a new piece of memory, in most cases it's used for
   >storing data, and you need to write to it.
   >- usize is a type designed for size and it has different length in
   >different platforms for the sake of efficiency. If you use i64 in 32-bit
   >platforms, it may be wasteful and hurt performance.
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > , or mute
   > the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445136#comment-16445136
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382943494
 
 
   Hi @paddyhoran,
   in fact these are conventions inherited from allocator api, and here are my 
thoughts:
   *  when you allocate a new piece of memory, in most cases it's used for 
storing data, and you need to write to it.
   * usize is a type designed for size and it has different length in different 
platforms for the sake of efficiency. If you use i64 in 32-bit platforms, it 
may be wasteful and hurt performance.  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16445125#comment-16445125
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382828635
 
 
   I'm trying to add windows support for these changes and I have a few 
questions:
- Why are we using `*mut u8` instead of `*const u8`?
- Why switch to `usize` for `size` instead of `i64`?
   
   @liurenjie1024 thoughts?  Not questioning your approach, just inexperienced 
and trying to understand...


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1697#comment-1697
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382828635
 
 
   I'm trying to add windows support for these changes and I have a few 
questions:
- Why are we using `*mut u8` instead of `*const u8`?
- Why switch to `usize` for `size` instead of `i64`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1696#comment-1696
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-382828635
 
 
   I'm trying to add windows support for these changes and I have a few 
questions:
- Why are we using `*mut u8` instead of `*const u8`?
- Why switch to `usize` for `size` instead of `i64`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438618#comment-16438618
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

xhochy closed pull request #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.gitignore b/.gitignore
index c902ba39c..d9c69e954 100644
--- a/.gitignore
+++ b/.gitignore
@@ -30,6 +30,7 @@ MANIFEST
 *.vcxproj
 *.vcxproj.*
 *.sln
+*.iml
 
 cpp/.idea/
 python/.eggs/
diff --git a/rust/src/error.rs b/rust/src/error.rs
index 6a342e063..d82ee1190 100644
--- a/rust/src/error.rs
+++ b/rust/src/error.rs
@@ -20,3 +20,5 @@ pub enum ArrowError {
 MemoryError(String),
 ParseError(String),
 }
+
+pub type Result = ::std::result::Result;
diff --git a/rust/src/lib.rs b/rust/src/lib.rs
index 6ab3daabb..2d2274029 100644
--- a/rust/src/lib.rs
+++ b/rust/src/lib.rs
@@ -29,3 +29,4 @@ pub mod datatypes;
 pub mod error;
 pub mod list;
 pub mod memory;
+pub mod memory_pool;
diff --git a/rust/src/memory_pool.rs b/rust/src/memory_pool.rs
new file mode 100644
index 0..acfcc3071
--- /dev/null
+++ b/rust/src/memory_pool.rs
@@ -0,0 +1,105 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use libc;
+use std::mem;
+use std::cmp;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+/// Memory pool for allocating memory. It's also responsible for tracking 
memory usage.
+pub trait MemoryPool {
+/// Allocate memory.
+/// The implementation should ensures that allocated memory is aligned.
+fn allocate(, size: usize) -> Result<*mut u8>;
+
+/// Reallocate memory.
+/// If the implementation doesn't support reallocating aligned memory, it 
allocates new memory
+/// and copied old memory to it.
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8>;
+
+/// Free memory.
+fn free(, ptr: *mut u8);
+}
+
+/// Implementation of memory pool using libc api.
+#[allow(dead_code)]
+struct LibcMemoryPool;
+
+impl MemoryPool for LibcMemoryPool {
+fn allocate(, size: usize) -> Result<*mut u8> {
+unsafe {
+let mut page: *mut libc::c_void = mem::uninitialized();
+let result = libc::posix_memalign( page, ALIGNMENT, size);
+match result {
+0 => Ok(mem::transmute::<*mut libc::c_void, *mut u8>(page)),
+_ => Err(ArrowError::MemoryError(
+"Failed to allocate memory".to_string(),
+)),
+}
+}
+}
+
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8> {
+unsafe {
+let old_src = mem::transmute::<*mut u8, *mut 
libc::c_void>(pointer);
+let result = self.allocate(new_size)?;
+let dst = mem::transmute::<*mut u8, *mut libc::c_void>(result);
+libc::memcpy(dst, old_src, cmp::min(old_size, new_size));
+libc::free(old_src);
+Ok(result)
+}
+}
+
+fn free(, ptr: *mut u8) {
+unsafe { libc::free(mem::transmute::<*mut u8, *mut libc::c_void>(ptr)) 
}
+}
+}
+
+#[cfg(test)]
+mod tests {
+use super::*;
+
+#[test]
+fn test_allocate() {
+let memory_pool = LibcMemoryPool {};
+
+for _ in 0..10 {
+let p = memory_pool.allocate(1024).unwrap();
+// make sure this is 64-byte aligned
+assert_eq!(0, (p as usize) % ALIGNMENT);
+memory_pool.free(p);
+}
+}
+
+#[test]
+fn test_reallocate() {
+let memory_pool = LibcMemoryPool {};
+
+for _ in 0..10 {
+let p1 = memory_pool.allocate(1024).unwrap();
+let p2 = memory_pool.reallocate(1024, 2048, p1).unwrap();
+// make sure this is 64-byte aligned
+

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438394#comment-16438394
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

pitrou commented on issue #1875: ARROW-2435: [Rust] Add memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381338090
 
 
   I'll let @xhochy handle it, since he commented on the PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438386#comment-16438386
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381336647
 
 
   @xhochy @pitrou I think we can merge this one now


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438385#comment-16438385
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381336600
 
 
   @crepererum I'm all for switching to allocator API once stable. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437380#comment-16437380
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381157762
 
 
   @crepererum We can have memory pool as a wrapper allocator api so that we 
can have more functionality, e.g. statistics about memory usage


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437341#comment-16437341
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

crepererum commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381147104
 
 
   Or in other words: do you (@andygrove) think we should switch to the 
upstream API once it is stable?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437337#comment-16437337
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

crepererum commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381146548
 
 
   Could we not use something closer to the hopefully-soon-stable [Allocator 
API](https://doc.rust-lang.org/alloc/allocator/trait.Alloc.html)?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437297#comment-16437297
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-381135426
 
 
   @xhochy I think this looks good now?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436730#comment-16436730
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on a change in pull request #1875: ARROW-2435: [Rust] 
Add memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r181276036
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,90 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use libc;
+use std::mem;
+use std::cmp;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+/// Memory pool for allocating memory. It's also responsible for tracking 
memory usage.
+pub trait MemoryPool {
+/// Allocate memory.
+/// The implementation should ensures that allocated memory is aligned.
+fn allocate(, size: usize) -> Result<*mut u8>;
+
+/// Reallocate memory.
+/// If the implementation doesn't support reallocating aligned memory, it 
allocates new memory
+/// and copied old memory to it.
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8>;
+
+/// Free memory.
+fn free(, ptr: *mut u8);
+}
+
+/// Implementation of memory pool using libc api.
+#[allow(dead_code)]
+struct LibcMemoryPool;
+
+impl MemoryPool for LibcMemoryPool {
+fn allocate(, size: usize) -> Result<*mut u8> {
+unsafe {
+let mut page: *mut libc::c_void = mem::uninitialized();
+let result = libc::posix_memalign( page, ALIGNMENT, size);
+match result {
+0 => Ok(mem::transmute::<*mut libc::c_void, *mut u8>(page)),
+_ => Err(ArrowError::MemoryError(
+"Failed to allocate memory".to_string(),
+)),
+}
+}
+}
+
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8> {
+unsafe {
+let old_src = mem::transmute::<*mut u8, *mut 
libc::c_void>(pointer);
+let result = self.allocate(new_size)?;
+let dst = mem::transmute::<*mut u8, *mut libc::c_void>(result);
+libc::memcpy(dst, old_src, cmp::min(old_size, new_size));
+libc::free(old_src);
+Ok(result)
+}
+}
+
+fn free(, ptr: *mut u8) {
+unsafe { libc::free(mem::transmute::<*mut u8, *mut libc::c_void>(ptr)) 
}
+}
+}
+
+#[cfg(test)]
+mod tests {
+use super::*;
+
+#[test]
+fn test_allocate() {
+let memory_pool = LibcMemoryPool {};
+
+for _ in 0..10 {
+let p = memory_pool.allocate(1024).unwrap();
+// make sure this is 64-byte aligned
+assert_eq!(0, (p as usize) % 64);
+}
 
 Review comment:
   Hi, @andygrove 
   
   Test  added. I tried also to test free but it seems that there is no easy 
way to test that.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434901#comment-16434901
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on a change in pull request #1875: ARROW-2435: [Rust] Add 
memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180959288
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,90 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use libc;
+use std::mem;
+use std::cmp;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+/// Memory pool for allocating memory. It's also responsible for tracking 
memory usage.
+pub trait MemoryPool {
+/// Allocate memory.
+/// The implementation should ensures that allocated memory is aligned.
+fn allocate(, size: usize) -> Result<*mut u8>;
+
+/// Reallocate memory.
+/// If the implementation doesn't support reallocating aligned memory, it 
allocates new memory
+/// and copied old memory to it.
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8>;
+
+/// Free memory.
+fn free(, ptr: *mut u8);
+}
+
+/// Implementation of memory pool using libc api.
+#[allow(dead_code)]
+struct LibcMemoryPool;
+
+impl MemoryPool for LibcMemoryPool {
+fn allocate(, size: usize) -> Result<*mut u8> {
+unsafe {
+let mut page: *mut libc::c_void = mem::uninitialized();
+let result = libc::posix_memalign( page, ALIGNMENT, size);
+match result {
+0 => Ok(mem::transmute::<*mut libc::c_void, *mut u8>(page)),
+_ => Err(ArrowError::MemoryError(
+"Failed to allocate memory".to_string(),
+)),
+}
+}
+}
+
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8> {
+unsafe {
+let old_src = mem::transmute::<*mut u8, *mut 
libc::c_void>(pointer);
+let result = self.allocate(new_size)?;
+let dst = mem::transmute::<*mut u8, *mut libc::c_void>(result);
+libc::memcpy(dst, old_src, cmp::min(old_size, new_size));
+libc::free(old_src);
+Ok(result)
+}
+}
+
+fn free(, ptr: *mut u8) {
+unsafe { libc::free(mem::transmute::<*mut u8, *mut libc::c_void>(ptr)) 
}
+}
+}
+
+#[cfg(test)]
+mod tests {
+use super::*;
+
+#[test]
+fn test_allocate() {
+let memory_pool = LibcMemoryPool {};
+
+for _ in 0..10 {
+let p = memory_pool.allocate(1024).unwrap();
+// make sure this is 64-byte aligned
+assert_eq!(0, (p as usize) % 64);
+}
 
 Review comment:
   Could you update this test to also call free(). Also would be good to add a 
test for reallocate as well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433673#comment-16433673
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-380398449
 
 
   Hi, can anybody help to merge this?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433294#comment-16433294
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on a change in pull request #1875: ARROW-2435: [Rust] 
Add memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180616961
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use libc;
+use std::mem;
+use std::cmp;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+/// Memory pool for allocating memory. It's also responsible for tracking 
memory usage.
+pub trait MemoryPool {
+/// Allocate memory.
+/// The implementation should ensures that allocated memory is aligned.
+fn allocate(, size: usize) -> Result<*mut u8>;
+
+/// Reallocate memory.
+/// If the implementation doesn't support reallocating aligned memory, it 
allocates new memory
+/// and copied old memory to it.
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8>;
+
+/// Free memory.
+fn free(, ptr: *mut u8);
+}
+
+/// Implementation of memory pool using lib api.
+#[allow(dead_code)]
+struct LibcMemoryPool;
+
+impl MemoryPool for LibcMemoryPool {
+fn allocate(, size: usize) -> Result<*mut u8> {
+unsafe {
+let mut page: *mut libc::c_void = mem::uninitialized();
+let result = libc::posix_memalign( page, ALIGNMENT, size);
+match result {
+0 => Ok(mem::transmute::<*mut libc::c_void, *mut u8>(page)),
+_ => Err(ArrowError::MemoryError(
+"Failed to allocate memory".to_string(),
+)),
+}
+}
+}
+
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8> {
+unsafe {
+let result = self.allocate(new_size)?;
+let dst = mem::transmute::<*mut u8, *mut libc::c_void>(result);
+libc::memcpy(
+dst,
+mem::transmute::<*mut u8, *const libc::c_void>(pointer),
+cmp::min(old_size, new_size),
+);
 
 Review comment:
   Thanks for the review, and I've fixed it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433135#comment-16433135
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-380273948
 
 
   As CI is not currently setup for Windows I think it’s fine to push ahead 
with this change without windows support, then reopen the windows support jira 
and I’ll address it in a separate PR.
   
   I seem to be the only person on Windows and momentum is more important at 
this stage of the rust implementation.
   
   Once we have windows CI setup I think we try to keep windows support working 
on all PRs.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432831#comment-16432831
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-380221529
 
 
   We should definitely isolate all memory alloc/free in the one file, which I 
think this PR does nicely. We just need to merge the windows changes and fix 
the oustanding issue where realloc is failing to free the old memory and I 
think we're good.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432485#comment-16432485
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-380141516
 
 
   This is similar to what we had in iron-arrow before the contribution to the 
main arrow repo but it seemed that @andygrove went a different direction 
initially.
   
   I just added windows support last night to memory.rs and friends.  Do we 
plan to isolate libc calls to this new memory pool (replacing the memory.rs 
approach)?
   
   Just wondering what the direction is as the above memory pool approach does 
not work for windows.
   
   I can add windows support for memory pool also once it's merged.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432458#comment-16432458
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

paddyhoran commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-380141516
 
 
   This is similar to what we had in iron-arrow before the contribution to the 
main arrow repo but it seemed that @andygrove went a different direction 
initially.
   
   I just added windows support last night to memory.rs and friends.  Do we 
plan to isolate libc calls to this new memory pool (replacing the memory.rs 
approach)?
   
   Just wondering what the direction is as the above memory pool approach does 
not work from windows.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432281#comment-16432281
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on a change in pull request #1875: ARROW-2435: [Rust] Add 
memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180425960
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use libc;
+use std::mem;
+use std::cmp;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+/// Memory pool for allocating memory. It's also responsible for tracking 
memory usage.
+pub trait MemoryPool {
+/// Allocate memory.
+/// The implementation should ensures that allocated memory is aligned.
+fn allocate(, size: usize) -> Result<*mut u8>;
+
+/// Reallocate memory.
+/// If the implementation doesn't support reallocating aligned memory, it 
allocates new memory
+/// and copied old memory to it.
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8>;
+
+/// Free memory.
+fn free(, ptr: *mut u8);
+}
+
+/// Implementation of memory pool using lib api.
+#[allow(dead_code)]
+struct LibcMemoryPool;
+
+impl MemoryPool for LibcMemoryPool {
+fn allocate(, size: usize) -> Result<*mut u8> {
+unsafe {
+let mut page: *mut libc::c_void = mem::uninitialized();
+let result = libc::posix_memalign( page, ALIGNMENT, size);
+match result {
+0 => Ok(mem::transmute::<*mut libc::c_void, *mut u8>(page)),
+_ => Err(ArrowError::MemoryError(
+"Failed to allocate memory".to_string(),
+)),
+}
+}
+}
+
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8> {
+unsafe {
+let result = self.allocate(new_size)?;
+let dst = mem::transmute::<*mut u8, *mut libc::c_void>(result);
+libc::memcpy(
+dst,
+mem::transmute::<*mut u8, *const libc::c_void>(pointer),
+cmp::min(old_size, new_size),
+);
 
 Review comment:
   I agree, the old buffer needs to be released


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1643#comment-1643
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

andygrove commented on issue #1875: ARROW-2435: [Rust] Add memory pool 
abstraction.
URL: https://github.com/apache/arrow/pull/1875#issuecomment-380092065
 
 
   This is looking good. Thanks @liurenjie1024 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432187#comment-16432187
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

xhochy commented on a change in pull request #1875: ARROW-2435: [Rust] Add 
memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180402305
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use libc;
+use std::mem;
+use std::cmp;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+/// Memory pool for allocating memory. It's also responsible for tracking 
memory usage.
+pub trait MemoryPool {
+/// Allocate memory.
+/// The implementation should ensures that allocated memory is aligned.
+fn allocate(, size: usize) -> Result<*mut u8>;
+
+/// Reallocate memory.
+/// If the implementation doesn't support reallocating aligned memory, it 
allocates new memory
+/// and copied old memory to it.
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8>;
+
+/// Free memory.
+fn free(, ptr: *mut u8);
+}
+
+/// Implementation of memory pool using lib api.
+#[allow(dead_code)]
+struct LibcMemoryPool;
+
+impl MemoryPool for LibcMemoryPool {
+fn allocate(, size: usize) -> Result<*mut u8> {
+unsafe {
+let mut page: *mut libc::c_void = mem::uninitialized();
+let result = libc::posix_memalign( page, ALIGNMENT, size);
+match result {
+0 => Ok(mem::transmute::<*mut libc::c_void, *mut u8>(page)),
+_ => Err(ArrowError::MemoryError(
+"Failed to allocate memory".to_string(),
+)),
+}
+}
+}
+
+fn reallocate(, old_size: usize, new_size: usize, pointer: *mut u8) 
-> Result<*mut u8> {
+unsafe {
+let result = self.allocate(new_size)?;
+let dst = mem::transmute::<*mut u8, *mut libc::c_void>(result);
+libc::memcpy(
+dst,
+mem::transmute::<*mut u8, *const libc::c_void>(pointer),
+cmp::min(old_size, new_size),
+);
 
 Review comment:
   I would have expected that we also free `pointer` here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432012#comment-16432012
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on a change in pull request #1875: ARROW-2435: [Rust] 
Add memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180362225
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,54 @@
+use libc;
+use std::sync::Arc;
+use std::mem;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+pub trait MemoryPool {
+  fn allocate_aligned(, size: usize) -> Result<*mut u8>;
 
 Review comment:
   Hi, xhochy:
   Thanks for the review and I've added an implementation for that method.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432010#comment-16432010
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 commented on a change in pull request #1875: ARROW-2435: [Rust] 
Add memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180362068
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,54 @@
+use libc;
 
 Review comment:
   Fixed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431970#comment-16431970
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

xhochy commented on a change in pull request #1875: ARROW-2435: [Rust] Add 
memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180347909
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,54 @@
+use libc;
+use std::sync::Arc;
+use std::mem;
+
+use super::error::ArrowError;
+use super::error::Result;
+
+const ALIGNMENT: usize = 64;
+
+pub trait MemoryPool {
+  fn allocate_aligned(, size: usize) -> Result<*mut u8>;
 
 Review comment:
   This should also get a `Reallocate` function as we have in C++ 
https://github.com/apache/arrow/blob/6633cc94c84f9ecc516f03175da56361cf4bd67f/cpp/src/arrow/memory_pool.h#L77
   
   When using jemalloc, this was one of the major performance improvements that 
we could avoid 50% of all reallocations.
   
   For starters, it will be ok to always allocate a new block of memory.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431971#comment-16431971
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

xhochy commented on a change in pull request #1875: ARROW-2435: [Rust] Add 
memory pool abstraction.
URL: https://github.com/apache/arrow/pull/1875#discussion_r180347430
 
 

 ##
 File path: rust/src/memory_pool.rs
 ##
 @@ -0,0 +1,54 @@
+use libc;
 
 Review comment:
   You need to add an Apache license header as we have in all other files.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Assignee: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ARROW-2435) [Rust] Add memory pool abstraction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ARROW-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431960#comment-16431960
 ] 

ASF GitHub Bot commented on ARROW-2435:
---

liurenjie1024 opened a new pull request #1875: ARROW-2435: [Rust] Add memory 
pool abstraction.
URL: https://github.com/apache/arrow/pull/1875
 
 
   Current implementation use libc for memory allocation and deallocation. This 
can't be ported cross platforms. We can add a memory pool abstraction for many 
use cases, such as:
   
   * Easy migration to other allocation api, such as the api use in rust alloc 
crate.
   * Logging.
   * Statistics updating.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> [Rust] Add memory pool abstraction.
> ---
>
> Key: ARROW-2435
> URL: https://issues.apache.org/jira/browse/ARROW-2435
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Rust
>Affects Versions: 0.9.0
>Reporter: Renjie Liu
>Priority: Major
>  Labels: pull-request-available
>
> Add memory pool abstraction as the c++ api.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)