[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-3191:
--
Labels: pull-request-available (was: )
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
>Reporter: Jacques Nadeau
>Assignee: Siddharth Teotia
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0
>
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {code}
> public abstract class Memory {
> protected final int length;
> protected final long address;
> protected abstract void release();
> }
> {code}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-3191:
Fix Version/s: (was: 0.13.0)
0.14.0
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
>Reporter: Jacques Nadeau
>Assignee: Siddharth Teotia
>Priority: Major
> Fix For: 0.14.0
>
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {code}
> public abstract class Memory {
> protected final int length;
> protected final long address;
> protected abstract void release();
> }
> {code}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-3191:
Fix Version/s: 0.13.0
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
>Reporter: Jacques Nadeau
>Priority: Major
> Fix For: 0.13.0
>
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {code}
> public abstract class Memory {
> protected final int length;
> protected final long address;
> protected abstract void release();
> }
> {code}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-3191:
Component/s: Java
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
>Reporter: Jacques Nadeau
>Priority: Major
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {code}
> public abstract class Memory {
> protected final int length;
> protected final long address;
> protected abstract void release();
> }
> {code}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated ARROW-3191:
--
Description:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{code}
public abstract class Memory {
protected final int length;
protected final long address;
protected abstract void release();
}
{code}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
was:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{ protected abstract void release();}}
{{}}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
>Reporter: Jacques Nadeau
>Priority: Major
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {code}
> public abstract class Memory {
> protected final int length;
> protected final long address;
> protected abstract void release();
> }
> {code}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated ARROW-3191:
--
Description:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{ protected abstract void release();}}
{{}}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
was:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{ protected abstract void release();}}
{{}}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
>Reporter: Jacques Nadeau
>Priority: Major
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {{public abstract class Memory {}}
> {{ protected final int length;}}
> {{ protected final long address;}}
> {{ protected abstract void release();}}
> {{}}}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated ARROW-3191:
--
Description:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{protected abstract void release();}}
{{}}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
was:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{ protected abstract void release(); }}
{{ }}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
>Reporter: Jacques Nadeau
>Priority: Major
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {{public abstract class Memory {}}
> {{ protected final int length;}}
> {{ protected final long address;}}
> {{protected abstract void release();}}
> {{}}}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated ARROW-3191:
--
Description:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{ protected abstract void release();}}
{{}}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
was:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{protected abstract void release();}}
{{}}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
>Reporter: Jacques Nadeau
>Priority: Major
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {{public abstract class Memory {}}
> {{ protected final int length;}}
> {{ protected final long address;}}
> {{ protected abstract void release();}}
> {{}}}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-3191) [Java] Add support for ArrowBuf to point to arbitrary memory.
[
https://issues.apache.org/jira/browse/ARROW-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated ARROW-3191:
--
Description:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
{{public abstract class Memory {}}
{{ protected final int length;}}
{{ protected final long address;}}
{{ protected abstract void release(); }}
{{ }}}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
was:
Right now ArrowBuf can only point to memory managed by an Arrow Allocator. This
is because in many cases we want to be able to support hierarchical accounting
of memory and the ability to transfer memory ownership between separate
allocators within the same hierarchy.
At the same time, there are definitely times where someone might want to map
some amount of arbitrary off-heap memory. In these situations they should still
be able to use ArrowBuf.
I propose we have a new ArrowBuf constructor that takes an input that
subclasses an interface similar to:
public abstract class Memory {
protected final int length;
protected final long address;
protected abstract void release();
}
We then make it so all the memory transfer semantics and accounting behavior
are noops for this type of memory. The target of this work will be to make sure
that all the fast paths continue to be efficient but some of the other paths
like transfer can include a conditional (either directly or through alternative
implementations of things like ledger).
> [Java] Add support for ArrowBuf to point to arbitrary memory.
> -
>
> Key: ARROW-3191
> URL: https://issues.apache.org/jira/browse/ARROW-3191
> Project: Apache Arrow
> Issue Type: New Feature
>Reporter: Jacques Nadeau
>Priority: Major
>
> Right now ArrowBuf can only point to memory managed by an Arrow Allocator.
> This is because in many cases we want to be able to support hierarchical
> accounting of memory and the ability to transfer memory ownership between
> separate allocators within the same hierarchy.
> At the same time, there are definitely times where someone might want to map
> some amount of arbitrary off-heap memory. In these situations they should
> still be able to use ArrowBuf.
> I propose we have a new ArrowBuf constructor that takes an input that
> subclasses an interface similar to:
> {{public abstract class Memory {}}
> {{ protected final int length;}}
> {{ protected final long address;}}
> {{ protected abstract void release(); }}
> {{ }}}
> We then make it so all the memory transfer semantics and accounting behavior
> are noops for this type of memory. The target of this work will be to make
> sure that all the fast paths continue to be efficient but some of the other
> paths like transfer can include a conditional (either directly or through
> alternative implementations of things like ledger).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
