[jira] [Assigned] (ARROW-257) Add a typeids Vector to Union type

2016-09-21 Thread Julien Le Dem (JIRA)

 [ 
https://issues.apache.org/jira/browse/ARROW-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julien Le Dem reassigned ARROW-257:
---

Assignee: Julien Le Dem

> Add a typeids Vector to Union type
> --
>
> Key: ARROW-257
> URL: https://issues.apache.org/jira/browse/ARROW-257
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: Format
>Reporter: Julien Le Dem
>Assignee: Julien Le Dem
>
> {noformat}
> enum UnionMode:int { Sparse, Dense }
> table Union {
>   mode: UnionMode;
>   typeIds: [Int32]; // optional, describes typeid of each child.
> }
> {noformat}
> The idea is to enable providing an id different from the child offset (the 
> default)
> This enables an optimization where we use predefined ids when constructing 
> the type vector of the union but want the children to be only the actually 
> used types.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (ARROW-261) [C++] Refactor BinaryArray/StringArray classes to not inherit from ListArray

2016-09-21 Thread Wes McKinney (JIRA)

[ 
https://issues.apache.org/jira/browse/ARROW-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511360#comment-15511360
 ] 

Wes McKinney commented on ARROW-261:


The scope of this issue will also involve fixing the IPC adapter to not write 
string as list-like metadata

> [C++] Refactor BinaryArray/StringArray classes to not inherit from ListArray
> 
>
> Key: ARROW-261
> URL: https://issues.apache.org/jira/browse/ARROW-261
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: C++
>Reporter: Micah Kornfield
>Assignee: Wes McKinney
>
> also change there builders to not inherit from ListBuilder.  These should be 
> first class types.  I think the consensus is to have them inherit from 
> PrimitiveArray but I think I this might be awkward, and having them inherit 
> directly from Array might be better. [~wesmckinn] thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (ARROW-261) [C++] Refactor BinaryArray/StringArray classes to not inherit from ListArray

2016-09-21 Thread Wes McKinney (JIRA)

 [ 
https://issues.apache.org/jira/browse/ARROW-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wes McKinney reassigned ARROW-261:
--

Assignee: Wes McKinney

> [C++] Refactor BinaryArray/StringArray classes to not inherit from ListArray
> 
>
> Key: ARROW-261
> URL: https://issues.apache.org/jira/browse/ARROW-261
> Project: Apache Arrow
>  Issue Type: Bug
>  Components: C++
>Reporter: Micah Kornfield
>Assignee: Wes McKinney
>
> also change there builders to not inherit from ListBuilder.  These should be 
> first class types.  I think the consensus is to have them inherit from 
> PrimitiveArray but I think I this might be awkward, and having them inherit 
> directly from Array might be better. [~wesmckinn] thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (ARROW-299) Use absolute namespace in macros

2016-09-21 Thread Wes McKinney (JIRA)

 [ 
https://issues.apache.org/jira/browse/ARROW-299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wes McKinney resolved ARROW-299.

Resolution: Fixed

Issue resolved by pull request 142
[https://github.com/apache/arrow/pull/142]

> Use absolute namespace in macros
> 
>
> Key: ARROW-299
> URL: https://issues.apache.org/jira/browse/ARROW-299
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: C++
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> As macros can be embedded anywhere, the namespaces in there should be 
> explicitly absolute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (ARROW-301) [Format] Add some form of user field metadata to IPC schemas

2016-09-21 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-301:
--

 Summary: [Format] Add some form of user field metadata to IPC 
schemas
 Key: ARROW-301
 URL: https://issues.apache.org/jira/browse/ARROW-301
 Project: Apache Arrow
  Issue Type: New Feature
  Components: Format
Reporter: Wes McKinney


Users may wish to use user-defined types that can be represented as some Arrow 
primitive type. We should design a mechanism (similar to the key-value metadata 
in Parquet) to make this possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (ARROW-300) [Format] Add buffer compression option to IPC file format

2016-09-21 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-300:
--

 Summary: [Format] Add buffer compression option to IPC file format
 Key: ARROW-300
 URL: https://issues.apache.org/jira/browse/ARROW-300
 Project: Apache Arrow
  Issue Type: New Feature
  Components: Format
Reporter: Wes McKinney


It may be useful if data is to be sent over the wire to compress the data 
buffers themselves as their being written in the file layout.

I would propose that we keep this extremely simple with a global buffer 
compression setting in the file Footer. Probably only two compressors worth 
supporting out of the box would be zlib (higher compression ratios) and lz4 
(better performance).

What does everyone think?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (ARROW-96) C++: API documentation using Doxygen

2016-09-21 Thread Uwe L. Korn (JIRA)

[ 
https://issues.apache.org/jira/browse/ARROW-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510985#comment-15510985
 ] 

Uwe L. Korn edited comment on ARROW-96 at 9/21/16 7:55 PM:
---

Currently we're using {{//}} for code comments. As far as I know doxygen, code 
comments that should be included in the API docs should have a more "special" 
comment, e.g. {{///}} or {{/** */}} (see 
https://www.stack.nl/~dimitri/doxygen/manual/docblocks.html).

[~wesmckinn] I'm happy with any of them. You had some preference if I remember 
correctly? I would also use the same choice then for parquet-cpp.


was (Author: xhochy):
Currently we're using {{//}} for code comments. As far as I known doxygen, code 
comments that should be included in the API docs should have a more "special" 
comment, e.g. {{///}} or {{/** */}} (see 
https://www.stack.nl/~dimitri/doxygen/manual/docblocks.html).

[~wesmckinn] I'm happy with any of them. You had some preference if I remember 
correctly? I would also use the same choice then for parquet-cpp.

> C++: API documentation using Doxygen 
> -
>
> Key: ARROW-96
> URL: https://issues.apache.org/jira/browse/ARROW-96
> Project: Apache Arrow
>  Issue Type: Task
>  Components: C++
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> For the developers using Arrow via C++, we should provide an automatically 
> generated API documentation via doxygen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (ARROW-96) C++: API documentation using Doxygen

2016-09-21 Thread Uwe L. Korn (JIRA)

[ 
https://issues.apache.org/jira/browse/ARROW-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510985#comment-15510985
 ] 

Uwe L. Korn commented on ARROW-96:
--

Currently we're using {{//}} for code comments. As far as I known doxygen, code 
comments that should be included in the API docs should have a more "special" 
comment, e.g. {{///}} or {{/** */}} (see 
https://www.stack.nl/~dimitri/doxygen/manual/docblocks.html).

[~wesmckinn] I'm happy with any of them. You had some preference if I remember 
correctly? I would also use the same choice then for parquet-cpp.

> C++: API documentation using Doxygen 
> -
>
> Key: ARROW-96
> URL: https://issues.apache.org/jira/browse/ARROW-96
> Project: Apache Arrow
>  Issue Type: Task
>  Components: C++
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> For the developers using Arrow via C++, we should provide an automatically 
> generated API documentation via doxygen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (ARROW-299) Use absolute namespace in macros

2016-09-21 Thread Uwe L. Korn (JIRA)

[ 
https://issues.apache.org/jira/browse/ARROW-299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510466#comment-15510466
 ] 

Uwe L. Korn commented on ARROW-299:
---

PR: https://github.com/apache/arrow/pull/142

> Use absolute namespace in macros
> 
>
> Key: ARROW-299
> URL: https://issues.apache.org/jira/browse/ARROW-299
> Project: Apache Arrow
>  Issue Type: Improvement
>  Components: C++
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> As macros can be embedded anywhere, the namespaces in there should be 
> explicitly absolute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (ARROW-299) Use absolute namespace in macros

2016-09-21 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-299:
-

 Summary: Use absolute namespace in macros
 Key: ARROW-299
 URL: https://issues.apache.org/jira/browse/ARROW-299
 Project: Apache Arrow
  Issue Type: Improvement
  Components: C++
Reporter: Uwe L. Korn
Assignee: Uwe L. Korn


As macros can be embedded anywhere, the namespaces in there should be 
explicitly absolute.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)