I would be willing to implement that. I’ll probably need some advice on my
patch though, as I’m fairly new to the parquet code.
Roman
Von: Wes McKinney
Gesendet: Donnerstag, 8. November 2018 23:22
An: dev@arrow.apache.org
Betreff: Re: Support for TIMESTAMP_NANOS in parquet-cpp
I opened an
Kouhei Sutou created ARROW-3733:
---
Summary: [GLib] Add to_string() to GArrowTable and GArrowColumn
Key: ARROW-3733
URL: https://issues.apache.org/jira/browse/ARROW-3733
Project: Apache Arrow
There is one database that I'm aware of that uses sentinels _and_ supports
complex types with missing values: Kx's KDB+. This has led to some
seriously strange choices like the ASCII space character being used as the
sentinel value for strings. See
https://code.kx.com/wiki/Reference/Datatypes for
Wes McKinney created ARROW-3732:
---
Summary: [R] Add functions to write RecordBatch or Schema to
Message value, then read back
Key: ARROW-3732
URL: https://issues.apache.org/jira/browse/ARROW-3732
I opened an issue here
https://issues.apache.org/jira/browse/ARROW-3729. Patches would be
welcome
On Sat, Oct 20, 2018 at 12:55 PM Wes McKinney wrote:
>
> hi Roman,
>
> We would welcome adding such a document to the Arrow wiki
> https://cwiki.apache.org/confluence/display/ARROW. As to your other
Wes McKinney created ARROW-3729:
---
Summary: [C++] Support for writing TIMESTAMP_NANOS Parquet metadata
Key: ARROW-3729
URL: https://issues.apache.org/jira/browse/ARROW-3729
Project: Apache Arrow
Wes McKinney created ARROW-3731:
---
Summary: [R] R API for reading and writing Parquet files
Key: ARROW-3731
URL: https://issues.apache.org/jira/browse/ARROW-3731
Project: Apache Arrow
Issue
Wes McKinney created ARROW-3730:
---
Summary: [Python] Output a representation of pyarrow.Schema that
can be used to reconstruct a schema in a script
Key: ARROW-3730
URL:
Congrats!
On Thu, Nov 8, 2018 at 4:02 PM Uwe L. Korn wrote:
> Congratulations Krisztián!
>
> On Thu, Nov 8, 2018, at 9:56 PM, Philipp Moritz wrote:
> > Congrats and welcome Krisztián!
> >
> > On Thu, Nov 8, 2018 at 11:48 AM Wes McKinney
> wrote:
> >
> > > The Project Management Committee (PMC)
Welcome!
On Thu, Nov 8, 2018 at 4:01 PM Uwe L. Korn wrote:
> Welcome to all of you!
>
> On Thu, Nov 8, 2018, at 8:56 PM, Wes McKinney wrote:
> > On behalf of the Arrow PMC, I'm happy to announce that Romain
> > François, Sebastien Binet, and Yosuke Shiro have been invited to be
> > committers
hey Matt,
Thanks for giving your perspective on the mailing list.
My objective in writing about this recently
(http://wesmckinney.com/blog/bitmaps-vs-sentinel-values/, though I
need to update since the sentinel case can be done more efficiently
than what's there now) was to help dispel the
Emmett McQuinn created ARROW-3717:
-
Summary: Add GCSFSWrapper for DaskFileSystem
Key: ARROW-3717
URL: https://issues.apache.org/jira/browse/ARROW-3717
Project: Apache Arrow
Issue Type: New
Kouhei Sutou created ARROW-3720:
---
Summary: [GLib] Use "indices" instead of "indexes"
Key: ARROW-3720
URL: https://issues.apache.org/jira/browse/ARROW-3720
Project: Apache Arrow
Issue Type:
Yosuke Shiro created ARROW-3723:
---
Summary: [Plasma] [Ruby] Add Ruby bindings of Plasma
Key: ARROW-3723
URL: https://issues.apache.org/jira/browse/ARROW-3723
Project: Apache Arrow
Issue Type:
Kouhei Sutou created ARROW-3725:
---
Summary: [GLib] Add field readers to GArrowStructDataType
Key: ARROW-3725
URL: https://issues.apache.org/jira/browse/ARROW-3725
Project: Apache Arrow
Issue
Congratulations Krisztián!
On Thu, Nov 8, 2018, at 9:56 PM, Philipp Moritz wrote:
> Congrats and welcome Krisztián!
>
> On Thu, Nov 8, 2018 at 11:48 AM Wes McKinney wrote:
>
> > The Project Management Committee (PMC) for Apache Arrow has invited
> > Krisztián Szűcs to become a PMC member and
Welcome to all of you!
On Thu, Nov 8, 2018, at 8:56 PM, Wes McKinney wrote:
> On behalf of the Arrow PMC, I'm happy to announce that Romain
> François, Sebastien Binet, and Yosuke Shiro have been invited to be
> committers on the project.
>
> Welcome, and thanks for your contributions!
It's nice to have new people onboard. Welcome everyone :-)
Le 08/11/2018 à 20:56, Wes McKinney a écrit :
> On behalf of the Arrow PMC, I'm happy to announce that Romain
> François, Sebastien Binet, and Yosuke Shiro have been invited to be
> committers on the project.
>
> Welcome, and thanks
Congrats and welcome Krisztián!
On Thu, Nov 8, 2018 at 11:48 AM Wes McKinney wrote:
> The Project Management Committee (PMC) for Apache Arrow has invited
> Krisztián Szűcs to become a PMC member and we are pleased to announce
> that he has accepted.
>
> Congratulations and welcome, Krisztián!
>
Yosuke Shiro created ARROW-3724:
---
Summary: [GLib] Update gitignore
Key: ARROW-3724
URL: https://issues.apache.org/jira/browse/ARROW-3724
Project: Apache Arrow
Issue Type: Improvement
The Project Management Committee (PMC) for Apache Arrow has invited
Krisztián Szűcs to become a PMC member and we are pleased to announce
that he has accepted.
Congratulations and welcome, Krisztián!
I opened https://issues.apache.org/jira/browse/ARROW-3727 about adding
examples. I will mention to add an example for CUDA also
On Thu, Nov 8, 2018 at 2:30 PM Randy Zwitch wrote:
>
> Thanks Uwe, Wes, Pearu and Antoine. This is in the pyarrow docs, but no
> example, so I'll open up a JIRA so that
Thanks Uwe, Wes, Pearu and Antoine. This is in the pyarrow docs, but no
example, so I'll open up a JIRA so that it might be more obvious the
next person.
On 11/8/18 12:59 PM, Uwe L. Korn wrote:
Hello Randy,
you are looking for
Hello Randy,
you are looking for
https://arrow.apache.org/docs/python/generated/pyarrow.foreign_buffer.html#pyarrow.foreign_buffer
This takes an address, size and a Python object for having a reference on the
object. In your case the last one can be None. Note that this will not do a
copy and
Micah Williamson created ARROW-3728:
---
Summary: Merging Parquet Files - Pandas Meta in Schema Mismatch
Key: ARROW-3728
URL: https://issues.apache.org/jira/browse/ARROW-3728
Project: Apache Arrow
nevi_me created ARROW-3726:
--
Summary: [Rust] CSV Reader & Writer
Key: ARROW-3726
URL: https://issues.apache.org/jira/browse/ARROW-3726
Project: Apache Arrow
Issue Type: New Feature
Wes McKinney created ARROW-3727:
---
Summary: [Python] Document use of pyarrow.foreign_buffer in Sphinx
Key: ARROW-3727
URL: https://issues.apache.org/jira/browse/ARROW-3727
Project: Apache Arrow
Hi,
For host memory, you can use pyarrow.foreign_buffer, see
https://arrow.apache.org/docs/python/generated/pyarrow.foreign_buffer.html
For device memory, one can use pyarrow.cuda.foreign_buffer.
HTH,
Pearu
On Thu, Nov 8, 2018 at 7:53 PM Randy Zwitch
wrote:
> Within OmniSci (MapD), we
Yes, see pyarrow.foreign_buffer
If this isn't in the documentation, could you open a JIRA to fix that?
Thanks
Wes
On Thu, Nov 8, 2018, 11:53 AM Randy Zwitch Within OmniSci (MapD), we have the following code that takes a pointer
> and length and reads to a NumPy array before calling py_buffer:
You should be able to use pa.foreign_buffer():
https://arrow.apache.org/docs/python/generated/pyarrow.foreign_buffer.html#pyarrow.foreign_buffer
Regards
Antoine.
Le 08/11/2018 à 18:49, Randy Zwitch a écrit :
> Within OmniSci (MapD), we have the following code that takes a pointer
> and
Within OmniSci (MapD), we have the following code that takes a pointer
and length and reads to a NumPy array before calling py_buffer:
https://github.com/omnisci/pymapd/blob/master/pymapd/shm.pyx#L31-L52
Is it possible to eliminate the NumPy step and go directly do an Arrow
buffer? There is
Philipp Moritz created ARROW-3718:
-
Summary: [Gandiva] Remove spurious gtest include
Key: ARROW-3718
URL: https://issues.apache.org/jira/browse/ARROW-3718
Project: Apache Arrow
Issue Type:
Antoine Pitrou created ARROW-3722:
-
Summary: [C++] Allow specifying column types to CSV reader
Key: ARROW-3722
URL: https://issues.apache.org/jira/browse/ARROW-3722
Project: Apache Arrow
Philipp Moritz created ARROW-3721:
-
Summary: [Gandiva] [Python] Support all Gandiva literals
Key: ARROW-3721
URL: https://issues.apache.org/jira/browse/ARROW-3721
Project: Apache Arrow
Issue
34 matches
Mail list logo
