@James Duong
You are absolutely right, I realized this and confirmed whether this
would be possible with Jacques to double-check.
It would amount to what I might call "dollar-store Substrait." It's not
elegant or a good solution, but definitely presents a good duct-tape hack
and is a crafty idea.
Couldn't agree more with alamb's proposal!
I started participating in the arrow rust community last year, and at
first, I just contributed code. Later, in order to have a deeper
understanding of the whole codebase and to help new people in the
community, I started to participate in code reviews, s
Hi all,
We also meet this issue:
"Failed to read artifact descriptor for
org.apache.arrow:flight-core:jar:7.0.0: Could not find artifact
org.apache.arrow:arrow-flight:pom:7.0.0"
and we work around this issue by adding the file to the local maven repo.
>From this ticket: https://issues.apache.or
Gavin, thanks for sharing. I'm not so sure you'll find an alternative to
Substrait, at least one that isn't even more nascent or one that's very tied to
a particular language, so perhaps it might be better to get involved in
Substrait and see if it suits your needs? Convincing a team to try some
James, I agree that you could use JSON but that feels a bit hacky (mis-use
of the paradigm). Instead, I'd really like to do something like David is
suggesting: support Substrait as an alternative to a SQL string.
Something like this:
https://github.com/jacques-n/arrow/commit/e22674fa882e77c2889cf95
I've come across several different environments where Arrow either fails to
configure with CMake or fails to link libraries. Some recent examples I've
come across:
* (Just fixed [1]) Windows, RTools4 (MSYS2), Debug, dynamic libraries
* (Still broken) same as above, but static libraries
* Window
In the same way that you could write an ODBC driver that takes in text
that's not SQL, you could write a Flight SQL server that takes in text
that's JSON.
Flight SQL doesn't parse the query, so you could create commands that are
just JSON text.
Is that the only bit you need, Gavin?
On Thu, Mar 3,
I am enthusiastic about Substrait and have followed it's progress eagerly =D
When I presented it as a tentative option, there were reservations because
of the project/spec being young and the functionality still being
fleshed out.
I think if I were having this conversation in say, 8-16 months, it
You probably want Substrait: https://substrait.io/
Which is being worked on by several people, including Arrow community members.
It might be interesting to generalize Flight SQL to include support for
Substrait. I'm curious what your application, if you're able to share more.
-David
On Thu, M
Hiya,
I am drafting a proposal for a way to enable services to express data
compute operations to each other.
However I think it'll be difficult to get buy-in if the only representation
for queries is as SQL strings.
Is there any kind of lower-level API that can be used to express operations?
I
Hi,
I would like to remind everyone in the Rust community that we not only
welcome, but encourage community members to help review each other's PRs in
addition to contributing code. It is a great way to help the community, get
more familiar with Rust, and the Arrow, Parquet, DataFusion and Ballist
RAPIDS/libcudf would definitely support this.
For what it's worth, libcudf's fixed_point decimal type implementation is
standalone and could easily be extracted/reused:
https://github.com/rapidsai/cudf/blob/branch-22.04/cpp/include/cudf/fixed_point/fixed_point.hpp
We even had a recent blog that d
I wanted to bring attention to an issue[1] that may be interesting to the
broader community. Namely how do you view Ballista (a library or a system).
There was some great discussion on a implementation ticket that I didn't
want to get lost on that PR
[1] https://github.com/apache/arrow-datafusion/
Libcudf / cuDF have supported 32-bit and 64-bit decimals for a few releases
now (as well as 128-bit decimals in the past couple of releases) and
they've generally been received positively from the community. Being able
to roundtrip them through Arrow would definitely be nice as well!
On Thu, Mar 3
I think this makes sense to add these. Typically when adding new types,
we've waited on the official vote until there are two reference
implementations demonstrating compatibility.
On Thu, Mar 3, 2022 at 6:55 AM Antoine Pitrou wrote:
>
> Hello,
>
> Currently, the Arrow format specification res
Hello,
Currently, the Arrow format specification restricts the bitwidth of
decimal numbers to either 128 or 256 bits.
However, there is interest in allowing other bitwidths, at least 32 and
64 bits for this proposal. A 64-bit (respectively 32-bit) decimal
datatype would allow for precision
16 matches
Mail list logo
