[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-12-14 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721208#comment-16721208
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

As part of my PR, I'm adding more benchmarks to gandiva/benchmarks.cc - this'll 
exercise both the arrow-decimal code and gandiva-decimal code.

 

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-12-13 Thread Wes McKinney (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720561#comment-16720561
 ] 

Wes McKinney commented on ARROW-3701:
-

Will these benchmarks be in the Arrow codebase somewhere? There may be 
variability in different hardware, CPUs (e.g. AMD vs Intel)

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-12-11 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717008#comment-16717008
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

After running more benchmarks, I found the following for (divide a/b)
 * The Decimal128 implementation has pretty consistent performance (between 300 
to 400ms, for 10M iterations) whereas the i128 division varies widely (between 
100ms to 2000ms) based on the value of b i.e the divisor.
 * when a and b are both large random numbers, i128 divide (both clang and 
llvm) is twice as fast as  the Decimal128 implementation.
 * when b is a small number (say, 10 or 100), Decimal128 implementation is 
about 7 times faster. I think the reason is that the Decimal128 implementation 
has short-cuts for small integers.

For decimal operations, there are several cases where we need to do small 
divisions (to adjust scale). So, I'm using the Decimal128 implementation for 
divides.

 

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-09 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681704#comment-16681704
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

@wesm - I tried the same cmd on my windows 10 home desktop using llc from 
llvm-4.1. It worked fine.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-07 Thread Wes McKinney (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678799#comment-16678799
 ] 

Wes McKinney commented on ARROW-3701:
-

Did you try on Windows? If not I can give it a try

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-06 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676963#comment-16676963
 ] 

Antoine Pitrou commented on ARROW-3701:
---

> Is this documented somewhere?

I don't know. That was the result of testing some years ago. That said, it 
seems better supported nowadays. If you take the following IR:
{code}
define i128 @increment(i128 %a) {
%result = add i128 %a, 1
ret i128 %result
}

define i128 @square(i128 %a) {
%result = mul i128 %a, %a
ret i128 %result
}

define i128 @divide(i128 %a, i128 %b) {
%result = udiv i128 %a, %b
ret i128 %result
}
{code}

Then compiling it seems to work fine for several targets I tried it with:
{code}
llc-6.0 -O3 -march=x86-64 testfile.ir
{code}


> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676112#comment-16676112
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

> Also note that Boost has a 128-bit integer type, though I don't know how fast 
> it is:

Repeated my earlier test with boost int128.

typedef boost::multiprecision::number>
int128_t;

||Operation||Time with 2 64-bit Integers||Time with boost int128||Time with 
int128||
|Add|31|57|28|
|Subtract|30|77|27|
|Multiply|63|60|28|
|Divide|409|181|170|

Boost implementation is better than the util/Decimal128 impl for divide, but 
worser for add/subtract. In all cases, int128 performs better.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Jacques Nadeau (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675804#comment-16675804
 ] 

Jacques Nadeau commented on ARROW-3701:
---

Thanks for the explanations/pointers Antoine.

This suggest to me that we need to look more closely at when we use C++ as a 
basis for IR. In cases like decimal operations it seems like it might be better 
to write operations directly in LLVM ir since it seems like most of the 
operations are primitive instead of using front-end translation. Thoughts?
{quote}only the most common ones (8, 16, 32, 64) will be generally available
{quote}
Is this documented somewhere?

We may have to pick an optimal path for the common linux/modern cpus path and 
then a lowest common denominator fallback :(

 

 

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675753#comment-16675753
 ] 

Antoine Pitrou commented on ARROW-3701:
---

> From an IR perspective, we probably want to map down to LLVM's int128 
> operations since LLVM has that support within its IR

It's not guaranteed to implement it on all backends, though. Technically, you 
can define ints of any size in LLVM IR (including "int73" or "int31415" if you 
want), but only the most common ones (8, 16, 32, 64) will be generally 
available. So we would have to check whether backend support for int128 is 
available on desired platforms.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675746#comment-16675746
 ] 

Antoine Pitrou commented on ARROW-3701:
---

LLVM IR is generally not cross-platform, as it will encode platform 
specificities such as type widths, ABI, etc.
 (see LLVM FAQ at 
[https://llvm.org/docs/FAQ.html#can-i-compile-c-or-c-code-to-platform-independent-llvm-bitcode])

Perhaps by being extra-careful it's possible to have IR generated for one 
platform compile and interface correctly on another platform, but it sounds a 
bit fragile.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Jacques Nadeau (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675728#comment-16675728
 ] 

Jacques Nadeau commented on ARROW-3701:
---

{quote}However, we'll need to keep a standards-compliant fallback path with two 
int64 for compilers which don't implement a 128-bit integer.
{quote}
I'm confused by this comment. Aren't these functions going to be compiled into 
LLVM IR at build time such that the target platform needs to be supported by 
LLVM (not the compiling platform). In that case, wouldn't we be able to stop 
worrying about compilers on different platforms for these operations? We could 
just generate the IR on one platform, right? (Assuming that we can use pure 
IR--which I believe is the most optimal pattern).

>From an IR perspective, we probably want to map down to LLVM's int128 
>operations since LLVM has that support within its IR and it allows future 
>optimizations to be clean (and LLVM to target/compile as appropriate), right?

It seems like we should have a very clear delineation in Gandiva between code 
that is compiled to IR versus code that is compiled for execution (I think 
Impala uses -ir.cc to identify the former).

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675684#comment-16675684
 ] 

Antoine Pitrou commented on ARROW-3701:
---

Also note that Boost has a 128-bit integer type, though I don't know how fast 
it is:
https://www.boost.org/doc/libs/1_68_0/libs/multiprecision/doc/html/boost_multiprecision/ref/cpp_int_ref.html

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675665#comment-16675665
 ] 

Antoine Pitrou commented on ARROW-3701:
---

That sounds like a nice improvement indeed. However, we'll need to keep a 
standards-compliant fallback path with two int64 for compilers which don't 
implement a 128-bit integer.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675643#comment-16675643
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

I ran a simple test on my desktop (macbook pro) to iterate over 10M entries of 
two input arrays, and perform an arithmetic operation (in a single thread). 
Compared the perf of arrow/util/Decimal128 with another implementation that 
does the same operations but with __int128 (instead of two int64).

The results were (in ms) :
||Operation||Time with 2 64-bit Integers||Time with int128||
|Add|31|28|
|Subtract|30|27|
|Multiply|63|28|
|Divide|409|170|

So, add/subtract are equally efficient. but, multiply and divide perform much 
better with gcc's __int128.

 

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675008#comment-16675008
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

> CPUs generally don't have 128 bit integer support. So any code written using 
> a "int128" - which is non-standard - will get > translated into 64-bit 
> instructions at the CPU level.

I'm assuming the compilers are more efficient at this, but I may be wrong.

[https://github.com/rust-lang/rfcs/blob/master/text/1504-int128.md#alternatives]

With my initial testing for adding decimals -  the perf with adding i128 is 
very good (almost same as adding three longs). I haven't tried doing the same 
with the current Decimal128 - let me give it a try.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Antoine Pitrou (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674947#comment-16674947
 ] 

Antoine Pitrou commented on ARROW-3701:
---

> This class is built on a pair of int64s - using a int128 will have better 
> perf. 

Can you explain why it would? CPUs generally don't have 128 bit integer 
support. So any code written using a "int128" - which is non-standard - will 
get translated into 64-bit instructions at the CPU level.

 

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674918#comment-16674918
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

ok, I'll make sure that this code works on windows too.

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-05 Thread Wes McKinney (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674831#comment-16674831
 ] 

Wes McKinney commented on ARROW-3701:
-

The Impala code is not cross platform. We need to make sure that Gandiva will 
also work on Windows. It may be that decimals work slower on Windows as a 
result if we need to maintain multiple implementations of things.  

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARROW-3701) [Gandiva] Add support for decimal operations

2018-11-04 Thread Pindikura Ravindra (JIRA)


[ 
https://issues.apache.org/jira/browse/ARROW-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674787#comment-16674787
 ] 

Pindikura Ravindra commented on ARROW-3701:
---

@wesm @cpcloud [~pitrou] @jacques-n [~praveenbingo]

 

I've looked at the re-using the code in arrow/util/decimal.h for implementing 
decimal operations in gandiva. However, it has the following issues :
 # This class is built on a pair of int64s - using a int128 will have better 
perf. 
 # The functions do not make an attempt to avoid overflow. 
 ** sql implementations have rules for each operation that adjust the scale 
when the result precision exceeds 38. 
[https://docs.microsoft.com/en-us/sql/t-sql/data-types/precision-scale-and-length-transact-sql?view=sql-server-2017]
 # The functions do not track overflow
 ** It will be nice if there is a warning/error generated when there is 
overflow.

 

I plan to use adapt the code from Apache Impala for this instead. Thoughts ?

> [Gandiva] Add support for decimal operations
> 
>
> Key: ARROW-3701
> URL: https://issues.apache.org/jira/browse/ARROW-3701
> Project: Apache Arrow
>  Issue Type: Task
>  Components: Gandiva
>Reporter: Pindikura Ravindra
>Assignee: Pindikura Ravindra
>Priority: Major
>
> To begin with, will add support for 128-bit decimals. There are two parts :
>  # llvm_generator needs to understand decimal types (value, precision, scale)
>  # code decimal operations : add/subtract/multiply/divide/mod/..
>  ** This will be c++ code that can be pre-compiled to emit IR code



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)