[
https://issues.apache.org/jira/browse/DRILL-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422915#comment-15422915
]
ASF GitHub Bot commented on DRILL-4834:
---------------------------------------
GitHub user daveoshinsky opened a pull request:
https://github.com/apache/drill/pull/570
DRILL-4834
decimal implementation is vulnerable to overflow errors, and extremely
complex
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/daveoshinsky/drill DRILL-4834
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/570.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #570
----
commit 9a47ca52125139d88adf39b5d894a02f870f37d9
Author: U-COMMVAULT-NJ\doshinsky <[email protected]>
Date: 2016-02-09T22:37:47Z
DRILL-4184: support variable length decimal fields in parquet
commit dec00a808c99554f008e23fd21b944b858aa9ae0
Author: daveoshinsky <[email protected]>
Date: 2016-02-09T22:56:28Z
DRILL-4184: changes to support variable length decimal fields in parquet
commit 39bedc5f460fa4ca6158038eb568436a1b0fe3fb
Author: daveoshinsky <[email protected]>
Date: 2016-02-11T15:41:45Z
remove trailing whitespace
commit a41f8d72c538776624d7e1ef79d58ab7e5961e94
Author: daveoshinsky <[email protected]>
Date: 2016-02-11T17:53:04Z
try removing trailing whitespace, again
commit 2a4b57c67bc5e9ad9863e5c1f5693fca9f42e706
Author: Dave Oshinsky <[email protected]>
Date: 2016-03-07T20:56:02Z
centralize decimalLengths access
move declaration to top
commit 68c6c6d72a6bb59362baefa38689f70a02bc0178
Author: Dave Oshinsky <[email protected]>
Date: 2016-03-07T21:52:46Z
change comment
commit 04f7879b44e388c0253b2b641d8ffc0194980cf1
Author: Dave Oshinsky <[email protected]>
Date: 2016-03-07T23:01:38Z
the infamous tab character made a reappearance
commit 838a85747f48e1ba587d2da7d3151d0b1ea295b4
Author: Dave Oshinsky <[email protected]>
Date: 2016-03-16T17:27:52Z
Merge remote-tracking branch 'refs/remotes/apache/master'
commit 4dbf150b17c4a3cc3fc622734f786f575ce87841
Author: Dave Oshinsky <[email protected]>
Date: 2016-06-09T20:28:48Z
Merge remote-tracking branch 'refs/remotes/apache/master'
commit d2372a892216269a71425175c2b2ad9b02941261
Author: Dave Oshinsky <[email protected]>
Date: 2016-08-16T15:27:12Z
Merge remote-tracking branch 'refs/remotes/apache/master'
----
> decimal implementation is vulnerable to overflow errors, and extremely complex
> ------------------------------------------------------------------------------
>
> Key: DRILL-4834
> URL: https://issues.apache.org/jira/browse/DRILL-4834
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
> Affects Versions: 1.6.0
> Environment: Drill 1.7 on any platform
> Reporter: Dave Oshinsky
> Fix For: Future
>
>
> While working on a fix for DRILL-4704, logic was added to CastIntDecimal.java
> template to handle the situation where a precision is not supplied (i.e., the
> supplied precision is zero) for an integer value that is to be casted to a
> decimal. The Drill decimal implementation uses a limited selection of fixed
> decimal precision data types (the total number of decimal digits, i.e.,
> Decimal9, 18, 28, 38) to represent decimal values. If the destination
> precision is too small to represent the input integer that is being casted,
> there is no clean way to deal with the overflow error properly.
> While using fixed decimal precisions as is being done currently can lead to
> more efficient use of memory, it often will actually lead to less efficient
> use of memory (when the fixed precision is specified significantly larger
> than is actually needed to represent the numbers), and it results in a
> tremendous mushrooming of the complexity of the code. For each fixed
> precision (and there are only a limited set of selections, 9, 18, 28, 38,
> which itself leads to memory inefficiency), there is a separate set of code
> generated from templates. For each pairwise combination of decimal or
> non-decimal numeric types, there are multiple places in the code where
> conversions must be handled, or conditions must be included to handle the
> difference in precision between the two types. A one-size-fits-all approach
> (using a variable width vector to represent any decimal precision) would
> usually be more memory-efficient (since precisions are often over-specified),
> and would greatly simplify the code.
> Also see the DRILL-4184 issue, which is related.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
