[jira] [Commented] (ARROW-2153) [C++] Decimal conversion not working for exponential notation
[
https://issues.apache.org/jira/browse/ARROW-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372198#comment-16372198
]
ASF GitHub Bot commented on ARROW-2153:
---
wesm commented on issue #1618: ARROW-2153/ARROW-2160: [C++/Python] Fix decimal
precision inference
URL: https://github.com/apache/arrow/pull/1618#issuecomment-367511884
needs rebase
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [C++] Decimal conversion not working for exponential notation
> -
>
> Key: ARROW-2153
> URL: https://issues.apache.org/jira/browse/ARROW-2153
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Antony Mayi
>Assignee: Phillip Cloud
>Priority: Major
> Labels: pull-request-available
>
> {code:java}
> import pyarrow as pa
> import pandas as pd
> import decimal
> pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'),
> decimal.Decimal('2E+1')]}))
> {code}
>
> {code:java}
> Traceback (most recent call last):
> File "", line 1, in
> File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 350, in dataframe_to_arrays
> convert_types)]
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 349, in
> for c, t in zip(columns_to_convert,
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 345, in convert_column
> return pa.array(col, from_pandas=True, type=ty)
> File "pyarrow/array.pxi", line 170, in pyarrow.lib.array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
> File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
> File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270)
> pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found
> 'E' instead.
> {code}
> In manual cases clearly we can write {{decimal.Decimal('20')}} instead of
> {{decimal.Decimal('2E+1')}} but during arithmetical operations inside an
> application the exponential notation can be produced out of control (it is
> actually the _normalized_ form of the decimal number) plus for some values
> the exponential notation is the only form expressing the significance so this
> should be accepted.
> The [documentation|https://docs.python.org/3/library/decimal.html] suggests
> using following transformation but that's only possible when the significance
> information doesn't need to be kept:
> {code:java}
> def remove_exponent(d):
> return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2153) [C++] Decimal conversion not working for exponential notation
[
https://issues.apache.org/jira/browse/ARROW-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369546#comment-16369546
]
ASF GitHub Bot commented on ARROW-2153:
---
cpcloud commented on issue #1618: ARROW-2153/ARROW-2160: [C++/Python] Fix
decimal precision inference
URL: https://github.com/apache/arrow/pull/1618#issuecomment-366808695
In gcc 4.8 that is. 4.9 may be implemented, but I don't think it supports
named capture groups.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [C++] Decimal conversion not working for exponential notation
> -
>
> Key: ARROW-2153
> URL: https://issues.apache.org/jira/browse/ARROW-2153
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Antony Mayi
>Assignee: Phillip Cloud
>Priority: Major
> Labels: pull-request-available
>
> {code:java}
> import pyarrow as pa
> import pandas as pd
> import decimal
> pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'),
> decimal.Decimal('2E+1')]}))
> {code}
>
> {code:java}
> Traceback (most recent call last):
> File "", line 1, in
> File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 350, in dataframe_to_arrays
> convert_types)]
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 349, in
> for c, t in zip(columns_to_convert,
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 345, in convert_column
> return pa.array(col, from_pandas=True, type=ty)
> File "pyarrow/array.pxi", line 170, in pyarrow.lib.array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
> File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
> File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270)
> pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found
> 'E' instead.
> {code}
> In manual cases clearly we can write {{decimal.Decimal('20')}} instead of
> {{decimal.Decimal('2E+1')}} but during arithmetical operations inside an
> application the exponential notation can be produced out of control (it is
> actually the _normalized_ form of the decimal number) plus for some values
> the exponential notation is the only form expressing the significance so this
> should be accepted.
> The [documentation|https://docs.python.org/3/library/decimal.html] suggests
> using following transformation but that's only possible when the significance
> information doesn't need to be kept:
> {code:java}
> def remove_exponent(d):
> return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2153) [C++] Decimal conversion not working for exponential notation
[
https://issues.apache.org/jira/browse/ARROW-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369542#comment-16369542
]
ASF GitHub Bot commented on ARROW-2153:
---
cpcloud commented on issue #1618: ARROW-2153/ARROW-2160: [C++/Python] Fix
decimal precision inference
URL: https://github.com/apache/arrow/pull/1618#issuecomment-366808528
Yep, I believe `std::regex_match` is implemented as `return false;` :(
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [C++] Decimal conversion not working for exponential notation
> -
>
> Key: ARROW-2153
> URL: https://issues.apache.org/jira/browse/ARROW-2153
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Antony Mayi
>Assignee: Phillip Cloud
>Priority: Major
> Labels: pull-request-available
>
> {code:java}
> import pyarrow as pa
> import pandas as pd
> import decimal
> pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'),
> decimal.Decimal('2E+1')]}))
> {code}
>
> {code:java}
> Traceback (most recent call last):
> File "", line 1, in
> File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 350, in dataframe_to_arrays
> convert_types)]
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 349, in
> for c, t in zip(columns_to_convert,
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 345, in convert_column
> return pa.array(col, from_pandas=True, type=ty)
> File "pyarrow/array.pxi", line 170, in pyarrow.lib.array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
> File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
> File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270)
> pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found
> 'E' instead.
> {code}
> In manual cases clearly we can write {{decimal.Decimal('20')}} instead of
> {{decimal.Decimal('2E+1')}} but during arithmetical operations inside an
> application the exponential notation can be produced out of control (it is
> actually the _normalized_ form of the decimal number) plus for some values
> the exponential notation is the only form expressing the significance so this
> should be accepted.
> The [documentation|https://docs.python.org/3/library/decimal.html] suggests
> using following transformation but that's only possible when the significance
> information doesn't need to be kept:
> {code:java}
> def remove_exponent(d):
> return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2153) [C++] Decimal conversion not working for exponential notation
[
https://issues.apache.org/jira/browse/ARROW-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369540#comment-16369540
]
ASF GitHub Bot commented on ARROW-2153:
---
wesm commented on issue #1618: ARROW-2153/ARROW-2160: [C++/Python] Fix decimal
precision inference
URL: https://github.com/apache/arrow/pull/1618#issuecomment-366808204
`std::regex` is totally broken in gcc 4.8.x (what we're using for conda/pip
releases) AFAIK so using `` isn't even an option right now. When we get
past gcc 4.8 it might be nice to use the STL regexen
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [C++] Decimal conversion not working for exponential notation
> -
>
> Key: ARROW-2153
> URL: https://issues.apache.org/jira/browse/ARROW-2153
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Antony Mayi
>Assignee: Phillip Cloud
>Priority: Major
> Labels: pull-request-available
>
> {code:java}
> import pyarrow as pa
> import pandas as pd
> import decimal
> pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'),
> decimal.Decimal('2E+1')]}))
> {code}
>
> {code:java}
> Traceback (most recent call last):
> File "", line 1, in
> File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 350, in dataframe_to_arrays
> convert_types)]
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 349, in
> for c, t in zip(columns_to_convert,
> File
> "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py",
> line 345, in convert_column
> return pa.array(col, from_pandas=True, type=ty)
> File "pyarrow/array.pxi", line 170, in pyarrow.lib.array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
> File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
> File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270)
> pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found
> 'E' instead.
> {code}
> In manual cases clearly we can write {{decimal.Decimal('20')}} instead of
> {{decimal.Decimal('2E+1')}} but during arithmetical operations inside an
> application the exponential notation can be produced out of control (it is
> actually the _normalized_ form of the decimal number) plus for some values
> the exponential notation is the only form expressing the significance so this
> should be accepted.
> The [documentation|https://docs.python.org/3/library/decimal.html] suggests
> using following transformation but that's only possible when the significance
> information doesn't need to be kept:
> {code:java}
> def remove_exponent(d):
> return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
