[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090700#comment-17090700 ] Antoine Pitrou commented on ARROW-3329: --- Thank you [~jacek.pliszka] for doing this. The PR unexpected uncovered two issues: ARROW-8567 and ARROW-8568. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084671#comment-17084671 ] Jacek Pliszka commented on ARROW-3329: -- Thanks. I forgot to add there are 2 FIXME - one is related to inconsistent exception returned and the other to missing Decimal overflow handling. Both things should be fixed elsewhere, I believe. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084669#comment-17084669 ] Antoine Pitrou commented on ARROW-3329: --- Sorry, we've been busy with the release process lately. I'll take a look soon. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084624#comment-17084624 ] Jacek Pliszka commented on ARROW-3329: -- Any chance of getting the PR accepted? This is basically just 4 small fixes of other stuff to pass tests + Python unit tests. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076316#comment-17076316 ] Jacek Pliszka commented on ARROW-3329: -- Pull request with Python tests https://github.com/apache/arrow/pull/6846 > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075915#comment-17075915 ] Jacek Pliszka commented on ARROW-3329: -- Added some corrections to my branch - waiting for tests to finish - when they are OK I will raise pull request. https://github.com/JacekPliszka/arrow/tree/ARROW-3329 > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075802#comment-17075802 ] Jacek Pliszka commented on ARROW-3329: -- With new instructions build works great so I can work on tests. Thank you. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073177#comment-17073177 ] Jacek Pliszka commented on ARROW-3329: -- Tried. Unfortunately I am running Fedora 31 which has cgroups v2 so docker does not work without going back to v1. And podman-compose does not work as not all options are implemented... Either I will go back to debugging the original method or install Debian in virtual machine and then use docker. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072210#comment-17072210 ] Wes McKinney commented on ARROW-3329: - Right, the docker-compose setup would be a good way for you to develop locally since these tasks work 100% as we run them daily in CI. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072070#comment-17072070
]
Antoine Pitrou commented on ARROW-3329:
---
We do have a docker-compose setup. For example:
{code:bash}
$ docker-compose pull ubuntu-python
Pulling ubuntu-python ... done
$ docker-compose run --rm ubuntu-python
[...]
{code}
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072066#comment-17072066 ] Jacek Pliszka commented on ARROW-3329: -- I did start from fresh git clone. Thank you for help anyway. Good docker file would be great. What do we do with Python tests? For me the most effective would be to add them in 0.17.1, alternative is to write them blindly and see Jenkins errors or to debug the above which would take most time but I will correct instructions too and maybe prepare docker file for Fedora. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071322#comment-17071322 ] Wes McKinney commented on ARROW-3329: - You might try reproducing the issue in a Dockerfile or delete your Arrow clone and try from a fresh git clone. I'm not familiar with this error occurring unless there are CMakeCache.txt files leftover from failed build attempts. I'll give up trying to help more and will make sure the Python developer instructions are reviewed in the near future and perhaps provide a minimal Docker setup showing a from-scratch build using either the virtualenv strategy or conda strategy > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071203#comment-17071203
]
Jacek Pliszka commented on ARROW-3329:
--
Same result - I believe I need some time to go slowly step by step.
{code:java}
-- Finished cmake for pyarrow
-- Running cmake --build for pyarrow
cmake --build . --config release --
Error: could not load cache
error: command 'cmake' failed with exit status 1
{code}
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071154#comment-17071154
]
Wes McKinney commented on ARROW-3329:
-
Here are the commands I use when building the Python library
{code}
pushd $ARROW_ROOT/python
rm -rf build/
python setup.py build_ext --inplace
popd
{code}
For maximum reproducibility I've found it's best to remove {{build/}} before
rebuilding.
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17069931#comment-17069931 ] Jacek Pliszka commented on ARROW-3329: -- But at what point shall I do it ? Running it at the end just prior to setup.py changes nothing in my case > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068969#comment-17068969
]
Wes McKinney commented on ARROW-3329:
-
You need to clean temporary files out of the python/ directory with {{git clean
-fdx python}}. This should be added to the documentation
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068688#comment-17068688 ] Jacek Pliszka commented on ARROW-3329: -- OK, tried that but there are errors there too 1 it also is inconsistent with pushd arrow 2 pushd arrow/cpp/build before cmake should be without build # libbz2 is missing even though it was not missing with pip # same error at the end python setup.py build_ext --inplace -- Running cmake --build for pyarrow cmake --build . --config release -- Error: could not load cache error: command 'cmake' failed with exit status 1 > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h 10m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068090#comment-17068090
]
Wes McKinney commented on ARROW-3329:
-
It's easy and safe to install an isolated conda that does not impact your system
Install miniconda locally:
{code}
MINICONDA_URL="https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh";
wget -O miniconda.sh $MINICONDA_URL
bash miniconda.sh -b -p $HOME/local-miniconda
rm -f miniconda.sh
{code}
Then when you want to use conda in a bash/zsh terminal:
{code}
. $HOME/local-miniconda/etc/profile.d/conda.sh
{code}
That's it. I ran these commands just now in a matter of seconds. Can be used
without affecting any of your system packages. Then you should be able to
follow the conda instructions in our documentation.
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067909#comment-17067909 ] Jacek Pliszka commented on ARROW-3329: -- Not in the near future, sorry. I had some bad experience with conda and I can not afford breaking other things in the next weeks. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067808#comment-17067808 ] Wes McKinney commented on ARROW-3329: - Can you please use the conda method? The virtualenv / pip method has not been actively maintained. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067449#comment-17067449
]
Jacek Pliszka commented on ARROW-3329:
--
# I am Fedora pip user so I need to adjust some commands
# this step
[https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst#environment-setup-and-build]
ends in arrow repo directory while this step
[https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst#using-pip]
states "Now, let's create a Python virtualenv with all Python dependencies in
the same folder as the repositories and a target installation folder:" going
up to repos dir (above arrow) I end up with error on cmake - CMake Error:
The source directory "/data/no_backup/git/arrow/cpp/build" does not appear to
contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
solution is to do cd .. but then pushd hierarchy possibly breaks
# After cd .. cmake works i assume this is enough for python parquet testing
{code:java}
cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME -DCMAKE_INSTALL_LIBDIR=lib
-DARROW_FLIGHT=OFF -DARROW_GANDIVA=OFF -DARROW_ORC=OFF
-DARROW_WITH_BZ2=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON
-DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_BROTLI=ON
-DARROW_PARQUET=ON -DARROW_PYTHON=ON -DARROW_PLASMA=OFF
-DARROW_BUILD_TESTS=ON
{code}
Then make -j4 fails on
{code:java}
// In file included from
/data/no_backup/git/arrow/cpp/src/arrow/ipc/json_integration.cc:29:
/data/no_backup/git/arrow/cpp/src/arrow/ipc/json_internal.h:26:10: fatal error:
rapidjson/document.h: No such file or directory
26 | #include // IWYU pragma: export
{code}
installed rapidjson-devel rpm - then it worked
# After pushd arrow/python
{code:java}
// python setup.py build_ext --inplace
{code}
Fails with Could NOT find Arrow (missing: ARROW_FULL_SO_VERSION
ARROW_SO_VERSION) I mentioned a month ago/ ARROW_HOME is set to the dist
directory in repos (so next to arrow repo). Full error message
{noformat}
-- Build output directory: /data/no_backup/git/arrow/python/build/release
CMake Error at cmake_modules/FindArrow.cmake:359 (file):
file failed to open for reading (No such file or directory):
/data/no_backup/git/arrow/dist/include/arrow/util/config.h
Call Stack (most recent call first):
cmake_modules/FindArrowPython.cmake:46 (find_package)
CMakeLists.txt:210 (find_package)
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:146
(message):
Could NOT find Arrow (missing: ARROW_FULL_SO_VERSION ARROW_SO_VERSION)
(found version "0.0.0")
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:393
(_FPHSA_FAILURE_MESSAGE)
cmake_modules/FindArrow.cmake:412 (find_package_handle_standard_args)
cmake_modules/FindArrowPython.cmake:46 (find_package)
CMakeLists.txt:210 (find_package)
-- Configuring incomplete, errors occurred!
See also "/data/no_backup/git/arrow/python/CMakeFiles/CMakeOutput.log".
See also "/data/no_backup/git/arrow/python/CMakeFiles/CMakeError.log".
error: command 'cmake' failed with exit status 1
{noformat}
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067081#comment-17067081 ] Wes McKinney commented on ARROW-3329: - Thanks. It's our intention that these instructions work consistently if followed faithfully, so if something doesn't work for you it's important that we are able to debug it and fix the documentation. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067041#comment-17067041 ] Jacek Pliszka commented on ARROW-3329: -- I will follow them again tommorrow. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067035#comment-17067035 ] Jacek Pliszka commented on ARROW-3329: -- I can have another shot at compilation during weekend but I may fail again. If you can add something minimal just to get into 0.17 then, once it is in pip, I can work on more without need for compilation. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067015#comment-17067015 ] Wes McKinney commented on ARROW-3329: - [~jacek.pliszka] can you show us what is going wrong when you are trying to follow the source build instructions in https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst ? > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067012#comment-17067012 ] Wes McKinney commented on ARROW-3329: - I think we should have some basic "it works" tests at the Python level > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067013#comment-17067013 ] Jacek Pliszka commented on ARROW-3329: -- Actually, if you agree to it, we can have it in 0.17.0 - then I can work on tests without compiling it and add tests in the next release. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067004#comment-17067004
]
Antoine Pitrou commented on ARROW-3329:
---
{code:python}
>>> arr = pa.array([Decimal("1234"), None], type=pa.decimal128(19,10))
>>>
>>> arr
>>>
[
1234.00,
null
]
>>> arr.cast(pa.int16())
>>>
[
1234,
null
]
>>> arr.cast(pa.int8())
>>>
Traceback (most recent call last):
File "", line 1, in
arr.cast(pa.int8())
File "pyarrow/array.pxi", line 674, in pyarrow.lib.Array.cast
check_status(Cast(_context(), self.ap[0], type.sp_type,
File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
raise ArrowInvalid(message)
ArrowInvalid: Integer value out of bounds
>>> arr.cast(pa.int8(), safe=False)
>>>
[
-46,
null
]
{code}
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067002#comment-17067002 ] Jacek Pliszka commented on ARROW-3329: -- Well, maybe we can close it - tests in C++ do conver all options. Do nulls work in Python as well? Options? > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066999#comment-17066999 ] Jacek Pliszka commented on ARROW-3329: -- i wish to but very unlikely. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066990#comment-17066990
]
Antoine Pitrou commented on ARROW-3329:
---
It works here:
{code}
>>> arr = pa.array([Decimal("1234")], type=pa.decimal128(19,10))
>>>
>>>
>>> arr
>>>
>>>
[
1234.00
]
>>> arr.cast(pa.int16())
>>>
>>>
[
1234
]
{code}
[~jacek.pliszka] I assume you want to add some tests? Or is it something else?
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Assignee: Jacek Pliszka
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066950#comment-17066950 ] Wes McKinney commented on ARROW-3329: - [~jacek.pliszka] do you think you'll be able to tackle this in the next week or less? > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Assignee: Jacek Pliszka >Priority: Major > Labels: pull-request-available > Fix For: 0.17.0 > > Time Spent: 7h > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038265#comment-17038265
]
Wes McKinney commented on ARROW-3329:
-
Have you set {{$ARROW_HOME}} to the installed location of the C++ libraries?
This is documented in
https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037278#comment-17037278 ] Jacek Pliszka commented on ARROW-3329: -- OK, above was just for C++. Now I would need some help in building Python. I am getting ``` CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:146 (message): Could NOT find Arrow (missing: ARROW_INCLUDE_DIR ARROW_LIB_DIR ARROW_FULL_SO_VERSION ARROW_SO_VERSION) ``` This is strange as I've just build arrow lib.. Do I need to set all these vars? How? > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17037266#comment-17037266 ] Jacek Pliszka commented on ARROW-3329: -- Raised simple PR - all tests green but *continuous-integration/appveyor/pr* which did not start. What to do? Simple decimal -> decimal and decimal -> int64 were implemented. The one that is missing is some kind of raw cast that could be used instead of scaling... > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035695#comment-17035695 ] Jacek Pliszka commented on ARROW-3329: -- Info from Wes: there is information about scale, example is: [https://github.com/apache/arrow/blob/220c43789fc0e2b6b623c30a18e5464ffa9f1e82/cpp/src/arrow/compute/kernels/cast.cc#L521] Asked about rescaling anyway as it still may be of some use > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035562#comment-17035562
]
Jacek Pliszka commented on ARROW-3329:
--
The key question is whether CastFunctor in cast.cc has access to scale
of the decimal? If yes how?
If not - these are the options I've came up with:
Let's assume Decimal128Type value is n
Then I expect that base call
.cast('int64') will return overflow for n beyond int64 values, value otherwise
Option 1:
.cast('int64', decimal_scale=s) would calculate n/10**s and return
overflow if it is beyond int64, value otherwise
Option 2:
.cast('int64', bytes_group=0) would return n & 0x
.cast('int64', bytes_group=1) would return (n >> 64) & 0x
.cast('int64') would have default value bytes_group=0
Option 3:
cast has no CastOptions but we add multiply compute kernel and have
something like this instead:
.compute('multiply', 10**-s).cast('int64')
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Priority: Major
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034951#comment-17034951 ] Jacek Pliszka commented on ARROW-3329: -- OK, I've resolved the issue but now arrived at probably the clue of the issues. Looks like at lowest level casting kernel knows nothing about scale, am I reading it correctly? So either we need to pass scale there or apply it somewhere above. Is there any suggestion for that? > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034846#comment-17034846 ] Jacek Pliszka commented on ARROW-3329: -- I've started work but got stuck - posted question to dev arrow list. casting from Decimal128 to Int64 otherwise seems simple enough. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034527#comment-17034527 ] Jacek Pliszka commented on ARROW-3329: -- Is there a reason why Decimal128 is not declared as numeric type in type_fwd.h ? Naively enabling this + couple config lines should make it work in simple cases. > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030117#comment-17030117 ] Wes McKinney commented on ARROW-3329: - You're welcome to contribute this, yes > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028473#comment-17028473 ] Jacek Pliszka commented on ARROW-3329: -- Would it be possible to implement at least easy corner cases? decimal(N, 0) to int decimal(P, S) to decimal(P-K, S-K) where K <= S with option of error if fractions are lost > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818085#comment-16818085 ] AdrianC commented on ARROW-3329: Similar issue occurs when casting from decimal(2,0) to decimal(10,2), from decimal(2,0) to string, from decimal(2,0) to int64, or from decimal(2,0) to float64. Precision should not be an issue in any of these cases. Occurs on python 2.7/ pyarrow 0.13 > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630093#comment-16630093 ] Wes McKinney commented on ARROW-3329: - Well, we _could_ cast, and truncate values as we do with other downcasts > [Python] Error casting decimal(38, 4) to int64 > -- > > Key: ARROW-3329 > URL: https://issues.apache.org/jira/browse/ARROW-3329 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Environment: Python version : 3.6.5 > Pyarrow version : 0.10.0 >Reporter: Kavita Sheth >Priority: Major > > Git issue LInk : https://github.com/apache/arrow/issues/2627 > I want to cast pyarrow table column from decimal(38,4) to int64. > col.cast(pa.int64()) > Error: > File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast > File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status > pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, > 4) to int64 > Python version : 3.6.5 > Pyarrow version : 0.10.0 > is it not implemented yet or I am not using it correctly? If not implemented > yet, then any work around to cast columns? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-3329) [Python] Error casting decimal(38, 4) to int64
[
https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629576#comment-16629576
]
Diego Argueta commented on ARROW-3329:
--
This isn't a bug, it's a mathematical issue. Let me explain:
To store a number with X digits of precision (in base 10) you need a number
with at least {{log2(X)}} bits in it.
So 38 digits of precision in base 10 requires {{log2(38)}} bits to represent in
binary. {{log2(38)}} is a little under 127, meaning you need your storage to be
at least 127 bits long to store a {{decimal(38, 4)}} losslessly. The error
message isn't exactly the most helpful, but it _is_ correct. There's no way to
cast a decimal of that size to a 64-bit integer.
> [Python] Error casting decimal(38, 4) to int64
> --
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
>Reporter: Kavita Sheth
>Priority: Major
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38,
> 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented
> yet, then any work around to cast columns?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
