Apologies, I checked colab and it's 0.15.0 there, I confirm that In [8]: pa.table([data2], schema=sss1)
does not work due to the missing cast implementation. It would be great if someone would contribute an implementation for ARROW-6990 On Fri, Oct 25, 2019 at 9:42 AM Wes McKinney <[email protected]> wrote: > > These code examples both work for me in 0.15.0 and master. What > version of the project are you using? > > In [6]: pa.array(data1).type > Out[6]: Decimal128Type(decimal(14, 2)) > > In [7]: pa.table([data1], schema=sss1) > Out[7]: > pyarrow.Table > decimal_col: decimal(22, 2) > > On Fri, Oct 25, 2019 at 8:46 AM Дима Фомин <[email protected]> wrote: > > > > Hello, Micah! > > > > Gotcha! Seems the problem in in pyarrow.array() handling of decimal values. > > > > data1 = [decimal.Decimal('234234234234.43')] > > > > #pa.array somehow broke precision and scale? > > data2 = pa.array(data1) > > > > sss1 = pa.schema({'decimal_col': pa.decimal128(22,2)}) > > > > pa.Table.from_arrays([data1], schema=sss1) > > > > is OK > > > > bur this one > > pa.Table.from_arrays([data2], schema=sss1) > > > > raise ArrowNotImplementedError: No cast implemented from decimal(14, 2) to > > decimal(22, 2) > > > > see it here > > https://colab.research.google.com/drive/1ejOyFBVs8GPFTosuRMDyTwPvt4145rgV > > > > Should I post in to JIRA as well? Or this is the same reason posted above? > > > > > > 25.10.2019, 08:00, "Micah Kornfield" <[email protected]>: > > > > Hi Dima, > > Thanks for finding this bug would you mind filing a JIRA > > (https://issues.apache.org/jira). > > > > Note that this code works fine if you put quotes around the decimal value > > (e.g. Decimal('123.234')), but as you state it crashes python because the > > Decimal created without quotes requires more then 38 decimal places to > > represent. > > > > Thanks, > > Micah > > > > > > > > On Thu, Oct 24, 2019 at 7:53 AM Дима Фомин <[email protected]> wrote: > > > > Digging this I just found that this code breaks my Python Kernel :( > > > > import sqlalchemy as sq > > import pyarrow as pa > > > > aaa, bbb = sq.sql.sqltypes.decimal.Decimal(123.234), > > sq.sql.sqltypes.decimal.Decimal(123.234) > > ddd = [aaa, bbb] > > pa.array(ddd) > > > > So seems pyarrow.array is limited with types of array elements, doesnt'it? > > > > WBR, > > Dima Fomin > > > > 24.10.2019, 15:47, "Dima Fomin" <[email protected]>: > > > > Hello! > > First of all thank you very much for your efforts on Apache Arrow! This > > looks very useful and prominent tech to be able use it in self-BI area. > > > > I faced such issues - get Subj Error > > > > ArrowNotImplementedError: No cast implemented from decimal(2,2) to > > decimal(12,2) > > > > when trying to pyarrow.Table.from_batches array of values defined in schema > > as pyarrow.decimal128(12,2). > > > > Which values are in fact column of > > sqlAlchemy.sql.sqltypes.DECIMAL()="DECIMAL(precision=12,scale=2)" values, > > which I map in schema like this pyarrow.decimal128(12,2). > > > > What could be a reason of issue? > > Could it be that somehow declared decimal type of data is reduced to > > minimum value which met in data? :/ > > > > With best regards, > > Dima Fomin
