We simply don't have conversions nor type inference implemented for inner elements with dimension greater than 1. You're welcome to propose this as a new feature / enhancement by opening a Jira issue.
On Thu, Feb 11, 2021 at 10:53 AM Bhavitvya Malik <[email protected]> wrote: > > Sure, here it is: > > data = np.zeros((10,8), dtype=np.uint8) > out = pa.array(list(data)) > out.type # ListType(list<item: uint8>) > > data = np.zeros((3,4,6), dtype=np.uint8) > out = pa.array(list(data)) # Throws error ArrowInvalid: Can only convert > 1-dimensional array values > > Even though it's working on 2D numpy arrays perfectly, it doesn't work on > N-Dimensional numpy arrays (where N > 2). Why is it so? > > > On Thu, 11 Feb 2021 at 21:18, Wes McKinney <[email protected]> wrote: >> >> Can you provide more detail about what you are trying? You've showed >> some exception here but haven't showed the exact code that results in >> those exceptions >> >> On Thu, Feb 11, 2021 at 4:34 AM Bhavitvya Malik >> <[email protected]> wrote: >> > >> > Hi, >> > It's a follow up question for #9462. Rewriting the issue here: >> > >> >> I came to know that pyarrow has this limitation of not storing >> >> N-dimensional array. After looking into this issue, I decided to >> >> represent a N-dimensional array as a list of arrays i.e. >> >> data = np.zeros((5, 3), dtype=np.uint8) >> >> data = list(data) >> >> inorder to preserve the dtype but when it comes to typecasting and >> >> writing it into array (from list) pyarrow.array(data, type=type) it gives >> >> the following error: >> >> pyarrow.lib.ArrowInvalid: Could not convert [0 0 0] with type >> >> numpy.ndarray: tried to convert to int >> >> Is there any way to avoid this issue? I just want to preserve the dtype >> >> from numpy array before converting it to list so that while writing it to >> >> pyarrow array format I can recognise its dtype and subsequently write it >> >> in that numpy dtype format. >> > >> > >> > I tried it with a 3D numpy array and it gave me this error even though >> > it's working fine with 2D numpy arrays. Can you please look into this? >> > >> > pyarrow.lib.ArrowInvalid: Can only convert 1-dimensional array values >> > >> > >> > My current pyarrow version is 2.0.0 and i tried it with pyarrow==3.0.0 too >> > >> > >> > >> > Thanks, >> > Bhavitvya
