Great! If you could provide a smidgen of guidance about where to start making this change, I would be happy to give it a shot.
Thanks, -Dan Nugent On Mar 7, 2020, 09:18 -0500, Wes McKinney <wesmck...@gmail.com>, wrote: > hi Dan, > > Yes, we should support constructing StringArray directly from > memoryview as we do with bytes and unicode -- you're the first person > to ask about this so far. I opened > https://issues.apache.org/jira/browse/ARROW-8026. This should not be a > huge amount of work so would be a good first contribution to the > project > > Thanks > > Wes > > On Fri, Mar 6, 2020 at 8:29 PM Nugent, Daniel <daniel.nug...@mlp.com> wrote: > > > > Hi, > > > > > > > > I have a short program which I’m wondering about the sensibility of. Could > > anyone let me know if this is reasonable or not: > > > > > > > > > > > import pyarrow as pa, third_party_library > > > > > > > memory_views = third_party_library.get_strings() > > > > > > > memory_views > > > > [<memory at 0x7f1745cc0870>, <memory at 0x7f1745cc0940>, <memory at > > 0x7f1745cc0a10>, <memory at 0x7f1745cc0ae0>] > > > > > > > pa.array(memory_views,pa.string()) > > > > Traceback (most recent call last): > > > > File "<stdin>", line 1, in <module> > > > > File "pyarrow/array.pxi", line 269, in pyarrow.lib.array > > > > File "pyarrow/array.pxi", line 38, in pyarrow.lib._sequence_to_array > > > > File "pyarrow/error.pxi", line 107, in pyarrow.lib.check_status > > > > pyarrow.lib.ArrowTypeError: Expected a string or bytes object, got a > > 'memoryview' object > > > > > > > pa.array(map(bytes,memory_views),pa.string()) > > > > <pyarrow.lib.StringArray object at 0x7f1745cbdd00> > > > > [ > > > > "this", > > > > "is", > > > > "a", > > > > "sample" > > > > ] > > > > > > > > I have a big list of byte sequences being provided to me as memoryviews > > from a third party library. I’d like to create an Arrow StringArray from > > them as efficiently as possible. Having to map and consequently copy them > > through a bytes constructor seems not great (and the memoryview tobytes > > function appears to just call the bytes constructor, afaict). > > > > > > > > To me, it seemed like pa.array should be able to use the memoryview objects > > directly in order to construct the StringArray, but it seems like Arrow > > wants them copied into fresh byte objects first. I don’t know if I > > understand why and was ultimately wondering if it’s a reasonable thing to > > desire. > > > > > > > > Thanks in advance, > > > > -Dan Nugent > > > > > > > > > > ###################################################################### > > > > The information contained in this communication is confidential and > > > > may contain information that is privileged or exempt from disclosure > > > > under applicable law. If you are not a named addressee, please notify > > > > the sender immediately and delete this email from your system. > > > > If you have received this communication, and are not a named > > > > recipient, you are hereby notified that any dissemination, > > > > distribution or copying of this communication is strictly prohibited. > > > > ######################################################################
