[ 
https://issues.apache.org/jira/browse/ARROW-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254708#comment-16254708
 ] 

ASF GitHub Bot commented on ARROW-1559:
---------------------------------------

wesm commented on issue #1266: ARROW-1559: [C++] Add Unique kernel and refactor 
DictionaryBuilder to be a stateful kernel
URL: https://github.com/apache/arrow/pull/1266#issuecomment-344807083
 
 
   So we're about a factor of 2 or so away from `pandas.unique`:
   
   ```
   In [1]: import pandas as pd
   import numpy as np
   import pyarrow as pa
      ...: 
   
   In [2]: arr = pa.array(list(range(10000)) * 100)
   
   In [3]: v = arr.to_pandas().copy()
   
   In [4]: timeit arr.unique()
   pd.unique(100 loops, best of 3: 10.4 ms per loop
   
   In [5]: timeit pd.unique(v)
   100 loops, best of 3: 4.82 ms per loop
   ```
   
   I suspect this is because of our hash function being slow, but we'll need to 
do some more analysis

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[email protected]


> [C++] Kernel implementations for "unique" (compute distinct elements of array)
> ------------------------------------------------------------------------------
>
>                 Key: ARROW-1559
>                 URL: https://issues.apache.org/jira/browse/ARROW-1559
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++
>            Reporter: Wes McKinney
>            Assignee: Uwe L. Korn
>              Labels: Analytics, pull-request-available
>             Fix For: 0.8.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to