[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[ https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448800#comment-16448800 ] Antoine Pitrou commented on ARROW-2499: --- Thanks! We can start from that indeed. > [C++] Add iterator facility for Python sequences > > > Key: ARROW-2499 > URL: https://issues.apache.org/jira/browse/ARROW-2499 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Reporter: Antoine Pitrou >Priority: Major > > The idea is to factor out something like the following: > https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78 > However I'm not sure which idiom or pattern we should choose. [~cpcloud] any > idea? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[
https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448799#comment-16448799
]
Phillip Cloud commented on ARROW-2499:
--
[~pitrou] Looks like there's {{LoopPySequence}} in {{numpy_to_arrow.cc}}, if
that satisfies the need.
> [C++] Add iterator facility for Python sequences
>
>
> Key: ARROW-2499
> URL: https://issues.apache.org/jira/browse/ARROW-2499
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++
>Reporter: Antoine Pitrou
>Priority: Major
>
> The idea is to factor out something like the following:
> https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78
> However I'm not sure which idiom or pattern we should choose. [~cpcloud] any
> idea?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[ https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448795#comment-16448795 ] Phillip Cloud commented on ARROW-2499: -- That works well too. > [C++] Add iterator facility for Python sequences > > > Key: ARROW-2499 > URL: https://issues.apache.org/jira/browse/ARROW-2499 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Reporter: Antoine Pitrou >Priority: Major > > The idea is to factor out something like the following: > https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78 > However I'm not sure which idiom or pattern we should choose. [~cpcloud] any > idea? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[
https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448790#comment-16448790
]
Antoine Pitrou commented on ARROW-2499:
---
That would be nice. Otherwise I might end up writing a {{for_each}}-like
primitive, something like:
{code:cpp}
// Iterate on the Python sequence, calling the given callable on each element.
// If the callable returns a non-OK status, iteration stops and the status is
returned.
Status IterateSequence(PyObject*, const std::function&);
{code}
> [C++] Add iterator facility for Python sequences
>
>
> Key: ARROW-2499
> URL: https://issues.apache.org/jira/browse/ARROW-2499
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++
>Reporter: Antoine Pitrou
>Priority: Major
>
> The idea is to factor out something like the following:
> https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78
> However I'm not sure which idiom or pattern we should choose. [~cpcloud] any
> idea?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[ https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448786#comment-16448786 ] Phillip Cloud commented on ARROW-2499: -- I can take a crack at this to show you what I mean (my suggestion may also not work out, there are still some things about idiomatic C++ that I don't yet grok). [~joshuastorck] might also have suggestions here. > [C++] Add iterator facility for Python sequences > > > Key: ARROW-2499 > URL: https://issues.apache.org/jira/browse/ARROW-2499 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Reporter: Antoine Pitrou >Priority: Major > > The idea is to factor out something like the following: > https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78 > However I'm not sure which idiom or pattern we should choose. [~cpcloud] any > idea? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[
https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448780#comment-16448780
]
Antoine Pitrou commented on ARROW-2499:
---
{{generic_iterator}} is a template class in pybind11, so I'm not sure how that
works.
> [C++] Add iterator facility for Python sequences
>
>
> Key: ARROW-2499
> URL: https://issues.apache.org/jira/browse/ARROW-2499
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++
>Reporter: Antoine Pitrou
>Priority: Major
>
> The idea is to factor out something like the following:
> https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78
> However I'm not sure which idiom or pattern we should choose. [~cpcloud] any
> idea?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[
https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448767#comment-16448767
]
Phillip Cloud commented on ARROW-2499:
--
You'd still have to do the checks and choose the right _specific_ iterator but
in theory you'd be able to have a single {{VisitSequence(const
generic_iterator& iter)}} and loop over the elements using range-based for
loops, which gets rid of the loop duplication. I'm not sure how to get rid of
the checks since we only have {{PyObject*}} at the call site.
> [C++] Add iterator facility for Python sequences
>
>
> Key: ARROW-2499
> URL: https://issues.apache.org/jira/browse/ARROW-2499
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++
>Reporter: Antoine Pitrou
>Priority: Major
>
> The idea is to factor out something like the following:
> https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78
> However I'm not sure which idiom or pattern we should choose. [~cpcloud] any
> idea?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[ https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448756#comment-16448756 ] Antoine Pitrou commented on ARROW-2499: --- That looks nice, but I would like to know how to abstract away the fact that we need two separate loops for performance in the example above. pybind11 basically has us write the checks and separate loops by hands. Perhaps a `for_each` like facility? But what should the signature look like? Also, should we take the pybind11 code as-is or write our own? > [C++] Add iterator facility for Python sequences > > > Key: ARROW-2499 > URL: https://issues.apache.org/jira/browse/ARROW-2499 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Reporter: Antoine Pitrou >Priority: Major > > The idea is to factor out something like the following: > https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78 > However I'm not sure which idiom or pattern we should choose. [~cpcloud] any > idea? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-2499) [C++] Add iterator facility for Python sequences
[ https://issues.apache.org/jira/browse/ARROW-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448739#comment-16448739 ] Phillip Cloud commented on ARROW-2499: -- pybind11 actually has a nice implementation of C++ iterators on the core python sequence types (https://github.com/pybind/pybind11/blob/master/include/pybind11/pytypes.h#L559-L682). Maybe we take some ideas from there? > [C++] Add iterator facility for Python sequences > > > Key: ARROW-2499 > URL: https://issues.apache.org/jira/browse/ARROW-2499 > Project: Apache Arrow > Issue Type: Wish > Components: C++ >Reporter: Antoine Pitrou >Priority: Major > > The idea is to factor out something like the following: > https://github.com/apache/arrow/pull/1935/files#diff-6ea0fcd65b95b76eab9ddfbd7a173725R78 > However I'm not sure which idiom or pattern we should choose. [~cpcloud] any > idea? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
