[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-05-04 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464284#comment-16464284 ] Apache Spark commented on SPARK-23325: -- User 'rdblue' has created a pull request for this issue:

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-04-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446313#comment-16446313 ] Apache Spark commented on SPARK-23325: -- User 'rdblue' has created a pull request for this issue:

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-12 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395752#comment-16395752 ] Ryan Blue commented on SPARK-23325: --- I created SPARK-23657 to track this in parallel. Feel free to

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-12 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395655#comment-16395655 ] Ryan Blue commented on SPARK-23325: --- I agree that the binary format is more work and probably out of

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-08 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392048#comment-16392048 ] Wenchen Fan commented on SPARK-23325: - It's hard to stabilize the binary format like `UnsafeRow` and

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-08 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392046#comment-16392046 ] Wenchen Fan commented on SPARK-23325: - I think it's mostly document work. We need to add document for

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391861#comment-16391861 ] Michael Armbrust commented on SPARK-23325: -- It does seem like it would be that hard to stabilize

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-08 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391768#comment-16391768 ] Ryan Blue commented on SPARK-23325: --- By exposing an interface that uses UnsafeRow, don't we already

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389135#comment-16389135 ] Reynold Xin commented on SPARK-23325: - Yes perhaps we should do that. It is a lot more work than what

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388520#comment-16388520 ] Wenchen Fan commented on SPARK-23325: - Making `InternalRow` stable is not only about stabilizing the

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388411#comment-16388411 ] Ryan Blue commented on SPARK-23325: --- I agree that we should declare \{{InternalRow}} stable. It is

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Jose Torres (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388399#comment-16388399 ] Jose Torres commented on SPARK-23325: - How hard would it be to just declare that InternalRow is

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388322#comment-16388322 ] Wenchen Fan commented on SPARK-23325: - The problem is that, `Row` is a stable class Spark promises it

[jira] [Commented] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

2018-02-02 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351043#comment-16351043 ] Ryan Blue commented on SPARK-23325: --- [~cloud_fan], FYI. > DataSourceV2 readers should always produce