[ https://issues.apache.org/jira/browse/SPARK-47085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818878#comment-17818878 ]
Dongjoon Hyun commented on SPARK-47085: --------------------------------------- Hi, [~igreenfi]and [~yao]. Could you provide some background why this is a regression at 3.4.1 and 3.5.0? If this is not a regression at that version, we should change `Affected Versions` to `4.0.0` because this is an improvement. > Preformance issue on thrift API > ------------------------------- > > Key: SPARK-47085 > URL: https://issues.apache.org/jira/browse/SPARK-47085 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.4.1, 3.5.0 > Reporter: Izek Greenfield > Assignee: Izek Greenfield > Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > This new complexity was introduced in SPARK-39041. > In class `RowSetUtils` there is a loop that has _*O(n^2)*_ complexity: > {code:scala} > ... > while (i < rowSize) { > val row = rows(I) > ... > {code} > It can be easily converted back into _*O( n )*_ complexity. > > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org