[jira] [Assigned] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Apache Spark (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-42789:


Assignee: Apache Spark

> rewrites multiple GetJsonObjects to a JsonTuple if their json expression is 
> the same
> 
>
> Key: SPARK-42789
> URL: https://issues.apache.org/jira/browse/SPARK-42789
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 3.5.0
>Reporter: Yuming Wang
>Assignee: Apache Spark
>Priority: Major
>
> Benchmark result:
> {noformat}
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 2
>   Stopped after 2 iterations, 80787 ms
>   Running case: Rewrite: 2
>   Stopped after 2 iterations, 48900 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 239026  40394
> 1935  0.25397.8   1.0X
> Rewrite: 224354  24450
>  137  0.33368.4   1.6X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 3
>   Stopped after 2 iterations, 115055 ms
>   Running case: Rewrite: 3
>   Stopped after 2 iterations, 62297 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 354652  57528
>  NaN  0.17559.1   1.0X
> Rewrite: 330702  31149
>  631  0.24246.6   1.8X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 4
>   Stopped after 2 iterations, 155392 ms
>   Running case: Rewrite: 4
>   Stopped after 2 iterations, 54776 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 475503  77696
>  NaN  0.1   10443.1   1.0X
> Rewrite: 426962  27388
>  602  0.33729.3   2.8X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 5
>   Stopped after 2 iterations, 192836 ms
>   Running case: Rewrite: 5
>   Stopped after 2 iterations, 51967 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 594923  96418
> 2115  0.1   13129.1   1.0X
> Rewrite: 525362  25984
>  880  0.33507.8   3.7X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 10
>   Stopped after 2 iterations, 317246 ms
>   Running case: Rewrite: 10
>   Stopped after 2 iterations, 56734 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 10  157458 158623
> 1648  0.0   21778.6   1.0X
> Rewrite: 10   28296  28367
>  100  0.33913.8   5.6X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 20
>   Stopped after 2 iterations, 618089 ms
>   Running case: Rewrite: 20
>

[jira] [Assigned] (SPARK-42789) rewrites multiple GetJsonObjects to a JsonTuple if their json expression is the same

2023-03-14 Thread Apache Spark (Jira) 


 [ 
https://issues.apache.org/jira/browse/SPARK-42789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-42789:


Assignee: (was: Apache Spark)

> rewrites multiple GetJsonObjects to a JsonTuple if their json expression is 
> the same
> 
>
> Key: SPARK-42789
> URL: https://issues.apache.org/jira/browse/SPARK-42789
> Project: Spark
>  Issue Type: Improvement
>  Components: SQL
>Affects Versions: 3.5.0
>Reporter: Yuming Wang
>Priority: Major
>
> Benchmark result:
> {noformat}
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 2
>   Stopped after 2 iterations, 80787 ms
>   Running case: Rewrite: 2
>   Stopped after 2 iterations, 48900 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 239026  40394
> 1935  0.25397.8   1.0X
> Rewrite: 224354  24450
>  137  0.33368.4   1.6X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 3
>   Stopped after 2 iterations, 115055 ms
>   Running case: Rewrite: 3
>   Stopped after 2 iterations, 62297 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 354652  57528
>  NaN  0.17559.1   1.0X
> Rewrite: 330702  31149
>  631  0.24246.6   1.8X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 4
>   Stopped after 2 iterations, 155392 ms
>   Running case: Rewrite: 4
>   Stopped after 2 iterations, 54776 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 475503  77696
>  NaN  0.1   10443.1   1.0X
> Rewrite: 426962  27388
>  602  0.33729.3   2.8X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 5
>   Stopped after 2 iterations, 192836 ms
>   Running case: Rewrite: 5
>   Stopped after 2 iterations, 51967 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 594923  96418
> 2115  0.1   13129.1   1.0X
> Rewrite: 525362  25984
>  880  0.33507.8   3.7X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 10
>   Stopped after 2 iterations, 317246 ms
>   Running case: Rewrite: 10
>   Stopped after 2 iterations, 56734 ms
> Java HotSpot(TM) 64-Bit Server VM 17.0.4.1+1-LTS-2 on Mac OS X 13.2.1
> Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
> Benchmark rewrite GetJsonObjects: Best Time(ms)   Avg Time(ms)   
> Stdev(ms)Rate(M/s)   Per Row(ns)   Relative
> 
> Default: 10  157458 158623
> 1648  0.0   21778.6   1.0X
> Rewrite: 10   28296  28367
>  100  0.33913.8   5.6X
> Running benchmark: Benchmark rewrite GetJsonObjects
>   Running case: Default: 20
>   Stopped after 2 iterations, 618089 ms
>   Running case: Rewrite: 20
>   Stopped after 2 iterations,