Hi all, Please disregard my previous announcement - the release is NOT yet officially released.
I missed the Incubator PMC voting step on [email protected]. I will initiate the proper vote immediately. Apologies for the confusion! Best regards, Ruilei Ma Ruilei Ma <[email protected]> 于2026年2月25日周三 15:07写道： > Hi all, > > The Apache Auron (Incubating) community is pleased to announce the new > release of Apache Auron (Incubating) 7.0.0. > > Auron is dedicated to improving the efficiency and elasticity of different > map-reduce engines and provides an elastic, highly efficient service for > intermediate data including shuffle data, spilled data, result data, etc. > > Currently Auron supports Spark full-featured and improves Spark job's > performance, stability and elasticity. > > Download Link: > ● https://auron.apache.org/archives/v7.0.0-incubating.html > (Starting from v6.0.0, Auron only provides source download because there > are many new build options. Users can build with their own environment and > options, or build on GitHub Actions by forking the repository.) > > GitHub Release Tag: > ● https://github.com/apache/auron/releases/tag/v7.0.0 > > Release Notes: > ● https://auron.apache.org/archives/v7.0.0-incubating.html > > Website: > ● https://auron.apache.org/ > > Auron Resources: > ● Issue: https://github.com/apache/auron/issues > ● Mailing list: [email protected] > > > Best regards, > > Ruilei Ma > On behalf of the Apache Auron(incubating) community > > > Release notes: > > ## New Features > * **AuronConfiguration**: Introduced new configuration system with dynamic > default values > * **New Modules**: Added auron-core, auron-flink-extension, auron-memmgr, > auron-iceberg, and auron-spark-tests modules for better modularity > * **Spark UI Integration**: Introduced comprehensive Spark UI support with > build info display and metrics > * **Extended Native Functions**: Implemented native functions for round, > pow/power, lpad, rpad, reverse, initcap, levenshtein, quarter, hour, > minute, second, find_in_set, nvl, nvl2, least, greatest, isnan, bround > * **Enhanced Expressions**: Native support for EqualNullSafe expression, > CollectLimit, monotonically_increasing_id(), and spark_partition_id() > * **Columnar Aggregation**: Implemented columnar aggregate buffers for > improved performance > * **JniBridge & AuronAdaptor**: Introduced for better Java-Rust > interoperability > > ## Improvements > * **Performance Optimization**: > - Improved SortMergeJoin by moving has_multi_equal out of loop > - Optimized join operation to flush in time on duplicated keys > * **Enhanced Compatibility**: > - Support for Spark 3.5.8 and Spark 4.0.2 > - Initial compatibility support for Spark 4.1 > - Support for JDK 21 > - Upgraded from JUnit 4 to JUnit 5 > * **Build & CI**: > - Unified GitHub Actions runner image to Ubuntu 24.04 > - Standardized Maven build summary output to align with Apache > formatting standards > - Introduced auron-it for enhanced CI integration testing > - Enabled Clippy lint with -D warnings enforcement > - Extracted Rust test logic to standalone CI workflow > * **Observability**: > - Improved Spark UI integration with build info and metrics > - Support for DataSourceScan metrics > - Enhanced memory logging readability > * **Configuration Management**: > - Added spark.auron.enable.broadcastexchange and > spark.auron.enable.shuffleExchange configs > - Added spark.auron.expression.singleChildFallback.enabled config > - Introduced SparkAuronConfiguration and deprecated AuronConf > * **Dependency Updates**: > - Upgraded Spark 3.5.7 to 3.5.8 > - Bumped Uniffle from 0.9.2 to 0.10.0 > - Bumped Celeborn from 0.6.0 to 0.6.2 > - Bumped Maven from 3.9.11 to 3.9.12 > * **Memory Management**: > - Introduced SparkOnHeapSpillManager for better memory control > - Fixed possible deadlock in OnHeapSpillManager > * **Code Quality**: > - Added Maven checkstyle and scalastyle plugins > - Removed "Exec" suffix for physical operators for cleaner naming > - Eliminated unwrap usage and enforced Clippy lint > - Cleaned up Clippy format/string/condition/logic lints > > ## Bug Fixes > * **Data Correctness**: > - Fixed concat_ws null handling > - Fixed Casting from Struct and Map to Utf8 not supported > - Fixed NULL handling for NOT IN subquery > - Fixed isNan semantics aligned with Spark > - Fixed initCap semantics aligned with Spark > - Fixed StringSplit conversion > - Fixed ORC incorrect results from RLEv2 patched base > - Fixed ORC check offset overflow > - Fixed Generate index out of bounds > - Fixed unexpected invocation of UDF > * **Type System**: > - Fixed type mismatch: cast lpad/rpad len to LongType > - Fixed string_lower only supports literal utf8 > - Fixed Casting from Array/Map/Struct to Utf8 not supported > - Fixed FileScanConfig.projection conversion > * **Execution Engine**: > - Fixed disable convert BroadcastExchange to native does not take effect > - Fixed disable convert ShuffleExchange to native does not take effect > - Fixed NativeEmptyExec should fallback on unsupported types > - Fixed IllegalArgumentException in buildTimePartExt > - Fixed MD5 output back to utf8 > - Fixed java.lang.NoClassDefFoundError with shaded-spark > * **Performance**: > - Fixed SortMergeJoin operation on duplicated keys > - Fixed possible deadlock in OnHeapSpillManager > * **Ecosystem Integration**: > - Fixed data size calculation in Celeborn shuffle writer > - Fixed UDAF fallback bug when handling DeclarativeAggregator > - Fixed spark_normalize_nan_and_zero None > * **Testing**: > - Fixed FileNotFoundException in BuildinfoInSparkUISuite > - Fixed NativeConvertersSuite cast trim-disable test failures > - Fixed and enhanced nvl2 unit test > * **Documentation**: > - Clarified Docker build instructions > - Improved README formatting and grammar > - Added contributing guidelines > > ## Notable Changes > * **Module Restructuring**: Refactored JniBridge and AuronConf, moved to > auron-core module > * **Testing Enhancement**: Added comprehensive correctness testing for > Spark 3.3 (Aggregate, Sort, Parquet, Functions, Expressions) > * **Docker Support**: Expanded build support to modern Linux distributions > and added Azure Linux 3 support > * **License Compliance**: Added Apache-2.0 license declaration to all > native-engine Cargo.toml files > > NOTE: This release includes a significant number of new features, > performance optimizations, bug fixes, and improvements across modularity, > compatibility, observability, and testing. The project continues to enhance > its ecosystem integration with Uniffle and Celeborn, while maintaining > strong backward compatibility with Spark 3.x and forward compatibility with > Spark 4.x. Some minor changes are not included in the above list, please > see the commit list for more details. > > ## What's Changed > * [RELEASE] Bump version 7.0.0-SNAPSHOT by @richox in > https://github.com/apache/auron/pull/1343 > * bump spark-3.5 version to 3.5.7 by @XorSum in > https://github.com/apache/auron/pull/1353 > * [AURON #1351] Unify GitHub Actions runner image to Ubuntu 24.04 by > @merrily01 in https://github.com/apache/auron/pull/1352 > * [AURON #1344] Delete unused profile in pom.xml by @guixiaowen in > https://github.com/apache/auron/pull/1345 > * [AURON #1355] Upgrade actions/cache from v3 to v4 in > `build-ce7-release.yml` by @merrily01 in > https://github.com/apache/auron/pull/1356 > * [AURON-1339] Introduce AuronConfiguration by @Tartarus0zm in > https://github.com/apache/auron/pull/1340 > * [AURON #1341] Eliminate hardcoded values in the > AuronSparkSessionExtension class by @guixiaowen in > https://github.com/apache/auron/pull/1342 > * [AURON #1358] Rename package name for hadoop-shim module by @Tartarus0zm > in https://github.com/apache/auron/pull/1359 > * bump maven version to 3.9.11 by @cxzl25 in > https://github.com/apache/auron/pull/1350 > * Bump serde from 1.0.226 to 1.0.227 by @dependabot[bot] in > https://github.com/apache/auron/pull/1357 > * Remove the "Exec" suffix for physical operators by @XorSum in > https://github.com/apache/auron/pull/1349 > * [AURON #1365] Fix artifact name formatting issue with conditional > version parameters by @merrily01 in > https://github.com/apache/auron/pull/1366 > * [AURON #1367] Bump Uniffle from 0.9.2 to 0.10.0 by @merrily01 in > https://github.com/apache/auron/pull/1354 > * [AURON #1362]Add function description() in ConfigOption by @guixiaowen > in https://github.com/apache/auron/pull/1363 > * [AURON-1332] Introduce JniBridge and AuronAdaptor for auron-core module > by @Tartarus0zm in https://github.com/apache/auron/pull/1333 > * fix UDAF fallback bug when handling DeclarativeAggregator by @richox in > https://github.com/apache/auron/pull/1369 > * Bump sonic-rs from 0.5.4 to 0.5.5 by @dependabot[bot] in > https://github.com/apache/auron/pull/1377 > * Bump serde from 1.0.227 to 1.0.228 by @dependabot[bot] in > https://github.com/apache/auron/pull/1376 > * [AURON #1374] Introduce AuronUDFWrapperContext by @Tartarus0zm in > https://github.com/apache/auron/pull/1375 > * [AURON #1383] fix UT fail by @Tartarus0zm in > https://github.com/apache/auron/pull/1384 > * [AURON #1385] Move Auron Native lib from spark-extension to auron-core > by @Tartarus0zm in https://github.com/apache/auron/pull/1386 > * [AURON #1387] Introduce auron-flink-extension module by @Tartarus0zm in > https://github.com/apache/auron/pull/1389 > * Fix typo in AuronConverters by @guixiaowen in > https://github.com/apache/auron/pull/1380 > * [AURON #1381] Modify Auron run unit tests by default by @Tartarus0zm in > https://github.com/apache/auron/pull/1382 > * [AURON #1423]Refactor convertBroadcastExchangeExec to use direct type > parameter instead of pattern matching by @guixiaowen in > https://github.com/apache/auron/pull/1424 > * [AURON #1415]Add version check in auron-build.sh by @guixiaowen in > https://github.com/apache/auron/pull/1416 > * [AURON #1419]Add neverConvertReasonTag to record the reason for > non-conversion. by @guixiaowen in > https://github.com/apache/auron/pull/1420 > * [AURON#1411] Doc: Update the Uniffle build message in auron-build.sh. by > @slfan1989 in https://github.com/apache/auron/pull/1412 > * Fix typo in README by @guixiaowen in > https://github.com/apache/auron/pull/1425 > * [AURON #1421]add flink in reformat by @guixiaowen in > https://github.com/apache/auron/pull/1422 > * [AURON#1407] Fix CentOS 7 Dockerfile build failure. by @slfan1989 in > https://github.com/apache/auron/pull/1408 > * [AURON#1396] Upgrade from JUnit 4 to JUnit 5. by @slfan1989 in > https://github.com/apache/auron/pull/1397 > * Bump Maven Surefire from 3.3.0 to 3.5.3. by @slfan1989 in > https://github.com/apache/auron/pull/1400 > * [AURON #1378]add spark.auron.enable.broadcastexchange to control > BroadcastExchangeExec into Native or not by @guixiaowen in > https://github.com/apache/auron/pull/1379 > * Bump parking_lot from 0.12.4 to 0.12.5 by @dependabot[bot] in > https://github.com/apache/auron/pull/1406 > * [AURON#1413] Standardize Maven build summary output to align with Apache > formatting standards. by @slfan1989 in > https://github.com/apache/auron/pull/1414 > * [AURON#1398] Support for Building on JDK 21. by @slfan1989 in > https://github.com/apache/auron/pull/1399 > * [AURON#1327] Implement native function of `round` by @slfan1989 in > https://github.com/apache/auron/pull/1426 > * [AURON #1428]Follow #1413 in auron-flink-extension by @guixiaowen in > https://github.com/apache/auron/pull/1429 > * [AURON #1372]add spark.auron.enable.shuffleExchange to control > ShuffleExchangeExec into Native or not by @guixiaowen in > https://github.com/apache/auron/pull/1373 > * [AURON #1409] Introduce Spark UI for auron by @guixiaowen in > https://github.com/apache/auron/pull/1410 > * [AURON #1314] Speed up CI by using Apache `closer.lua` syntax for > dependency downloads by @merrily01 in > https://github.com/apache/auron/pull/1315 > * [AURON #1440]Follow #1413 in spark-ui by @guixiaowen in > https://github.com/apache/auron/pull/1441 > * [AURON #1417] Expand build support to modern Linux distributions. by > @slfan1989 in https://github.com/apache/auron/pull/1418 > * [AURON #1431] Add EmptyNativeRDD extending NativeRDD by @guixiaowen in > https://github.com/apache/auron/pull/1430 > * [AURON #1437] Introduce AuronArrowFFIExporter by @Tartarus0zm in > https://github.com/apache/auron/pull/1439 > * [AURON #1445] Add license check for flink module by @Tartarus0zm in > https://github.com/apache/auron/pull/1446 > * [AURON #1447] Fix auron-flink-extension module junit dependency conflict > with flink by @Tartarus0zm in https://github.com/apache/auron/pull/1448 > * [AURON #1449] introduce flink test workflow by @Tartarus0zm in > https://github.com/apache/auron/pull/1450 > * Bump tokio from 1.47.1 to 1.48.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1453 > * [AURON #1454] Fix redundant hyphen in log filename construction by > @merrily01 in https://github.com/apache/auron/pull/1455 > * [AURON #1434] Use Maven-native version extraction instead of xmllint for > better portability by @merrily01 in > https://github.com/apache/auron/pull/1435 > * [AURON #1456] Introduce SparkAuronConfiguration by @Tartarus0zm in > https://github.com/apache/auron/pull/1457 > * [AURON #1409] followup to always get active context by @zuston in > https://github.com/apache/auron/pull/1433 > * [AURON #1461] Align Auron tab style with Spark UI naming conventions by > @merrily01 in https://github.com/apache/auron/pull/1462 > * [AURON #1458] Centralize supported versions and simplify validation with > unified build info output by @merrily01 in > https://github.com/apache/auron/pull/1459 > * [AURON #1468] Introduce SparkMetricNode by @Tartarus0zm in > https://github.com/apache/auron/pull/1469 > * [AURON #1476] Remove extra parenthesis in assignment causing invalid > Docker build context by @merrily01 in > https://github.com/apache/auron/pull/1477 > * [AURON #1482] Support native avg result cast to return type by > @Tartarus0zm in https://github.com/apache/auron/pull/1483 > * Bump tikv-jemalloc-ctl from 0.6.0 to 0.6.1 by @dependabot[bot] in > https://github.com/apache/auron/pull/1467 > * Bump foldhash from 0.1.5 to 0.2.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1466 > * Bump procfs from 0.17.0 to 0.18.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1465 > * [AURON #1488] Remove isDriverSide method from JniBridge by @Tartarus0zm > in https://github.com/apache/auron/pull/1489 > * [AURON #1493] Remove unused getSparkEnvConfAsString method from > JniBridge by @Tartarus0zm in https://github.com/apache/auron/pull/1494 > * Bump bigdecimal from 0.4.8 to 0.4.9 by @dependabot[bot] in > https://github.com/apache/auron/pull/1475 > * [AURON #1496] Introduce SparkOnHeapSpillManager by @Tartarus0zm in > https://github.com/apache/auron/pull/1497 > * Bump tikv-jemallocator from 0.6.0 to 0.6.1 by @dependabot[bot] in > https://github.com/apache/auron/pull/1495 > * [AURON #1473] Rename module spark-extension-shims-spark3 to > spark-extension-shims-spark. by @slfan1989 in > https://github.com/apache/auron/pull/1474 > * [AURON #1491] Add workflow for paimon #1491 by @guixiaowen in > https://github.com/apache/auron/pull/1490 > * split memory manager to module auron-memmgr by @richox in > https://github.com/apache/auron/pull/1506 > * Bump byte-buddy from 1.12.10 to 1.14.11. by @slfan1989 in > https://github.com/apache/auron/pull/1505 > * [AURON #1498] Enable the JniBridge in the auron-core module by > @Tartarus0zm in https://github.com/apache/auron/pull/1499 > * [AURON #1507] Enable AuronAdaptor#getAuronUDFWrapperContext for spark by > @Tartarus0zm in https://github.com/apache/auron/pull/1508 > * [AURON #1509] Deprecated JniBridge and AuronCallNativeWrapper under the > spark-extension module by @Tartarus0zm in > https://github.com/apache/auron/pull/1510 > * [AURON #1525] Fallback outputs the exception stack when the log level is > debug by @cxzl25 in https://github.com/apache/auron/pull/1525 > * [AURON #1523] Fix CheckStyle Violations in Auron Codebase. by @slfan1989 > in https://github.com/apache/auron/pull/1523 > * [AURON #1516] Add work flow for jdk 21 #1516 by @guixiaowen in > https://github.com/apache/auron/pull/1517 > * [AURON #1513] Implement native function of pow/power. by @slfan1989 in > https://github.com/apache/auron/pull/1514 > * [AURON #1480] Add auron-iceberg project #1480 by @guixiaowen in > https://github.com/apache/auron/pull/1486 > * [AURON #1526] Add `auron-build.sh` to release workflow trigger paths by > @merrily01 in https://github.com/apache/auron/pull/1527 > * [AURON #1528] Add `auron-build-info.properties` to `.gitignore` by > @cxzl25 in https://github.com/apache/auron/pull/1528 > * [AURON #1533] Implement native function of lpad, rpad. by @slfan1989 in > https://github.com/apache/auron/pull/1534 > * [AURON #1529] Remove obsolete `version` attribute to prevent confusion > by @merrily01 in https://github.com/apache/auron/pull/1530 > * [AURON #1538] Implement native function of `reverse`. by @slfan1989 in > https://github.com/apache/auron/pull/1540 > * [AURON #1531] Enable SparkAuronConfiguration and deprecated AuronConf by > @Tartarus0zm in https://github.com/apache/auron/pull/1532 > * [AURON #1545] Clarify Docker build instructions to reflect supported OS > images by @merrily01 in https://github.com/apache/auron/pull/1545 > * [AURON #1544] Remove outdated `RELEASES.md` file by @merrily01 in > https://github.com/apache/auron/pull/1544 > * [AURON #1543] Add Docker build support for Azure Linux 3 by @adityavaish > in https://github.com/apache/auron/pull/1542 > * [AURON #1549] Implement native function of `initcap`. by @slfan1989 in > https://github.com/apache/auron/pull/1550 > * [AURON #1571]Implement native function of levenshtein(80) #1571 by > @guixiaowen in https://github.com/apache/auron/pull/1573 > * [AURON #1518] Implement native function of quarter. by @slfan1989 in > https://github.com/apache/auron/pull/1519 > * [AURON #1593] Add Spark prefix for log option key by @Tartarus0zm in > https://github.com/apache/auron/pull/1594 > * [AURON #1520] Implement native function of hour, minute, second. by > @slfan1989 in https://github.com/apache/auron/pull/1522 > * [AURON #1588]Implement native function of find_in_set(81)，nvl(82), > nv2(83) ,least(84), greatest(85), isnan(87) #1588 by @guixiaowen in > https://github.com/apache/auron/pull/1585 > * [AURON #1442] Fix `BuildinfoInSparkUISuite.scala` failure due to missing > `/tmp/spark-events/` directory by @merrily01 in > https://github.com/apache/auron/pull/1444 > * [AURON #1597] Fix java.lang.NoClassDefFoundError: > io/netty/buffer/Unpooled when running with shaded-spark by > @ShreyeshArangath in https://github.com/apache/auron/pull/1598 > * [AURON #1442] Fix FileNotFoundException in BuildinfoInSparkUISuite by > properly configuring temp event log dir by @yew1eb in > https://github.com/apache/auron/pull/1609 > * [AURON #1613] Introduce AuronExtFunctions by @Tartarus0zm in > https://github.com/apache/auron/pull/1614 > * chore: Clean up redundant NOTICE entry by @CalvinKirs in > https://github.com/apache/auron/pull/1616 > * Bump sonic-rs from 0.5.5 to 0.5.6 by @dependabot[bot] in > https://github.com/apache/auron/pull/1603 > * [AURON #1516][FOLLOWUP] Use JDK 21 for CentOS 7 release by @cxzl25 in > https://github.com/apache/auron/pull/1608 > * [AURON #1621] Introduces dynamic default value for ConfigOption by > @Tartarus0zm in https://github.com/apache/auron/pull/1622 > * Bump bytesize from 2.1.0 to 2.2.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1619 > * [AURON #1604] Make MD5 output back to utf8 by @Flyangz in > https://github.com/apache/auron/pull/1605 > * [AURON #1612] Filter out empty build-info values to improve Spark UI > display by @merrily01 in https://github.com/apache/auron/pull/1615 > * [AURON #1629] Fix out-of-date module name in labeler.yml by @yew1eb in > https://github.com/apache/auron/pull/1629 > * Bump bytes from 1.10.1 to 1.11.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1634 > * [AURON #1631] Updates the labeler workflow to cover the newly added > modules in the Auron project by @merrily01 in > https://github.com/apache/auron/pull/1632 > * [AURON #1602] Implement AuronAdaptor SPI discovery with Spark provider > by @yew1eb in https://github.com/apache/auron/pull/1620 > * [AURON #1650] Override nodeName for NativeEmptyExec by @yew1eb in > https://github.com/apache/auron/pull/1651 > * Bump bytesize from 2.2.0 to 2.3.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1648 > * Bump lz4_flex from 0.11.5 to 0.12.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1623 > * [AURON #1579] Fix and enhance nvl2 unit test by @yew1eb in > https://github.com/apache/auron/pull/1635 > * [AURON #1617] Fix NativeConvertersSuite cast trim-disable test failures > by @yew1eb in https://github.com/apache/auron/pull/1618 > * [AURON #1652] Fix: add Spark_ prefix for Hashing Functions in Auron ext > functions by @yew1eb in https://github.com/apache/auron/pull/1653 > * [AURON #1654] Add AuronQueryTest base class for SQL testing by @yew1eb > in https://github.com/apache/auron/pull/1655 > * [AURON #1644] Fix build-native.sh ensure native lib cache takes effect > by @yew1eb in https://github.com/apache/auron/pull/1645 > * [AURON #1537] Fix incorrect FileScanConfig.projection conversion by > @richox in https://github.com/apache/auron/pull/1591 > * [AURON #1636] Bump orc-rust 0.7.0 by @cxzl25 in > https://github.com/apache/auron/pull/1661 > * [AURON #1663] Fix: NativeEmptyExec should fallback on unsupported types > by @yew1eb in https://github.com/apache/auron/pull/1664 > * [AURON #1659] Remove ubuntu-22.04-arm runner by @cxzl25 in > https://github.com/apache/auron/pull/1660 > * [AURON #1656] Support to disable the scan timestamp for Parquet and ORC > formats by @cxzl25 in https://github.com/apache/auron/pull/1657 > * Bump bytesize from 2.3.0 to 2.3.1 by @dependabot[bot] in > https://github.com/apache/auron/pull/1669 > * [AURON #1674] Fix: route Upper to Spark_StringUpper by @yew1eb in > https://github.com/apache/auron/pull/1675 > * [AURON #1595] Enforce skiptest=false to not miss failing UT by > @ShreyeshArangath in https://github.com/apache/auron/pull/1596 > * [AURON #1665] Override verboseStringWithOperatorId in > NativeFileSourceScanBase by @yew1eb in > https://github.com/apache/auron/pull/1643 > * [AURON #1668] Remove the usage of BASH associative array for bash 3.x > version by @xumingming in https://github.com/apache/auron/pull/1672 > * [AURON #1665] Override verboseStringWithOperatorId in NativeFilterBase > by @yew1eb in https://github.com/apache/auron/pull/1667 > * [AURON #1665] Override verboseStringWithOperatorId in NativeProjectBase > by @yew1eb in https://github.com/apache/auron/pull/1666 > * [AURON #1625] Fix disable convert BroadcastExchange to native does not > take effect by @yew1eb in https://github.com/apache/auron/pull/1627 > * [AURON #1625] Fix disable convert ShuffleExchange to native does not > take effect by @yew1eb in https://github.com/apache/auron/pull/1628 > * Bump log from 0.4.28 to 0.4.29 by @dependabot[bot] in > https://github.com/apache/auron/pull/1692 > * [AURON #1690] Improving build-native.sh by @yew1eb in > https://github.com/apache/auron/pull/1691 > * [AURON #1646] `isNan` semantics are aligned with Spark by > @ShreyeshArangath in https://github.com/apache/auron/pull/1647 > * [AURON #1686]Fix IllegalArgumentException in buildTimePartExt leading to > fallback to Spark by @yew1eb in https://github.com/apache/auron/pull/1687 > * [AURON #1702] Fix typo in join_hash_map.rs by @xumingming in > https://github.com/apache/auron/pull/1703 > * [AURON #1720] Exclude temp file: auron-build-info.properties from > apache-rat plugin by @xumingming in > https://github.com/apache/auron/pull/1721 > * [AURON #1727] Fix concat_ws null by @cxzl25 in > https://github.com/apache/auron/pull/1728 > * [AURON #1713] `dev/reformat` format Rust early by @cxzl25 in > https://github.com/apache/auron/pull/1719 > * [AURON #1722] Use PR title and description for commit message by @cxzl25 > in https://github.com/apache/auron/pull/1723 > * [AURON #1708] Fix spark_normalize_nan_and_zero None by @cxzl25 in > https://github.com/apache/auron/pull/1709 > * [AURON #1730] Fix data size calculation in Celeborn shuffle writer by > @DDDominik in https://github.com/apache/auron/pull/1731 > * Revert "[AURON #1730] Fix data size calculation in Celeborn shuffle > writer" by @richox in https://github.com/apache/auron/pull/1736 > * [AURON #1688] Fix type mismatch: cast lpad/rpad len (2nd arg) to > LongType by @yew1eb in https://github.com/apache/auron/pull/1689 > * [AURON #1711] Fix Casting from Struct to Utf8 not supported by @cxzl25 > in https://github.com/apache/auron/pull/1718 > * [AURON #1707] Bump Celeborn 0.6.2 by @XorSum in > https://github.com/apache/auron/pull/1741 > * [AURON #1751] rename scalarTypeSupported -> isTypeSupported by > @xumingming in https://github.com/apache/auron/pull/1753 > * [AURON #1742] Disable Spark UI in test by @XorSum in > https://github.com/apache/auron/pull/1742 > * [AURON #1755] Use test-specific Log4j2 config in Maven test runs by > @yew1eb in https://github.com/apache/auron/pull/1756 > * [AURON #1748] ORC check offset overflow by @cxzl25 in > https://github.com/apache/auron/pull/1749 > * [AURON #1638] Support scan ORC data using microsecond precision by > @cxzl25 in https://github.com/apache/auron/pull/1684 > * [AURON #1695] Improve SortMergeJoin: move has_multi_equal out of loop by > @xumingming in https://github.com/apache/auron/pull/1696 > * [AURON #1680] initCap semantics are aligned with Spark by @yew1eb in > https://github.com/apache/auron/pull/1681 > * [AURON #1710] Fix string_lower only supports literal utf8 by @cxzl25 in > https://github.com/apache/auron/pull/1714 > * [AURON #1639] Avoid ByteBuddyAgent inject when class not found by > @cxzl25 in https://github.com/apache/auron/pull/1640 > * [AURON #1732] Add maven repository by @cxzl25 in > https://github.com/apache/auron/pull/1733 > * [AURON #1737] fix Generate index out of bounds by @cxzl25 in > https://github.com/apache/auron/pull/1738 > * [AURON #1712] Fix Casting from Map to Utf8 not supported by @cxzl25 in > https://github.com/apache/auron/pull/1744 > * [AURON #1725] Support Native CollectLimit by @yew1eb in > https://github.com/apache/auron/pull/1726 > * [AURON #1764] Fix Javadoc typo in AuronConfiguration: "mapper" → > "mapped" by @yew1eb in https://github.com/apache/auron/pull/1765 > * [AURON #1766] Improve memory log readability in NativeHelper by @yew1eb > in https://github.com/apache/auron/pull/1767 > * [AURON #1762] Fix ORC incorrect results from RLEv2 patched base by @dh20 > in https://github.com/apache/auron/pull/1768 > * [AURON #1771] build-native.sh set -e by @cxzl25 in > https://github.com/apache/auron/pull/1772 > * [AURON #1785] Rename NativeParquetSinkExec NativeRDD name to > NativeRDD.ParquetSink by @yew1eb in > https://github.com/apache/auron/pull/1786 > * [AURON #1776] Implement native support for EqualNullSafe expression by > @yew1eb in https://github.com/apache/auron/pull/1777 > * [AURON #1763] Fix unexpected invocation of UDF by @cxzl25 in > https://github.com/apache/auron/pull/1787 > * [AURON #1794] Fix IDEA import error by using fixed shim module name by > @yew1eb in https://github.com/apache/auron/pull/1795 > * Bump bigdecimal from 0.4.9 to 0.4.10 by @dependabot[bot] in > https://github.com/apache/auron/pull/1798 > * [AURON #1745] Introduce auron-spark-tests submodule for correctness > testing by @xumingming in https://github.com/apache/auron/pull/1752 > * [AURON #1484] Change test-jar dependencies scope to 'test' in > spark-extension-shims-spark module by @cxzl25 in > https://github.com/apache/auron/pull/1796 > * [AURON #1792] Keep the null result in the reverse connection result by > @dh20 in https://github.com/apache/auron/pull/1793 > * [AURON #1780] Fix the ORC table written by Hive to read null uppercase > fields by @dh20 in https://github.com/apache/auron/pull/1781 > * [AURON #1803] fix possible deadlock in OnHeapSpillManager by @richox in > https://github.com/apache/auron/pull/1804 > * [AURON #1822] Add label for auron-spark-tests PRs by @ShreyeshArangath > in https://github.com/apache/auron/pull/1823 > * [AURON #1760] Enable Dependabot updates for GitHub Actions by @yew1eb in > https://github.com/apache/auron/pull/1761 > * Bump actions/stale from 5 to 10 by @dependabot[bot] in > https://github.com/apache/auron/pull/1824 > * Bump actions/upload-artifact from 4 to 6 by @dependabot[bot] in > https://github.com/apache/auron/pull/1829 > * [AURON #1805] Add contributing guidelines by @ShreyeshArangath in > https://github.com/apache/auron/pull/1806 > * Bump arduino/setup-protoc from 2 to 3 by @dependabot[bot] in > https://github.com/apache/auron/pull/1826 > * [AURON #1790] Rename workflow build-ce-releases.yml by @yew1eb in > https://github.com/apache/auron/pull/1791 > * [AURON #1790] Fix: correct build workflow badge link in README by > @yew1eb in https://github.com/apache/auron/pull/1836 > * Bump actions/labeler from 4 to 6 by @dependabot[bot] in > https://github.com/apache/auron/pull/1830 > * Bump actions/cache from 4 to 5 by @dependabot[bot] in > https://github.com/apache/auron/pull/1825 > * Bump actions/setup-java from 4 to 5 by @dependabot[bot] in > https://github.com/apache/auron/pull/1827 > * Bump actions/checkout from 4 to 6 by @dependabot[bot] in > https://github.com/apache/auron/pull/1831 > * Bump tokio from 1.48.0 to 1.49.0 by @dependabot[bot] in > https://github.com/apache/auron/pull/1832 > * [AURON #1841] fix(ci): adapt labeler.yml to action/labeler v6 by @yew1eb > in https://github.com/apache/auron/pull/1842 > * [AURON #1834] Support DataSourceScan metrics by @XorSum in > https://github.com/apache/auron/pull/1835 > * [AURON #1837] ci: Clean up injected JARs to keep Spark bin cache pure by > @yew1eb in https://github.com/apache/auron/pull/1838 > * Bump actions/download-artifact from 4 to 7 by @dependabot[bot] in > https://github.com/apache/auron/pull/1828 > * [AURON #1877] Fix spelling errors via the typos tool by @yew1eb in > https://github.com/apache/auron/pull/1878 > * [AURON #1792][FOLLOWUP] Broadcast isNullAwareAntiJoin flag by @cxzl25 in > https://github.com/apache/auron/pull/1866 > * Bump prost from 0.14.1 to 0.14.3 by @dependabot[bot] in > https://github.com/apache/auron/pull/1884 > * [AURON #1883] Bump maven version to 3.9.12 by @cxzl25 in > https://github.com/apache/auron/pull/1885 > * [AURON #1812][Correctness Testing] Spark 3.3 - Add Aggregate operator > related tests by @ShreyeshArangath in > https://github.com/apache/auron/pull/1819 > * [AURON #1833] Refactor from_proto.rs to planner.rs by @yew1eb in > https://github.com/apache/auron/pull/1843 > * [AURON #1852] Init flink module by @Tartarus0zm in > https://github.com/apache/auron/pull/1887 > * [AURON #1895] Disable auto-close for stale issues by @ShreyeshArangath > in https://github.com/apache/auron/pull/1896 > * [AURON #1898] Fix NULL handling for NOT IN subquery by @yew1eb in > https://github.com/apache/auron/pull/1899 > * [AURON #1624] Support the pushdown of ORC predicates by @dh20 in > https://github.com/apache/auron/pull/1886 > * [AURON #1900] Upgrade Spark 3.5.7 to 3.5.8 by @yew1eb in > https://github.com/apache/auron/pull/1901 > * Bump chrono from 0.4.42 to 0.4.43 by @dependabot[bot] in > https://github.com/apache/auron/pull/1894 > * [AURON #1845] Fix StringSplit conversion by @Flyangz in > https://github.com/apache/auron/pull/1846 > * [AURON #1873] Add unified join BuildSide abstraction for > cross-spark-version compatibility by @yew1eb in > https://github.com/apache/auron/pull/1874 > * [AURON #1502] Implement native function of bround. by @slfan1989 in > https://github.com/apache/auron/pull/1706 > * [AURON #1799][ci] Fix Spark cache key with hadoop-profile by @yew1eb in > https://github.com/apache/auron/pull/1800 > * [AURON #1769][AIP-3]: Introduce auron-it for Enhanced CI Integration > Testing by @yew1eb in https://github.com/apache/auron/pull/1802 > * [AURON #1637] Enforce Clippy lint and Eliminate unwrap by @yew1eb in > https://github.com/apache/auron/pull/1649 > * [AURON #1610] Handle case-insensitive columns in NativeHiveTableScanBase > by @Flyangz in https://github.com/apache/auron/pull/1611 > * [AURON #1693] join operation should flush in time on duplicated keys by > @XorSum in https://github.com/apache/auron/pull/1701 > * [AURON #1904] Fix CI pipeline failure by @yew1eb in > https://github.com/apache/auron/pull/1905 > * [AURON #1867][BUILD] Add scalastyle-maven-plugin by @yew1eb in > https://github.com/apache/auron/pull/1868 > * [AURON #1879] Add maven checkstyle plugin by @yew1eb in > https://github.com/apache/auron/pull/1880 > * [AURON #1907][CI] Fix YAML syntax error and restore TPCDS CI execution > by @yew1eb in https://github.com/apache/auron/pull/1908 > * Bump actions/upload-artifact from 4 to 6 by @dependabot[bot] in > https://github.com/apache/auron/pull/1934 > * [AURON #1911][clippy] Enable -D warnings and resolve related issues by > @yew1eb in https://github.com/apache/auron/pull/1922 > * [AURON #1814][Correctness Testing] Spark 3.3 - Add Sort operator related > tests by @ShreyeshArangath in https://github.com/apache/auron/pull/1820 > * [AURON #1926] ci: Remove opt-level=0 sed command to improve TPC-DS > performance by @yew1eb in https://github.com/apache/auron/pull/1927 > * [AURON #1769] Remove unused legacy files by @yew1eb in > https://github.com/apache/auron/pull/1931 > * [AURON #1937][CI] Skip TPCDS test q14b by @yew1eb in > https://github.com/apache/auron/pull/1939 > * [AURON #1888] Implement spark_partition_id() function by > @ShreyeshArangath in https://github.com/apache/auron/pull/1928 > * [AURON #1944] fix(clippy): fix panic error in spark_partition_id test by > @yew1eb in https://github.com/apache/auron/pull/1945 > * [AURON #1917] Clean up Clippy format/string lints by @lemorage in > https://github.com/apache/auron/pull/1948 > * [AURON #1816][Correctness Testing] Spark 3.3 - Add Parquet related tests > by @ShreyeshArangath in https://github.com/apache/auron/pull/1954 > * [AURON #1937][CI] Enable TPCDS test q14b by @cxzl25 in > https://github.com/apache/auron/pull/1946 > * [AURON #1920] Clean up Clippy condition/logic lints by @lemorage in > https://github.com/apache/auron/pull/1960 > * [AURON #1942] Implement columnar aggregate buffers by @richox in > https://github.com/apache/auron/pull/1943 > * [AURON #1778] Add spark.auron.expression.singleChildFallback.enabled > config by @yew1eb in https://github.com/apache/auron/pull/1779 > * [AURON #1769][CI] Improve auron-it build by @yew1eb in > https://github.com/apache/auron/pull/1932 > * [AURON #1875] Minor refactor: move legacy spark version compatibility > methods to Shims.scala by @yew1eb in > https://github.com/apache/auron/pull/1876 > * [AURON #1739] Support LIMIT with OFFSET by @yew1eb in > https://github.com/apache/auron/pull/1740 > * [AURON #1757] Add cleanup logic to prevent flaky tests caused by > leftover locations by @yew1eb in https://github.com/apache/auron/pull/1758 > * [AURON #1940] Extract Rust test logic to standalone CI workflow by > @yew1eb in https://github.com/apache/auron/pull/1941 > * [AURON #1870][BUILD] Add `-Xfatal-warnings` to scala-maven-plugin and > fix related compilation failures by @yew1eb in > https://github.com/apache/auron/pull/1872 > * [AURON #1963] Refactor SparkAuronConfiguration and remove deprecated > AuronConf classes by @richox in https://github.com/apache/auron/pull/1964 > * [AURON #1817][Correctness Testing] Spark 3.3 - Add Functions related > tests by @ShreyeshArangath in https://github.com/apache/auron/pull/1966 > * [AURON #1956] Add initial compatibility support for Spark 4.1 (UT/CI > Pass) by @yew1eb in https://github.com/apache/auron/pull/1958 > * [AURON #1818][Correctness Testing] Spark 3.3 - Add Expressions related > tests by @ShreyeshArangath in https://github.com/apache/auron/pull/1965 > * [AURON #1961] Fix Spark 4.0+: unit test catalyst codegen failure due to > session artifact isolation by @yew1eb in > https://github.com/apache/auron/pull/1971 > * [AURON #1974] ci: add missing q30 q32 to tpcds queries by @yew1eb in > https://github.com/apache/auron/pull/1973 > * [AURON #1969] Disable auron.udf.singleChildFallback.enabled in unit > tests by @yew1eb in https://github.com/apache/auron/pull/1970 > * [AURON #1889] Implement monotonically_increasing_id() function by > @ShreyeshArangath in https://github.com/apache/auron/pull/1955 > * [AURON #1318][FOLLOWUP] Add test for `common_prefix_len` by @cxzl25 in > https://github.com/apache/auron/pull/1897 > * [AURON #1850] Add FlinkArrowUtils for Flink-Arrow type conversion by > @x-tong in https://github.com/apache/auron/pull/1959 > * [AURON #1988] Fix empty log message in AuronUniffleShuffleReader. by > @slfan1989 in https://github.com/apache/auron/pull/1989 > * [AURON#1991] Improve README formatting and grammar. by @slfan1989 in > https://github.com/apache/auron/pull/1992 > * [AURON #1404] Support for Spark 4.0.2 Compatibility in Auron. by > @slfan1989 in https://github.com/apache/auron/pull/1990 > * fix(bloom_filter): skip 1-byte marker in unfreeze_from_arrays to fix > version mismatch by @BrytonLee in > https://github.com/apache/auron/pull/1979 > * Bump jemalloc_pprof from 0.8.1 to 0.8.2 by @dependabot[bot] in > https://github.com/apache/auron/pull/1998 > * [AURON #1999] Add Apache-2.0 license declaration to all native-engine > Cargo.toml files. by @slfan1989 in > https://github.com/apache/auron/pull/2000 > * [RELEASE] Bump version 7.0.0-incubating by @merrily01 in > https://github.com/apache/auron/pull/2007 > > ## New Contributors > * @guixiaowen made their first contribution in > https://github.com/apache/auron/pull/1345 > * @adityavaish made their first contribution in > https://github.com/apache/auron/pull/1542 > * @ShreyeshArangath made their first contribution in > https://github.com/apache/auron/pull/1598 > * @yew1eb made their first contribution in > https://github.com/apache/auron/pull/1609 > * @CalvinKirs made their first contribution in > https://github.com/apache/auron/pull/1616 > * @xumingming made their first contribution in > https://github.com/apache/auron/pull/1672 > * @dh20 made their first contribution in > https://github.com/apache/auron/pull/1768 > * @lemorage made their first contribution in > https://github.com/apache/auron/pull/1948 > * @x-tong made their first contribution in > https://github.com/apache/auron/pull/1959 > * @BrytonLee made their first contribution in > https://github.com/apache/auron/pull/1979 > > **Full Changelog**: > https://github.com/apache/auron/compare/v6.0.0...v7.0.0 >
