xiaoyanxie commented on code in PR #2395:
URL: https://github.com/apache/auron/pull/2395#discussion_r3817797329
##########
.github/workflows/tpcds-reusable.yml:
##########
@@ -380,13 +403,30 @@ jobs:
SPARK_HOME: spark-bin-${{ inputs.sparkver }}_${{ inputs.scalaver }}
run: |
ls -la
+ set -o pipefail
dev/auron-it/run-it.sh \
${{ inputs.extrasparkconf }} \
--type tpcds \
--data-location dev/tpcds_1g \
--query-filter ${{ matrix.query }} \
--result-check \
- --plan-check
+ --plan-check 2>&1 | tee tpcds-run-${{ matrix.query }}.log
+
+ # Task-level deserialization failures are absorbed by Spark's task
retries, so the
+ # queries can still report PASS while throwing hundreds of exceptions.
Assert on the
+ # log directly, otherwise a regression of AURON #2386 goes unnoticed.
+ - name: Assert no deserialization ClassCastException
+ if: ${{ inputs.assert-no-classcastexception == 'true' }}
+ env:
+ QUERY_LOG: tpcds-run-${{ matrix.query }}.log
+ run: |
Review Comment:
Agree with both points.
The job did not set queries, so it used the default matrix and ran all
queries. Since q1, q2 and q3 are sufficient to reproduce this problem issue
(the plan contains scalar subqueries that trigger the bug), so I changed it to
`queries: '["q1,q2,q3"]'` and now it uses only one runner.
I also made the pattern more specific. Before it matched any
`ClassCastException`. Now it is
`ClassCastException.*DefaultSerializationProxy`, so other unrelated failures
will not be reported as expression deserialization problem.
The fix is in the commit
[e2dc0b3](https://github.com/apache/auron/pull/2395/changes/e2dc0b308eaa6657a4a40101736a176bbdd12ff8).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
