Re: [PR] [SPARK-54338][INFRA] Add pyspark-pandas to coverage run [spark]

Thu, 13 Nov 2025 12:25:55 -0800 


dongjoon-hyun commented on PR #53042:
URL: https://github.com/apache/spark/pull/53042#issuecomment-3529582792

   > Those are all commit triggered CIs which could happen multiple times every 
day. Of course it makes sense to try to do less workflows on those. Coverage 
run only runs once per day and that provides our coverage number. If we don't 
care about a correct coverage number, why do coverage run at all? We can just 
disable that workflow for more resources. A wrong coverage number is just not 
useful.
   > 
   > I think that might be the only normal python3.11 run? If that's the case, 
we are missing some tests on python3.11 + Linux + pandas for our daily CI.
   > 
   > I just want to be crystal clear that I'm talking about this number: <img 
alt="image" width="126" height="25" 
src="https://private-user-images.githubusercontent.com/13121107/514076155-293c67c2-a23e-46a1-b52c-615a7662a682.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjMwNjUzMDgsIm5iZiI6MTc2MzA2NTAwOCwicGF0aCI6Ii8xMzEyMTEwNy81MTQwNzYxNTUtMjkzYzY3YzItYTIzZS00NmExLWI1MmMtNjE1YTc2NjJhNjgyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTExMTMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMTEzVDIwMTY0OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTMxZTgxN2M3NDBlMTNlN2M4MmMzNDdiMjU1MmQ4MWZiNDYzMjc0MDA1ZTM4ZmFlOTBkNmQzM2JhOWMxYmUxZTUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.KCmoIRoAHw-GD6EcxOrksHfUbqeMJ6PQSl9gEJDZw9M";>
   > 
   > and the specific daily run workflow 
https://github.com/apache/spark/blob/master/.github/workflows/build_coverage.yml
 . Almost all the other daily workflows run pandas tests 
(https://github.com/apache/spark/blob/master/.github/workflows/build_python_3.12.yml
 for example), I don't know why this specific coverage run would be an 
exception. It's just so arbitrary.
   
   cc @HyukjinKwon for the above claim.
   
   BTW, for `pandas` CI, we care it much like you and that's the reason why all 
PR builders enforce it as I mentioned already.
   
   <img width="823" height="157" alt="Screenshot 2025-11-13 at 12 18 30" 
src="https://github.com/user-attachments/assets/dd0d12d7-b03b-4484-a3e7-4d1d14e4b837";
 />
   
   - https://github.com/apache/spark/pull/53033
     - https://github.com/zhengruifeng/spark/runs/55260113415
   
   Here is the PR builder example. We are using the contributors' GitHub Action 
resources. Please consider that we have many PRs per day.
   
   <img width="636" height="374" alt="Screenshot 2025-11-13 at 12 23 34" 
src="https://github.com/user-attachments/assets/59021f72-8c3a-426e-b815-7d16909d5ce7";
 />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to