Github user nishkamravi2 commented on the pull request:
https://github.com/apache/spark/pull/5085#issuecomment-84137316
That's not a 30% gain. That's a constant gain of 0.05 seconds.
> What backwards compatibility concerns do you have? The library didn't
exist before, so it's pretty hard to think of any.
The library didn't exist before and now it's required. That's a backward
compatibility issue.
> As for performance, I didn't measure explicitly, but even if the gain is
not that big, I really don't see why you see this as complicating anything.
Not sure what it brings to the table then?
> The issue with the breakage is not that it can be easily fixed. Is that
it would exist at all. You'd be forcing people to fix their build when they did
absolutely nothing wrong. What I haven't seen here is any argument that
supports relaxing the regex without using some use case where the user is
intentionally breaking the Spark distribution.
Your arguments may be considered somewhat contradictory to idea of this
launcher library which is to provide additional flexibility in deploying Spark.
For every customer who wants to launch Spark programmatically (which itself is
a corner case), there is one who has a custom Spark deployment setup.
Besides, as I said, if you are convinced that strict checking of assembly
file name is a good idea (which is very subjective), it is better to do that in
a shell script than hardcode it in Spark code that gets compiled.
I think it may be best to weigh in opinion from others at this point
@srowen @pwendell @andrewor14
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
