Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/19447
Here the answers to your questions @gatorsmile , please tell me if I need
to elaborate more deeply.
This conf controls how many inner classes are generated. A big value means
that we will have few inner classes and that the outer class and the inner
classes will have a lot of methods. A small one means a lot of classes with few
values. So,
1 - I don't think there is a big perf impact. Maybe, if this value is very
small, we might have a lot nested class instances and therefore a waste of
memory. But since these classes don't have any variable, this is quite
negligible to me.
2 - If the user encounters an Exception like
```
Caused by: org.codehaus.janino.JaninoRuntimeException: Constant pool for
class
org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection$NestedClass
has grown past JVM limit of 0xFFFF
```
then he/she must reduce this value
3 - I don't think so, because I have not been able to reproduce the
problem other than in a very complex use case. So probably that use case is a
corner case which can benefit by this change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
