[ 
https://issues.apache.org/jira/browse/ARROW-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156598#comment-16156598
 ] 

Gonzalo Ortiz commented on ARROW-1463:
--------------------------------------

On the one hand, what [~wesmckinn] is right. They could be the same structure. 
On the other, they are not exactly semantically equal. Non nullable structures 
doesn't return nulls, which can be used by some static checkers or languages 
like Kotlin. 

There are also two performance issues, one on each model. They are just a 
theoretical reasoning (that may be wrong!), we would need to verify that some 
JMH test (and it can change from one JVM to another):
* A structure that doesn't need to check the nullable array would have shorter 
and simpler methods (without branches) and therefore it is more probable that 
the JVM can optimize that. So this is a point on having two implementations: 
one with nulls and one without them.
* If there are two structures (a nullable and a non nullable) and the nullable 
is just a decoration/delegation on the non-nullable, nullable structures may 
behave quite worst than we expect for the same reason. As soon as the methods 
get complex (calling the delegate class) the JVM may or may not optimize them.

TL;RD: From the library developer perspective, I would like to have only one 
implementation. From a user perspective, I would like to have both 
implementations, ideally without delegation (which means repeated code on 
nullable-structure)

> [JAVA] Restructure ValueVector hierarchy to minimize compile-time generated 
> code
> --------------------------------------------------------------------------------
>
>                 Key: ARROW-1463
>                 URL: https://issues.apache.org/jira/browse/ARROW-1463
>             Project: Apache Arrow
>          Issue Type: Improvement
>            Reporter: Jacques Nadeau
>            Assignee: SIDDHARTH TEOTIA
>
> The templates used in the java package are very high mainteance and the if 
> conditions are hard to track. As started in the discussion here: 
> https://github.com/apache/arrow/pull/1012, I'd like to propose that we modify 
> the structure of the internal value vectors and code generation dynamics.
> Create new abstract base vectors:
> BaseFixedVector
> BaseVariableVector
> BaseNullableVector
> For each of these, implement all the basic functionality of a vector without 
> using templating.
> Evaluate whether to use code generation to generate specific specializations 
> of this functionality for each type where needed for performance purposes 
> (probably constrained to mutator and accessor set/get methods). Giant and 
> complex if conditions in the templates are actually worse from my perspective 
> than a small amount of hand written duplicated code since templates are much 
> harder to work with. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to