Sangjin Lee commented on YARN-3906:

bq. I've also noticed that the newly added Application*.java files overlap 
significantly with Entity*.java.

Thanks for bringing up that point [~gtCarrera9]. I should have added some 
explanations on why I wrote it this way. That is the first thing I noticed as I 
looked into adding the new table.

\*Table and \*RowKey are not so bad, but \*ColumnFamily, \*Column, and 
\*ColumnPrefix have definitely a lot of overlapping code. That is largely an 
artifact of the design decision to use enums to implement these classes. Enums 
are nice because it lets us seal the list of members cleanly, and the code that 
uses the API becomes very strongly typed. On the other hand, the downside is 
that enums cannot be extended.

If enums could be extended, we could have created a base class that's common 
both for the entity table and the application table, and have the entity table 
and the application table extend it pretty trivially. But unfortunately it 
doesn't work with enums. Nor does Java have an option of mix-ins like scala.

As a way to minimize the duplication, we introduced {{ColumnHelper}} to provide 
many of the common operations into that helper class. You'll notice that most 
of the implementations in the \*Column\* classes are simple pass-through to 

This issue is more pronounced because the entity table and the application 
table are so similar. For example, for the app-to-flow table (which Zhijie is 
working on), this might not be as big an issue.

We could think of some alternatives, but I think they also have their own 
challenges. First, we could think of having only one set of classes both for 
the entity table and the application table, and controlling which one to use 
via some sort of an argument/flag. But then the problem is that we would have 
lots of {{if application ... else ...}} code scattered around in that single 
implementation. I'm not sure if it is an improvement.

Eventually, if this becomes more of a need, we could envision writing some sort 
of code generation and the table/schema description instruction so that given 
the schema description these classes can be simply code-generated. However, as 
you may know, code generation is not without problems...

I hope this clarifies some of the thinking that went into this.

> split the application table from the entity table
> -------------------------------------------------
>                 Key: YARN-3906
>                 URL: https://issues.apache.org/jira/browse/YARN-3906
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>         Attachments: YARN-3906-YARN-2928.001.patch, 
> YARN-3906-YARN-2928.002.patch
> Per discussions on YARN-3815, we need to split the application entities from 
> the main entity table into its own table (application).

This message was sent by Atlassian JIRA

Reply via email to