[ 
https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540220#comment-14540220
 ] 

Junping Du commented on YARN-3411:
----------------------------------

Also comments on latest (v5) patch:
{code}
+public class CreateSchema {
{code}
Can we rename it to a more concrete name, something like: TimelineSchemaCreator?

{code}
+  private static int createTimelineEntityTable() {
+    try {
+      Configuration config = HBaseConfiguration.create();
+      // add the hbase configuration details from classpath
+      config.addResource("hbase-site.xml");
+      Connection conn = ConnectionFactory.createConnection(config);
+      Admin admin = conn.getAdmin();
...
{code}
All of these code should be reused by create other tables. May be we should 
move it out of createTimelineEntityTable() and make it as static part of Class?

{code}
+      if (admin.tableExists(table)) {
+        // do not disable / delete existing table
+        // similar to the approach taken by map-reduce jobs when
+        // output directory exists
+        LOG.error("Table " + table.getNameAsString() + " already exists.");
+        return 1;
+      }
{code}
We would like to throw exception here so user can get notified the failed 
reason immediately?

{code}
+      // TTL is 30 days, need to make it configurable perhaps
+      cf3.setTimeToLive(2592000);
{code}
We shouldn't have a hard code value here. At least, add a "TODO" in comment to 
fix it later.

In HBaseTimelineWriterImpl.java,
{code}
+    // TODO right now using a default table name
+    // change later to use a config driven table name
+    entityTableName = TableName
+        .valueOf(EntityTableDetails.DEFAULT_ENTITY_TABLE_NAME);
{code}
Shall we consistent with table name of Pheonix writer if haven't make it 
configurable? Or we intent to do so for some reasons?

{code}
+      if (entityPuts.size() > 0) {
+        LOG.info("Storing " + entityPuts.size() + " to "
+            + this.entityTableName.getNameAsString());
+        entityTable.put(entityPuts);
+      } else {
+        LOG.warn("empty entity object?");
+      }
{code}
The first log should be DEBUG level and wrap with if block of 
"LOG.isDebugEnabled()" which help performance.

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, 
> YARN-3411.poc.2.txt, YARN-3411.poc.3.txt, YARN-3411.poc.4.txt, 
> YARN-3411.poc.5.txt, YARN-3411.poc.6.txt, YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix 
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native 
> HBase schema for the write path. Such a schema does not exclude using 
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in 
> terms of performance, scalability, usability, etc. and make a call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to