[
https://issues.apache.org/jira/browse/HDFS-11740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995325#comment-15995325
]
Weiwei Yang commented on HDFS-11740:
------------------------------------
Hi [~anu]
Thanks for your quick response, appreciate. I read your comments, I have some
thoughts.
bq. The original 90 seconds is the time that datanode would use to read the
various containers and make sure that it is ready to communicate with both SCM
and the world.
What I saw the time was spent on state transitions, given the heartbeat time is
30s (default), it moves "slowly" from one state to another which is not
supposed to behave like this.
bq. Right now, we have some missing pieces, we need to launch a background
thread – that does a directory scan for containers and volumes – so that when
SCM asks for container reports we are ready.
Yes this is probably still a missing piece, but it doesn't seem to relate to
this work. If dn is not ready for container reports, it would just response an
error when SCM pulls, so SCM can try again later.
bq. So I am doubtful if we will gain anything by accelerating the initial boot
time.
Say dn needs time to scan containers/volumes and it takes a while, I am
guessing that will be another (new) state. When we add that, it will be much
easier to utilize the work in this jira so that we can control the time
necessary for this task, other than a hard coded heartbeat interval. This time
could be very different on different data size, e.g few seconds on brand new
cluster, minutes on large cluster (I am just saying).
I uploaded a patch, it doesn't have UT and I will add later if the patch is on
the right direction. Some note about the fix, datanode state machine currently
uses {{ScmConfigKeys#OZONE_SCM_HEARTBEAT_INTERVAL_SECONDS}} as the fixed state
interval, it now looks like [^statemachine_1.png]; I am proposing to change to
[^statemachine_2.png]. I added a {{EndpointTask}} interface, all end point
tasks need to implement {{long getTaskDuration()}} to decide how long to wait
before scheduling the task. For {{VersionEndpointTask}} and
{{RegisterEndpointTask}}, they simply return 0 which means no lag and directly
schedule; for {{HeartbeatEndpointTask}}, it returns the heartbeat interval so
it sends heartbeat with respect to that interval.
Hope it helps. Thanks
> Ozone: Differentiate time interval for different DatanodeStateMachine state
> tasks
> ---------------------------------------------------------------------------------
>
> Key: HDFS-11740
> URL: https://issues.apache.org/jira/browse/HDFS-11740
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-11725-HDFS-7240.001.patch, statemachine_1.png,
> statemachine_2.png
>
>
> Currently datanode state machine transitioned between tasks in a fixed time
> interval, defined by {{ScmConfigKeys#OZONE_SCM_HEARTBEAT_INTERVAL_SECONDS}},
> the default value is 30s. Once datanode is started, it will need 90s before
> transited to {{Heartbeat}} state, such a long lag is not necessary. Propose
> to improve the logic of time interval handling, it seems only the heartbeat
> task needs to be scheduled in {{OZONE_SCM_HEARTBEAT_INTERVAL_SECONDS}}
> interval, rest should be done without any lagging.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
