Github user vijoshi commented on the issue:
    1. display "Last updated <date-time>" with a generic message, possibly 
omitting the count of logs
       Ok, while looking at this I notice that the scan date-time the 
FsHistoryProvider currently internally captures `lastScanTime` is based on the 
filesystem date-time:
      private def getNewLastScanTime(): Long = {
        val fileName = "." + UUID.randomUUID().toString
        val path = new Path(logDir, fileName)
        val fos = fs.create(path)
        try {
    This probably made sense when earlier versions of FsHistoryProvider used 
this value to determine new/updated logs it needed to replay. This is no longer 
the case. In `checkForLogs()` the check is based on file size now. So my 
question is that the fact that `getNewLastScanTime` is still present - is there 
a reason to rely on or use the filesystem time value? We don't get a timezone 
out of the `FileStatus` api.  If I were to display an "Updated as of: " value 
on the app listing page, I would rather read the value from the JVM and ensure 
the Web UI converted it to user's local timezone for display.
    2. Display applications that are pending, instead of a count
    Though agree that this would be better UX-wise, but we would only have the 
filenames (`logInfos`)  to go by to build this list until a clean way to 
extract app metadata exists without parsing event log file contents. If the UI 
just displayed the pending ones is a different table as "names" of pending logs 
without asserting that they are necessarily the app-id (though presently the 
names may match the app-ids) - would that be good enough ? The message on the 
top could link to a new page listing pending logs to avoid cluttering the 
current page?

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to