[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user Aalnafessah commented on the issue: https://github.com/apache/spark/pull/15803 Hi All I have an issue with the Spark history time zone When I submit a job on a set of machines machine located in London timezone, the Spark Master on the dashboard has the correct time, but the dashboard of history server shows time that is 1 hrs ahead which is GMT. Is there a way to fix this in Apache Spark? Any help ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 ok ,thanks~ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15803 @windpiger given the related JIRA was closed as duplicate to a fixed issue could you close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 It is slways showing UTC in the main page, but with server timezone in other pages like last page I've pasted. Not sure if it's true. Any way we'll waiting for the results from @windpiger , thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 I don't think that's the conclusion here @windpiger -- it would however be nice to correctly show the time zone in all UIs, whatever it is currently. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 I will check other UIs ,and fix them to the UTC --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 To be clear, the history server already shows UTC always right? this change depends on that being true. Yes, I think it's not true of the other UIs. We already have this inconsistency. Fixing the header of other UIs would be great too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 It's always better to show a timezone in table header, i think, no matter what the timezone it really uses. But changing to show GMT/UTC always? I have to say it's a bold move even there's a case that many drivers run at different timezones. If we're sure to do this, we need to make it known to users in form of like writing it into release notes, as UI is the most frequently-used interface. Current fix makes some sense, but easily makes people confused: when they check applications list, the time says "The time zone is GMT, and the time is 2016-01-01 15:00", but when they check details of applications they get "The Job 1 of this application starts at 2016-01-01 14:00 or 2016-01-01 12:30". This kind of inconsistance is unacceptable from my point. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 OK, I suppose there is some logic to showing GMT/UTC always in the history server; it shows a history over jobs from many drivers and so doesn't as much make sense to be specific to the timezone of any server. I still disagree with 1, but that's separate. Anyway. I suppose this change still makes sense. The time is always GMT? I realize it's not clear to me, but was assuming this was true based on the JSON tests. I suppose it would be better if the other UIs showed a timezone too, but that's a separate issue too I guess. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 I would like solution 1, as other time string in Spark UI shows, like JobPage:  --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 there are three things one could do with a date: 1. Display in server-side timezone 2. Display in client-side timezone 3. Display in UTC/GMT always The original change went from 3 to 2. Other UIs appear to do a mix of 1 and 2 (?). We have a mix of opinions about what the right thing to do is. I was suggesting leaving this doing 3, but display the timezone. I'm still sort of concerned that this doesn't make it any more consistent though with other UIs, and, I'm wondering if I'm just wrong about it? or do in fact the other app UIs do something different? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 we need the timezone of the server side, or we just use epoch time to converted to GMT readable time(then show the timezone of the serverside dependent on the page which is configured by the spark conf) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 Do we have a guy who's good at JAX-RS? maybe he can explain the theory and help us to understand better :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 @srowen Before the code changes, browser get date string from server side, now instead it get Date(this conclusion comes from codes debugging(https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/ApiRootResource.scala#L46), I'm not sure personally, please correct me if I'm wrong), and parse its string format(`hacks a date string to drop seconds and timezone`). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 We are talking about the history server's human-readable API, right? the code being changed does not deal with JSON Dates, but hacks a date string to drop seconds and timezone. My point was that it doesn't quite match how dates are rendered in other services like the application web UI. It looks like dates may have different timezones. It's a different problem, but related. I suppose this is a good step forward, because it is at least clearer, even if it doesn't make the behavior consistent, right? but are we sure the dates are always GMT? it seems like it from the tests but I want to be sure we're not missing something. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 @ajbozarth In browser side the timezone used to build Date from epoch time is the one **at browser side**, not that one in **History Server side**. These two are different in many cases. So yes, it is harder. :( --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15803 The json also contains the epoch time as an int, can't we use that to build a local time in the browser, or is that harder than I imagine? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 I think the problem is that Date tranfered in REST ways take no timezone, one possible reason is : http://stackoverflow.com/questions/23730062/use-iso-8601-dates-in-jax-rs-responses. I'm not a expert in REST server or jax-rs framework, so I got another solution: transferring Date using its string format rather than Date object itself. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803  Like what showed in image, user can get app infos in two ways: using browser and using REST api. When a browser send a http request, History Server will return history page(a html page) containing application infos in which the Date is translated to string. Because the translation is done in server side, so the date string is represented with server timezone. If user uses REST api to get app infos, History Server will return json objects containing Date object w/o timezone, so the REST response always shows date like ` "startTime" : "2016-11-11T06:27:25.802GMT"`, which means in GMT time(as no timezone available). After 2.0, one way accessing History Server changed: browser ways(the red highlighten part). Browser will get app infos using REST api rather than getting whole history page (in order to generate table in front end to implementing sort and search). The result it got only containing Date w/o timezone instead of date string translated by Server using server timezone. The way browser get app datas changed, that's why the browser behavior changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 I'll post how UI works and what changes it did to be different before later :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 Hm, so I looked at how the other UIs work, and they seem to not be in GMT always. They happen to use the machines' default time zone by way of using a `SimpleDateFormat` to render times. So it has the same problem. If we're sure that the history server always uses GMT, then this is a good change although it just highlights the inconsistency. Does it always show GMT? right now all I can see is that it throws away the timezone. Although I'd like to change all UIs and outputs everywhere to use UTC/GMT that might be something people disagree on. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15803 ah, ok I did misunderstand, I thought you were talking changing timezone based on browser local time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 I tend to agree with the view that all times should be UTC on the server side, and perhaps even client side for consistency. This is maybe a separate discussion though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 Nah i think we have a misunderstand here. @tgravescs If understand right, what you mean is that most companies run their server in UTC timezone. That's OK. Under that condition, History Server UI should show date using UTC time. But for servers which runs in other timezone, History Server UI should show date using same timezone as the backend server uses. Currently History Server UI shows date in GMT what ever timezone the server runs at(after the big change: https://github.com/apache/spark/commit/e4c1162b6b3dbc8fc95cfe75c6e0bc2915575fb2). I think that's the point. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 Right now, the times are already all in GMT. It would be at least less change to leave it that way. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15803 The timezone is going to vary by setup. Normally I would expect any large company that covers more then one timezone to run machines in UTC to consistency and auditing purposes since they could have machines across timezones. I would also expect most cloud services, aws, etc to run UTC by default but its been a while since I used those so maybe I'm mistaken. I also see UTC much more standard. If this is a big issue for enough people then having it configurable makes sense. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 I agree with showing the timezone with date string. But always using GMT/UTC time is not a good choice, logs of application(using log4j) usually are printed using local timezone(like CST). That means if I wanna check what happens using application logs with EventLog, I must do the translation between them manually. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 @srowen I have modify it by put GMT on the column name to tell user the time showed in the list is GMT time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15803 Have you looked into using a JS Date object? I don't have much experience with it but it may solve this issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 This makes it harder to read though, because it undoes the slightly hacky 'parsing' of the ISO 8601 string to make it a bit more readable, taking out the partial seconds and timezone and 'T'. What about just writing "GMT" in the column name? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 @srowen @tgravescs I aggree with you that, we should explicit show the timezone, so I modify the pr simply that it show the GMT time, such as the picture in the pr description. But I think we could consider if it can show GMT+8 or GMT+5 etc, not only the GMT time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15803 I agree with @srowen we should specify the time zone. Personally I prefer this to stay UTC because its more standard and because that way the times on UI can be easily used to correlate times in logs, which generally are UTC, although that could be setup specific. Its also easier when talking about a job with people in different time zones. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 Aha,I got it ~ Thanks~ Add the timezone info after the time showed to the user, is it ok? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 @srowen If show the same GMT time, I think the user maybe more ambiguous, or the user should translate it to their local time in their mind when they see the GMT, which is not nicely. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 I'm not disagreeing with that. I am saying that whatever the timezone is, it should be displayed. That's not reflected in this change right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15803 (FWIW, this reminds me of [SPARK-18350](https://issues.apache.org/jira/browse/SPARK-18350)) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 (Timestamps don't even have a timezone.) Of course, time should be stored as a timestamp. I am saying something else, that when rendered as human-readable time, it needs to render a timezone explicitly. "2016-10-01 20:10:03" is ambiguous. I tend to want to always display in GMT so that times are easily correlated with things like system logs which are often in GMT. Whatever the timezone is, it should be explicit when displayed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 @srowen UTC timestamp(long) is the standard time, no matter it is translated to which timezone, the long value is the same. But the readable date string should be nicely adapter to the user's browser's local timezone, for example, I run the spark job in Cluster which is located in UTC+8(utc timestamp is ts)ï¼and the user A view the historyserver in UTC+7, the brower of A should translate ts to UTC+7's date string, while the user B view in UTC+5, the brower of B should translate ts to UTC+5's data string. so I think UTC/GMT timestamp should keep in spark system, and the browser is responsible for the translate from UTC to local timezone. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15803 General comment: we should always render times with a timezone for clarity, even if it's in a column header. This has been the source of so many problems I've seen over the years. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 @ajbozarth could you help to review this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 UTC timestamp can translate to different zone's local time,which the user see from the history ui. It is reasonable to return UTC/GMT time to the user browser, and show it by translate to the local zone time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 thanks for the fix. This patch parse the timestamp instead of the Date String returned. The REST api still return the GMT time, which is insistent with UI showing. I've googled and found one possible reason: http://stackoverflow.com/questions/23730062/use-iso-8601-dates-in-jax-rs-responses, which is caused by (de)seriallization. Currently, i think there're two solutions for this: 1. use this patch, let UI show GMT time. If user wanna translate it to other timezone, they can use epoch like: "startTimeEpoch" etc. 2. We transfer the Date with its timezone and get time in local no matter we use UI or REST api. How do you think guys? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15803 If I remember correctly, does not UI change require a screenshot in the PR description? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15803 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
