[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12689594#action_12689594
]
Ashish Thusoo commented on HIVE-278:
+1
Looks good to me.
Will run the tests and checkin once the tests run clean.
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.3.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.3.0
>
> Attachments: HIVE-278-1.diff, HIVE-278-2.diff, HIVE-278.diff,
> hive_history.png, session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683704#action_12683704
]
Ashish Thusoo commented on HIVE-278:
Thanks for the explanation.
This looks reasonable and cleaner than before. Taking out the dead code is also
fine with me. The changes look good except some indentations which seem to be
out of whack. The logic looks fine to me.
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.3.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.3.0
>
> Attachments: HIVE-278-1.diff, HIVE-278.diff, hive_history.png,
> session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683266#action_12683266
]
Edward Capriolo commented on HIVE-278:
--
1. Why remove the getJobTrackerURI() and getKillCommand() functions?
These were never reachable from the web interface. I notice that the
HiveHistory is written structured but reading it results in string value pairs.
The orignal goal of this method was to return a URL to this job on the
jobtracker. Now its seems like there are multiple URLs the completed and the
current. This method might be misnamed. Should we keep them in and
unimplemented?
I can explain the answer to 2 and 3 in a use case of the old and the new.
Use Case 1 OLD FLOW: User starts a session and runs a query.
1) HWISessionManager.createSession is sent the HWIAuth (username and password)
as well as a session name.
2) HWISessionManager creates an HWISessionItem.
3) HWISessionManager then needed to call itemInit() for step 4
4) HWISessionManager then used the si.setProcessorQuery() to set the auth
information
User now sets uses the web interface to set the query up.
Finally the user sets calls clientStart() the session items status is set to
QUERY_SET
5) The HWISessionManager is a thread that wakes up and iterates a List looking
for QUERY_SET
6) if QUERY_SET call HWISessionItem.startIt()
7) startIt() created a thread
8) the thread ran the query to completion then ended. Basically a one shot
thread.
The problem with the Hive History highlighted this one shot thread problem. The
nature of servlet threading posed its challenges. HWISessionManager and
HWISessionItem posed its own set of challenges.
The troubleshooting you guys did concluded that some code would have to be
moved around. After looking at the code I decided that steps 3 and 4 could be
moved to HWISessionItem. Once steps 3 and 4 are moved HWISessionManager, you
realize steps 5 6 7 are a two thread interaction and it can be done with one.
Use Case 2 NEW FLOW: User starts a session and runs a query
1) HWISessionManager.createSession is sent the HWIAuth (username and password)
as well as a session name.
2) HWIAuth is passed to HWISessionItem HWISessionItem.status=NEW
3) HWISessionItem enters its run()
User now sets uses the web interface to set the query up.
4) HWISessionItem.status changes to QUERY_SET. notify() is called
5) Thread will now wake up
6) conf will be null so itemInit() will be called
7) the status is QUERY_SET this.runQuery() is called status is now QUERY_RUNNING
8) The query completes the status is now QUERY_COMPLETE
Moving more code to HWISessionItem cuts down on the back and forth between the
two classes.
So the logic of run()
if (status = DESTROY)
fall out of the thread loop.
if (status = QUERY_SET)
start the query
if (conf = null)
itemInit()
else wait() --user notify() will eventually wake this thread up
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.3.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.3.0
>
> Attachments: HIVE-278-1.diff, HIVE-278.diff, hive_history.png,
> session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683221#action_12683221
]
Ashish Thusoo commented on HIVE-278:
I had a few questions/clarifications:
1. Why remove the getJobTrackerURI() and getKillCommand() functions?
2. As I understand, you now create a thread in the constructor itself and you
reuse that thread later when the actual request is being handled. Is that how
you get around the problem of the thread specific state?
3. Also can you explain how the run() function works.
Thanks,
Ashish
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.3.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.3.0
>
> Attachments: HIVE-278-1.diff, HIVE-278.diff, hive_history.png,
> session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683116#action_12683116
]
Ashish Thusoo commented on HIVE-278:
Sorry for the delay. I am looking at this now and will have some comments over
to you today. Thanks.
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.3.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.3.0
>
> Attachments: HIVE-278-1.diff, HIVE-278.diff, hive_history.png,
> session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678539#action_12678539
]
Edward Capriolo commented on HIVE-278:
--
I understand what you have described. As far as I know page requests in the
servlet engine can possibly reuse a thread. What the code is doing is
initializing a session in the HWISessionManager thread but then starting the
query from a thread inside HWISessionItem.
Currently there is a CLISessionState as a private member of HWISessionItem.
I think the complication of my design is that the HWISessionItem class should
extend SessionState and implement Runnable.
Hopefully this can be done without modified the HWISessionManager .
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.3.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.3.0
>
> Attachments: session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672744#action_12672744
]
Ashish Thusoo commented on HIVE-278:
Looked at this with Suresh. We feel that the Session object is being created in
the SessionManager where as the Driver is created in the SessionItem thread. As
a result the Driver looks at it thread specific state and does not find a
Session there and therefore does not put anything in the log. Suresh pointed
this out and can chime in to clarify this.
But basically we would have to create the session object and the driver in the
same thread. Maybe moving the Driver creation to the SessionManager thread will
fix this.
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.2.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.2.0
>
> Attachments: session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672117#action_12672117
]
Edward Capriolo commented on HIVE-278:
--
I did try to follow the test case. The issue I am having is when running from
the cli the history works fine. Running the same query from From HWI I only get
the first line out output.
{noformat}
more
/vservers/hadoop1/tmp/hadoop/hive_job_log_hadoop_200902081105_-695057565.txt
SessionStart SESSION_ID="hadoop_200902081105" TIME="1234109159064"
{noformat}
HWISessionItem.init() seems to emulate the test case.
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.2.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.2.0
>
> Attachments: session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-278) Add HiveHistory to Hive web interface
[
https://issues.apache.org/jira/browse/HIVE-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672056#action_12672056
]
Ashish Thusoo commented on HIVE-278:
Just follow the example in the test
TestHiveHistory.java
testSimpleQuery()
what problem are you hitting with your code?
> Add HiveHistory to Hive web interface
> -
>
> Key: HIVE-278
> URL: https://issues.apache.org/jira/browse/HIVE-278
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Logging, Web UI
>Affects Versions: 0.2.0
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
> Fix For: 0.2.0
>
> Attachments: session_logging.diff
>
>
> In order for HIVE-176 to be utilized by the Hive web interface a few changes
> need to be made.
> * HWISessionItem needs a method with an argument signature
> {noformat}
> public HiveHistoryViewer getHistoryViewer() throws HWIException
> {noformat}
> * session_manage.jsp needs an addition
> {noformat}
> Hive History: href=/hwi/session_history.jsp?sessionName=<%=sessionName%>"><%=sessionName%>
> {noformat}
> * session_history.jsp will have to be created to use the ql.history api
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
