Re: is there a way to map Tez UI graphs back to the script?
Correct. It does not log per vertex information to be used in the UI. However, it does log the full query text to show the query that is being processed by the given Dag. TEZ-2453 is currently open for making the required changes in the UI to show the hive query text on the dag details page. thanks — Hitesh On May 16, 2015, at 2:15 AM, Xiaoyong Zhu wrote: > I see, thanks Hitesh! > > That being said - currently Hive does not use ProcessorDescriptor to log the > query info, right? > > Xiaoyong > > -Original Message- > From: Hitesh Shah [mailto:hit...@apache.org] > Sent: Saturday, May 16, 2015 12:23 AM > To: user@tez.apache.org > Subject: Re: is there a way to map Tez UI graphs back to the script? > > This will require a change in Hive/Pig and a modification to the Tez UI. > > For every vertex, there is a ProcessorDescriptor. This supports a > setHistoryText API. > > /** > * Provide a human-readable version of the user payload that can be > * used in the TEZ UI > * @param historyText History text > * For better support in the UI, the history text should be a json-encoded > string. > * The following keys in the json object will be recognized: > *"desc" : A string-value describing the entity > *"config" : A key-value map to represent configuration > * @return this object for further chained method calls */ public T > setHistoryText(String historyText) > > The "desc" from above can be used to store the list of operators, query > fragment being processed in the particular vertex. > > To start with, you can file a jira for Hive to add this information as well > file a jira for this to be displayed in the Tez UI. Also, if you are looking > to learn about either Hive or Tez, these jiras might be a good starting point > for your first patch contribution. > > If you are looking to do this manually, you can run an explain on your query > which should set some light on what processing is being done in which vertex. > For more details on this, you should ask on the hive user list. > > thanks > - Hitesh > > > > > On May 15, 2015, at 4:41 AM, Jianfeng (Jeff) Zhang > wrote: > >> >> I believe there's one jira that try to add context info on the vertex, so >> that you can see what the operator that vertex is mapped to. >> >> >> >> >> Best Regard, >> Jeff Zhang >> >> >> From: Xiaoyong Zhu >> Reply-To: "user@tez.apache.org" >> Date: Friday, May 15, 2015 at 5:50 PM >> To: "user@tez.apache.org" >> Subject: is there a way to map Tez UI graphs back to the script? >> >> Not sure if there is a way to map the TezUI graph view back to the script? >> For example, below should be some JOIN operator - is there a way to >> map it back to the script to see which is the corresponding part? >> (attached is the script) >> >> Thanks! >> >> Xiaoyong >> >> >
RE: is there a way to map Tez UI graphs back to the script?
I see, thanks Hitesh! That being said - currently Hive does not use ProcessorDescriptor to log the query info, right? Xiaoyong -Original Message- From: Hitesh Shah [mailto:hit...@apache.org] Sent: Saturday, May 16, 2015 12:23 AM To: user@tez.apache.org Subject: Re: is there a way to map Tez UI graphs back to the script? This will require a change in Hive/Pig and a modification to the Tez UI. For every vertex, there is a ProcessorDescriptor. This supports a setHistoryText API. /** * Provide a human-readable version of the user payload that can be * used in the TEZ UI * @param historyText History text * For better support in the UI, the history text should be a json-encoded string. * The following keys in the json object will be recognized: *"desc" : A string-value describing the entity *"config" : A key-value map to represent configuration * @return this object for further chained method calls */ public T setHistoryText(String historyText) The "desc" from above can be used to store the list of operators, query fragment being processed in the particular vertex. To start with, you can file a jira for Hive to add this information as well file a jira for this to be displayed in the Tez UI. Also, if you are looking to learn about either Hive or Tez, these jiras might be a good starting point for your first patch contribution. If you are looking to do this manually, you can run an explain on your query which should set some light on what processing is being done in which vertex. For more details on this, you should ask on the hive user list. thanks - Hitesh On May 15, 2015, at 4:41 AM, Jianfeng (Jeff) Zhang wrote: > > I believe there's one jira that try to add context info on the vertex, so > that you can see what the operator that vertex is mapped to. > > > > > Best Regard, > Jeff Zhang > > > From: Xiaoyong Zhu > Reply-To: "user@tez.apache.org" > Date: Friday, May 15, 2015 at 5:50 PM > To: "user@tez.apache.org" > Subject: is there a way to map Tez UI graphs back to the script? > > Not sure if there is a way to map the TezUI graph view back to the script? > For example, below should be some JOIN operator - is there a way to > map it back to the script to see which is the corresponding part? > (attached is the script) > > Thanks! > > Xiaoyong > >
Re: is there a way to map Tez UI graphs back to the script?
This will require a change in Hive/Pig and a modification to the Tez UI. For every vertex, there is a ProcessorDescriptor. This supports a setHistoryText API. /** * Provide a human-readable version of the user payload that can be * used in the TEZ UI * @param historyText History text * For better support in the UI, the history text should be a json-encoded string. * The following keys in the json object will be recognized: *"desc" : A string-value describing the entity *"config" : A key-value map to represent configuration * @return this object for further chained method calls */ public T setHistoryText(String historyText) The “desc” from above can be used to store the list of operators, query fragment being processed in the particular vertex. To start with, you can file a jira for Hive to add this information as well file a jira for this to be displayed in the Tez UI. Also, if you are looking to learn about either Hive or Tez, these jiras might be a good starting point for your first patch contribution. If you are looking to do this manually, you can run an explain on your query which should set some light on what processing is being done in which vertex. For more details on this, you should ask on the hive user list. thanks — Hitesh On May 15, 2015, at 4:41 AM, Jianfeng (Jeff) Zhang wrote: > > I believe there’s one jira that try to add context info on the vertex, so > that you can see what the operator that vertex is mapped to. > > > > > Best Regard, > Jeff Zhang > > > From: Xiaoyong Zhu > Reply-To: "user@tez.apache.org" > Date: Friday, May 15, 2015 at 5:50 PM > To: "user@tez.apache.org" > Subject: is there a way to map Tez UI graphs back to the script? > > Not sure if there is a way to map the TezUI graph view back to the script? > For example, below should be some JOIN operator – is there a way to map it > back to the script to see which is the corresponding part? (attached is the > script) > > > Thanks! > > Xiaoyong > >
Re: is there a way to map Tez UI graphs back to the script?
I believe there's one jira that try to add context info on the vertex, so that you can see what the operator that vertex is mapped to. Best Regard, Jeff Zhang From: Xiaoyong Zhu mailto:xiaoy...@microsoft.com>> Reply-To: "user@tez.apache.org<mailto:user@tez.apache.org>" mailto:user@tez.apache.org>> Date: Friday, May 15, 2015 at 5:50 PM To: "user@tez.apache.org<mailto:user@tez.apache.org>" mailto:user@tez.apache.org>> Subject: is there a way to map Tez UI graphs back to the script? Not sure if there is a way to map the TezUI graph view back to the script? For example, below should be some JOIN operator - is there a way to map it back to the script to see which is the corresponding part? (attached is the script) [cid:image001.png@01D08F37.9EC72950] Thanks! Xiaoyong
is there a way to map Tez UI graphs back to the script?
Not sure if there is a way to map the TezUI graph view back to the script? For example, below should be some JOIN operator - is there a way to map it back to the script to see which is the corresponding part? (attached is the script) [cid:image001.png@01D08F37.9EC72950] Thanks! Xiaoyong job_1431332645018_0038.hql Description: job_1431332645018_0038.hql
