Re: Node unable to start vertex

2016-08-26 Thread Madhusudan Ramanna 
Have added an entry
https://cwiki.apache.org/confluence/display/TEZ/FAQ
 

On Thursday, August 25, 2016 1:36 PM, Hitesh Shah  wrote:
 

 Created https://cwiki.apache.org/confluence/display/TEZ/FAQ which might be a 
better fit for such content and other related questions down the line. 

> On Aug 25, 2016, at 1:16 PM, Hitesh Shah  wrote:
> 
> +1. Would you like to contribute the content? You should be able to add an 
> article under 
> https://cwiki.apache.org/confluence/display/TEZ/Troubleshooting+articles.
> 
> If you hit any permission issues, feel free to reply back with your 
> confluence id. 
> 
> thanks
> — Hitesh 
> 
> 
>> On Aug 25, 2016, at 12:59 PM, Madhusudan Ramanna  wrote:
>> 
>> Thanks, #2 worked !
>> 
>> Might be a good idea to add to confluence ?
>> 
>> Madhu
>> 
>> 
>> On Thursday, August 25, 2016 12:00 PM, Hitesh Shah  wrote:
>> 
>> 
>> Hello Madhu, 
>> 
>> There are 2 approaches for this:
>> 
>> 1) Programmatically, for user code running in tasks, you would need to use 
>> either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if 
>> the same jars are needed in all tasks of the DAG.  
>> TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster. 
>> 
>> 2) Configure tez.aux.uris. This will ensure that all files specified here 
>> will be available in the AM and all tasks.  
>> 
>> thanks
>> — Hitesh
>> 
>>> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna  
>>> wrote:
>>> 
>>> Hello,
>>> 
>>> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo 
>>> cluster mode on my host).
>>> 
>>> For some reason, I'm running into class not found exception on the node
>>> 
>>> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, 
>>> diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_00, 
>>> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
>>> failure ) : 
>>> attempt_1471907702278_0030_1_00_00_0:org.apache.tez.dag.api.TezReflectionException:
>>>  Unable to load class: sample.sampletez.OnetoOne$V1Processor
>>> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
>>> at 
>>> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
>>> at 
>>> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
>>> 
>>> 
>>> Should I be adding my application jar somewhere so that it can get 
>>> distributed ? I tried adding my jar via  tezClient.addAppMasterLocalFiles() 
>>> but it didn't help.
>>> 
>>> What am I not doing ? 
>>> 
>>> thanks!
>>> Madhu
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>

Re: Node unable to start vertex

2016-08-25 Thread Hitesh Shah 
Created https://cwiki.apache.org/confluence/display/TEZ/FAQ which might be a 
better fit for such content and other related questions down the line. 

> On Aug 25, 2016, at 1:16 PM, Hitesh Shah  wrote:
> 
> +1. Would you like to contribute the content? You should be able to add an 
> article under 
> https://cwiki.apache.org/confluence/display/TEZ/Troubleshooting+articles.
> 
> If you hit any permission issues, feel free to reply back with your 
> confluence id. 
> 
> thanks
> — Hitesh 
> 
> 
>> On Aug 25, 2016, at 12:59 PM, Madhusudan Ramanna  wrote:
>> 
>> Thanks, #2 worked !
>> 
>> Might be a good idea to add to confluence ?
>> 
>> Madhu
>> 
>> 
>> On Thursday, August 25, 2016 12:00 PM, Hitesh Shah  wrote:
>> 
>> 
>> Hello Madhu, 
>> 
>> There are 2 approaches for this:
>> 
>> 1) Programmatically, for user code running in tasks, you would need to use 
>> either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if 
>> the same jars are needed in all tasks of the DAG.  
>> TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster. 
>> 
>> 2) Configure tez.aux.uris. This will ensure that all files specified here 
>> will be available in the AM and all tasks.  
>> 
>> thanks
>> — Hitesh
>> 
>>> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna  
>>> wrote:
>>> 
>>> Hello,
>>> 
>>> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo 
>>> cluster mode on my host).
>>> 
>>> For some reason, I'm running into class not found exception on the node
>>> 
>>> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, 
>>> diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_00, 
>>> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
>>> failure ) : 
>>> attempt_1471907702278_0030_1_00_00_0:org.apache.tez.dag.api.TezReflectionException:
>>>  Unable to load class: sample.sampletez.OnetoOne$V1Processor
>>> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
>>> at 
>>> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
>>> at 
>>> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
>>> 
>>> 
>>> Should I be adding my application jar somewhere so that it can get 
>>> distributed ? I tried adding my jar via  tezClient.addAppMasterLocalFiles() 
>>> but it didn't help.
>>> 
>>> What am I not doing ? 
>>> 
>>> thanks!
>>> Madhu
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>

Re: Node unable to start vertex

2016-08-25 Thread Hitesh Shah 
+1. Would you like to contribute the content? You should be able to add an 
article under 
https://cwiki.apache.org/confluence/display/TEZ/Troubleshooting+articles.

If you hit any permission issues, feel free to reply back with your confluence 
id. 

thanks
— Hitesh 


> On Aug 25, 2016, at 12:59 PM, Madhusudan Ramanna  wrote:
> 
> Thanks, #2 worked !
> 
> Might be a good idea to add to confluence ?
> 
> Madhu
> 
> 
> On Thursday, August 25, 2016 12:00 PM, Hitesh Shah  wrote:
> 
> 
> Hello Madhu, 
> 
> There are 2 approaches for this:
> 
> 1) Programmatically, for user code running in tasks, you would need to use 
> either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if 
> the same jars are needed in all tasks of the DAG.  
> TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster. 
> 
> 2) Configure tez.aux.uris. This will ensure that all files specified here 
> will be available in the AM and all tasks.  
> 
> thanks
> — Hitesh
> 
> > On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna  
> > wrote:
> > 
> > Hello,
> > 
> > I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo 
> > cluster mode on my host).
> > 
> > For some reason, I'm running into class not found exception on the node
> > 
> > Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, 
> > diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_00, 
> > diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
> > failure ) : 
> > attempt_1471907702278_0030_1_00_00_0:org.apache.tez.dag.api.TezReflectionException:
> >  Unable to load class: sample.sampletez.OnetoOne$V1Processor
> > at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
> > at 
> > org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
> > at 
> > org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
> > 
> > 
> > Should I be adding my application jar somewhere so that it can get 
> > distributed ? I tried adding my jar via  tezClient.addAppMasterLocalFiles() 
> > but it didn't help.
> > 
> > What am I not doing ? 
> > 
> > thanks!
> > Madhu
> > 
> > 
> > 
> > 
> > 
> 
>

Re: Node unable to start vertex

2016-08-25 Thread Madhusudan Ramanna 
Thanks, #2 worked !
Might be a good idea to add to confluence ?
Madhu 

On Thursday, August 25, 2016 12:00 PM, Hitesh Shah  
wrote:
 

 Hello Madhu, 

There are 2 approaches for this:

1) Programmatically, for user code running in tasks, you would need to use 
either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the 
same jars are needed in all tasks of the DAG.  
TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster. 

2) Configure tez.aux.uris. This will ensure that all files specified here will 
be available in the AM and all tasks.  

thanks
— Hitesh

> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna  wrote:
> 
> Hello,
> 
> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo 
> cluster mode on my host).
> 
> For some reason, I'm running into class not found exception on the node
> 
> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, 
> diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_00, 
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
> failure ) : 
> attempt_1471907702278_0030_1_00_00_0:org.apache.tez.dag.api.TezReflectionException:
>  Unable to load class: sample.sampletez.OnetoOne$V1Processor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
> at 
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
> 
> 
> Should I be adding my application jar somewhere so that it can get 
> distributed ? I tried adding my jar via  tezClient.addAppMasterLocalFiles() 
> but it didn't help.
> 
> What am I not doing ? 
> 
> thanks!
> Madhu
> 
> 
> 
> 
>

Re: Node unable to start vertex

2016-08-25 Thread Hitesh Shah 
Hello Madhu, 

There are 2 approaches for this:

1) Programmatically, for user code running in tasks, you would need to use 
either DAG::addTaskLocalFiles() or Vertex::addTaskLocalFiles() - former if the 
same jars are needed in all tasks of the DAG.  
TezClient::addAppMasterLocalFiles only impacts the ApplicationMaster. 

2) Configure tez.aux.uris. This will ensure that all files specified here will 
be available in the AM and all tasks.  

thanks
— Hitesh

> On Aug 25, 2016, at 11:46 AM, Madhusudan Ramanna  wrote:
> 
> Hello,
> 
> I'm trying to extend TezExamplesBase and get a dag running on yarn (pseudo 
> cluster mode on my host).
> 
> For some reason, I'm running into class not found exception on the node
> 
> Vertex failed, vertexName=v1, vertexId=vertex_1471907702278_0030_1_00, 
> diagnostics=[Task failed, taskId=task_1471907702278_0030_1_00_00, 
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
> failure ) : 
> attempt_1471907702278_0030_1_00_00_0:org.apache.tez.dag.api.TezReflectionException:
>  Unable to load class: sample.sampletez.OnetoOne$V1Processor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:46)
> at 
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:87)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:668)
> 
> 
> Should I be adding my application jar somewhere so that it can get 
> distributed ? I tried adding my jar via  tezClient.addAppMasterLocalFiles() 
> but it didn't help.
> 
> What am I not doing ? 
> 
> thanks!
> Madhu
> 
> 
> 
> 
>