Re: How to build ozzie 4.2.0 with hadoop-2.7.1

mmohanga...@gmail.com> wrote: > > > Hi, > > > > I am trying to build ozzie 2.4.0 with hadoop 2.7.1, build successful but > > it doesn't have hadoop-2 under hadooplibs to create war file. can someone > > help me? > > > > Ram > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie error with pig

6-oozie-oozi-C@1]::CoordActionInputCheck:: Missing > deps: > 2016-06-21 19:33:34,619 WARN ParameterVerifier:546 - > SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0002430-160613201309706-oozie-oozi-C] > ACTION[0002430-160613201309706-oozie-oozi-C@1] The application does not > define formal parameters in its XML definition > 2016-06-21 19:33:54,696 INFO CoordActionUpdateXCommand:543 - > SERVER[hadoop.oss.ads] USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[0002430-160613201309706-oozie-oozi-C] > ACTION[0002430-160613201309706-oozie-oozi-C@1] Updating Coordintaor > action id :0002430-160613201309706-oozie-oozi-C@1 status to KILLED, > pending = 0 > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: ShareLibService Error

> > What is the processes to upgrade common-io-lib? i already have ( manually > copied) that jar under $OOZIE_HOME/lib dir but still having issues. > > Thanks, > Ram > > On Wed, Jun 15, 2016 at 12:10 PM, Peter Cseh <gezap...@cloudera.com> > wrote: > > >

Re: Workflow submission time

with a higher number of actions. Is it a > known issue? Is there some tuning to perform? are there workarounds? should > I use sub-workflows? > > Thanks for your help, > Best regards, > Pierre > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Workflow submission time

be able to limit the number of > concurrent actions that can run in parallel from the fork. Is it something > we can do? > > Thanks, > Pierre. > > > > > > > 2016-06-27 17:01 GMT+02:00 Peter Cseh <gezap...@cloudera.com>: > > > Hi Pierre, > > > >

Re: Extended attributes

know if it's a long wait? I'd like to contribute a patch soon. > > On Mon, Jan 16, 2017 at 3:00 AM, Peter Cseh <gezap...@cloudera.com> wrote: > > > Hi, > > > > the patch for OOZIE-1770 > > <https://issues.apache.org/jira/browse/OOZIE-1770> chang

Re: Extended attributes

xtend FS action functionality to extended > attributes and I realized that this functionality is not exposed in Hadoop > 2.4. All major distros moved to 2.6.1 at the least, what are the plans to > migrate these libraries? > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie sharelib does not exists

> > but when I verify the available sharelib using > > oozie admin -shareliblist > > it shows only [Availble Sharelib] message with no libs actually listed. > > Any idea, what can be the issue? > > ​Thank you, > Pushkar > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie sharelib does not exists

folder /user/pushkargujar/share/lib/ > lib_ > > > > *​* > > > Thank you, > *Pushkar Gujar* > > > On Sun, Feb 26, 2017 at 3:24 PM, Peter Cseh <gezap...@cloudera.com> wrote: > >> HI, >> >> What's the user you're running the oozie server with?

Re: oozie execute shell(content hive or sqoop command)

e.sh > > #!/bin/bash > > hive -e "select count(*) from test;" > > my workflow content:workflow.xml > > > > The following error occurred： > > > > How to solve?,Please > > > > > > > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: 答复: Launcher exception: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf

Hi, I've just created a build from Oozie's master and checked and it seems hive-exec jar is needed and included in hive2 action's sharelib: > GezapetiMBP:bin gezapeti$ hdfs dfs -ls share/lib/lib_20160911145035/hive2 > Found 48 items > ... > -rw-r--r-- 3 gezapeti supergroup 115618

Re: Unable to configure custom mapreduce.job.queuename in Oozie shell action

support 'mapreduce.job.quenename specification? > > I am using MR2, Hadoop version 2.7.1, oozie version 4.2.0. > > Any help would be really appreciated. > > Thanks > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Launcher exception: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf

; > oozie.wf.application.path > hdfs://emr-cluster/user/oozie/examples/apps/hive2 > > > oozie.use.system.libpath > true > > > queueName > default > > > jdbcURL > jdbc:hive2://localhost:1/default > > > user.name > oozie > > > jobTracker > rm1,rm2 > > > mapreduce.job.user.name > oozie > > > nameNode > hdfs://emr-cluster > > > > > > How can I fix this error, can you give me a hand, thanks in advance! > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie Hive2 Action with Kerberos security and HS2 HTTP transport mode

jdbc:hive2://myhiveserver: > 10001/;sasl.qop=auth-conf;transportMode=http;httpPath=cliservice > > > script.hql > > > > > > > > > > > > Action failed, error > > message[${wf:errorMessage(wf: > lastErrorNode())}] > > > > > > > > > > > > *Jiří Kaplan* > Software Developer > > *Dell** | *R Database Management, EMEA > > [image: dell_software] > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Problems with properties in Java action

for examle > > "some.custom.property=true", but it also didn't set. > > Then I've tried to change some oozie property, for example > > "oozie.launcher.mapreduce.map.memory.mb" and it worked. > > > > So I'm confused how oozie is working with job properties. > > > > I'm checking all properties in JobHistory -> configuration. > > > > > > > > Regards, > > Andrey > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Spark-action failed launching MRAppMaster

t; at javax.security.auth.Subject.doAs(Subject.java:422) > > at org.apache.hadoop.security.UserGroupInformation.doAs( > UserGroupInformation.java:1491) > > at org.apache.hadoop.mapreduce.v2.app.MRAppMaster. > initAndStartAppMaster(MRAppMaster.java:1441) > > at org.a

Re: Getting Permission denied to run shell action

he ${hadoop.tmp.dir}=/tmp/ but oozie > still saying /data/* permission denied, i am not sure whats going on here, > can some one help me to figure it out? > > Thanks, > Ram > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: how to execute HDFS file actions from shell action on Kerberized Cluster

OCATION > to authenticate for HDFS file actions ? > > Thanks and Regards > Aniruddh > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Does Oozie support run sparkR with spark action?

Stream.defaultReadFields(ObjectInpu > Calls: sql -> callJMethod -> invokeJava > Execution halted > Intercepting System.exit(1) > > Does oozie support run sparkR in spark action? Or we should only wrap > it in ssh action? > > Thanks a lot > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Does Oozie support run sparkR with spark action?

6-11-15 13:59 GMT+08:00 Dongying Jiao <pineapple...@gmail.com>: > > > Hi Peter: > > Thank you very much for your reply. > > I will have a try and tell you the result. > > > > 2016-11-12 5:02 GMT+08:00 Peter Cseh <gezap...@cloudera.com>: > > > >>

Re: Welcome new Oozie Committers - Abhishek Bafna and Satish Saley

gt; Congrats! > > > - Robert, on behalf of the Oozie PMC > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Question about spark-bagel in oozie 4.3.0

.0, as there is no spark-bagel > module in spark 2.X, is there any risk if I delete this module in oozie > 4.3.0? > > Thanks very much > > Best Regards, > Dongying Jiao > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Welcome new Oozie Committers - Peter Bacsko and Peter Cseh

gt;> > > >> It is my pleasure to announce that the Oozie PMC has invited > > >> Peter Bacsko and Peter (Geza) Cseh to become Oozie committers > > >> and they have both accepted our invitation. > > >> > > >> Please join me congratulating them. > > >> Congrats! > > >> > > >> > > >> - Robert, on behalf of the Oozie PMC > > >> > > > > > > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: workflow is not running

r. My python script > is at ~/emr/lib/my_python.py I am running it via command:- oozie job > --oozie http://ip-10-163-125-124.ap-southeast-1.compute.internal: > 11000/oozie -config ~/emr/job.properties -runwhen I check info of job > via :- oozie job info -020-170417052643712-o

Re: Job Submission to Cross Cluster Fails

t; > > Why job.xml (sent to RM via Oozie) has Cluster-1's NN and RM > information (if that is the case)? > > > Do I require any other configuration setting to enable this cross > cluster job submission work? > > > Thanks. > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: A problem about oozie workflow about shell Action.

077a] No Notification URL is defined. > Therefore nothing to notify for job 127-170506114458023-oozie- > oozi-W@shell-077a > 2017-05-09 19:51:06,995 INFO WorkflowNotificationXCommand:520 - > SERVER[tw-master] USER[-] GROUP[-] TOKEN[-] APP[-] > JOB[127-170506114458023-oozie-oozi-W] ACTION[] No Notification URL is > defined. Therefore nothing to notify for job 127-170506114458023-oozie- > oozi-W > 3. I don't know how to do with ERROR is considered as FAILED > for SLA.Is there something wrong with my permission,or the solution is not > supported. > Thanks > > Best Regards, > Hollis > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

OYA! - no more LauncherMappers

a lot to do: you can check out OOZIE-2889 for details. Thanks again everyone! gp -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie Job + Version Control Tool

a.i...@gmail.com > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > Oozie does not have version control for jobs. When you submit a > > > > > > > workflow/coordinator/bundle to oozie, it stores it into DB uses > > it > > > > from > > > > > > > there for further execution. > > > > > > > > > > > > > > Thanks, > > > > > > > Abhishek > > > > > > > > On Oct 19, 2016, at 1:16 PM, goun na <gou...@gmail.com > > > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > > Hi users, > > > > > > > > > > > > > > > > What is the best to manage Oozie jobs? Is there a built-in > > > version > > > > > > > control > > > > > > > > feature? > > > > > > > > > > > > > > > > Best, > > > > > > > > Goun Na > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > *Per Ullberg* > > > > > > Data Vault Tech Lead > > > > > > Odin Uppsala > > > > > > +46 701612693 <+46+701612693> > > > > > > > > > > > > Klarna AB (publ) > > > > > > Sveavägen 46, 111 34 Stockholm > > > > > > Tel: +46 8 120 120 00 <+46812012000> > > > > > > Reg no: 556737-0431 > > > > > > klarna.com > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > *Per Ullberg* > > > > Data Vault Tech Lead > > > > Odin Uppsala > > > > +46 701612693 <+46+701612693> > > > > > > > > Klarna AB (publ) > > > > Sveavägen 46, 111 34 Stockholm > > > > Tel: +46 8 120 120 00 <+46812012000> > > > > Reg no: 556737-0431 > > > > klarna.com > > > > > > > > > > > > > > > -- > > > > *Per Ullberg* > > Data Vault Tech Lead > > Odin Uppsala > > +46 701612693 <+46+701612693> > > > > Klarna AB (publ) > > Sveavägen 46, 111 34 Stockholm > > Tel: +46 8 120 120 00 <+46812012000> > > Reg no: 556737-0431 > > klarna.com > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie Sqoop Wallet Specification

; ${walletLocation}/sqlnet.ora#sqlnet.ora > ${walletLocation}/tnsnames.ora#tnsnames.ora > > Kindly help. > > Thanks, > Arun > > -- > > *Clarity Solution Group is now Clarity Insights. Check out our website > using the link below. * > *ClarityInsights.com* <http://ClarityInsights.com> > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Error in running coord-input-logic example

Hi, Can you attach a full stack trace, the coordinator example and the command you're executing when getting the error? Thanks gp On Fri, Sep 8, 2017 at 1:22 PM 罗 辉 wrote: > hi there > > I got a problem in running the coord-input-logic example. I've > modified

Re: Set SMTP settings in workflow

gs > directly in the workflow would have the advantage to use a different FROM > address for different workflows instead of one central address. > > > > Thanks for your help. > > > > Best, Jan > > > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Ozzie Spark 2.X action

Hey! You could compile Oozie using the spark-2 profile and it should work fine. You may even overwrite the versions it's pulling in: https://github.com/apache/oozie/blob/master/pom.xml#L1956-L1964 gp On Wed, Oct 25, 2017 at 7:32 AM Aravindakshan Srinivasan wrote:

Re: Oozie email action truncating the string containing newline chars

ete string is part of the body of > the email? > > Thanks! > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image: Clouder

Re: How To Set Environment Variables For Spark Action Script From XML Definition

en unable > to find a similar straightforward way of doing it for a spark action. > > Oozie Version: 4.1.0-cdh5.12.1 > > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitt

Re: Oozie Variable Substitution In XML's Workflow Name

ain. At this moment I'm waiting for a more elegant approach but I've > failed to come to it on my own, so I decided to reach out to other oozie > users out there and see what comes up. > > Thanks in advance. > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.clou

Re: Spark 2.3 in oozie

com> wrote: > Hello, > > Does oozie supports spark 2.3? Or will it even care of the spark version > > I want to use spark action > > > > Thanks, > Purna > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera]

Re: Oozie for spark jobs without Hadoop

>>>> >>>> Also When I submit a job with new http client jar, I get >>>> >>>> ```Error: IO_ERROR : java.io.IOException: Error while connecting Oozie >>>> server. No of retries = 1. Exception = Could not authenticate, >>>> Authe

Re: Oozie for spark jobs without Hadoop

3a ? > > I’m using s3 but if this is for s3a do you know which jar I need to > include I mean Hadoop-aws jar or any other jar if required > > Hadoop-aws-2.8.3.jar is what I’m using > > On Wed, May 16, 2018 at 5:19 PM Peter Cseh <gezap...@cloudera.com> wrote: > >>

Re: Oozie for spark jobs without Hadoop

> > > > components. How to install Oozie please *find here > > > > <https://oozie.apache.org/docs/5.0.0/AG_Install.html>*. > > > > > > > > Regards, > > > > > > > > Andras > > > > > > > > On Tue, May 15, 2018 at 4

Re: Spark 2.3 in oozie

; local:///opt/spark/examples/jars/spark-examples_2.11-2.3. > 0.jar > > > > > Is this because of uri:oozie:spark-action:1.0 in spark xml tag? Does it > needs to be spark-action:2.0 as I’m using spark 2.3? > > > Please suggest! > > > On Tue, May 15, 2018 at 12

Re: Oozie for spark jobs without Hadoop

like to know if I can use sparkaction in oozie without having > > Hadoop > > > cluster? > > > > > > I want to use oozie to schedule spark jobs on Kubernetes cluster > > > > > > I’m a beginner in oozie > > > > > > Thanks > > &g

Re: Spark 2.3 in oozie

ute goal on project oozie-examples: Could not > resolve > > dependencies for project org.apache.oozie:oozie-examples:jar:5.0.0: Could > > not find artifact org.apache.spark:spark-core_2.10:jar:2.3.0 in > resolution > > * > > > > On Tue, May 15, 2018 at 11:14 AM Pe

Re: Oozie for spark jobs without Hadoop

anks On Wed, May 16, 2018 at 5:47 PM, purna pradeep <purna2prad...@gmail.com> wrote: > This is what I already have in my oozie-site.xml > > > > oozie.service.HadoopAccessorService. > supported.filesystems > > * > > > > On Wed, May

Re: Oozie for spark jobs without Hadoop

Could not authenticate, >> Authentication failed, status: 500, message: Server Error``` >> >> >> On Thu, May 17, 2018 at 12:14 PM purna pradeep <purna2prad...@gmail.com> >> wrote: >> >>> Ok I have tried this >>> >>> It appears tha

Re: Spark 2.3 in oozie

pradeep <purna2prad...@gmail.com> wrote: > Thanks Peter! > > I’m able to run spark pi example on Kubernetes cluster from oozie after > this change > > On Wed, May 16, 2018 at 10:27 AM Peter Cseh <gezap...@cloudera.com> wrote: > > > The version of the xml sc

Re: Oozie for spark jobs without Hadoop

rg.apache.oozie.service.HadoopAccessorService$5.run( >> HadoopAccessorService.java:623 >> >> >> On Wed, May 16, 2018 at 2:19 PM purna pradeep <purna2prad...@gmail.com> >> wrote: >> >>> This is what is in the logs >>> >>> 2018-05-16

Re: Reg: Oozie 4.1.0 having jdbc error

ne.TimelineUtils. > buildTimelineTokenService > * > [hadoop@svdt5neonhadoop01 lib]$ > [hadoop@svdt5neonhadoop01 lib]$ ls |wc -l > 160 > [hadoop@svdt5neonhadoop01 lib]$ pwd > /usr/local/oozie/lib > [hadoop@svdt5neonhadoop01 lib]$ > > > Please help ! > > Thanks & Re

Re: Delegation Token Expiring

ot > the addressee, please promptly delete this message and notify the sender of > the delivery error by e-mail or you may call Cerner's corporate offices in > Kansas City, Missouri, U.S.A at (+1) (816)221-1024. > -- *Peter Cseh *| Software Engineer cloudera.com <https://www

Re: Reg: Oozie 4.1.0 having jdbc error

dir=/usr/local/oozie/data -Doozie.instance.id=svdt5neonh > adoop01.safaricom.net -Doozie.config.file=oozie-site.xml > -Doozie.log4j.file=oozie-log4j.properties -Doozie.log4j.reload=10 > -Doozie.http.hostname=svdt5neonhadoop01.safaricom.net > -Doozie.admin.port=11001 -Doozie.http.port=11000 -

Re: oozie in Shell script

d run that until the shell script hits end date. > > I saw examples the other way, but it is not what i wanted. > > Is it possible? any suggestions will help. > > Thanks > Sowjanya > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com>

Re: Oozie Presentation - New Features Overview and Ambari GUI

quot;Cluster Continuous Delivery > with Oozie": > Slides: http://bit.ly/ApacheCon_Cluster_Continuous_Delivery_with_Oozie > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera

Re: HDP3.0: oozie cannot start

gt; { > "oozie-site": { > "properties": { > "oozie.service.JPAService.jdbc.username": "admin", > "oozie.service.JPAService.jdbc.password": "%SERVICE_PASSWORD%", > "oozie.email.from.address"

Re: HDP3.0: oozie cannot start

Jiang wrote: > > > Thanks Peter. I attached sanitized > /etc/oozie/3.0.0.0-1634/0/oozie-site.xml. > > Please let me know if you see any issue. > > > > On Mon, Jul 30, 2018 at 1:23 AM, Peter Cseh > > > wrote: > > > >> Hi! > >> &

Re: Oozie 5.0.0 Launcher Out of Memory error

nning > beyond physical memory limits. Current usage: 2.2 GB of 2 GB physical > memory used; 9.9 GB of 10 GB virtual memory used. Killing container. > > > Thank you > Suresh. > -- *Peter Cseh *| Software Engineer cloudera.com <https://www.cloudera.com> [image: Cloudera]

Re: Oozie web console - links to job

job in yarn > console to control counters and logs of the job. > > Is this a feature or a bug? Is there an option to restore previous > behavior? If so, how to do this? > > -- Peter Cseh Software Engineer <http://www.cloudera.com>

Re: Oozie 5.1.0 release and plans for the year

; year. The git action is the closest one, but there are others in the > > > > pipeline like Maven and callback. > > > > > > > > Is anyone aware of issues in which we should wait for to start > working > > on > > > > 5.1.0? > > > > If nobody

Re: oozie 5.0.0 on AWS EMR

Service.java#L167 > > > URI uri = new URI(jobConf.get(OozieClient.APP_PATH)); > > Configuration conf = has.createJobConf(uri.getAuthority()); > > > I am NOT 100% sure, but the above code indeed returns the FileSystem > eventually complains "WRO

Re: oozie 5.0.0 on AWS EMR

priorities. Once we have the reproduction steps, we should file a Jira for this. gp On Mon, Mar 25, 2019 at 8:34 PM wrote: > Hi Yong > > Have you also tried s3a in place of s3? > > > - > Suresh. > > > > On Mar 25, 2019, at 2:03 PM, Peter Cseh > wro

Re: Oozie 5.2.0 Build Fail

information about the errors and possible solutions, > please read the following articles: > > [*ERROR*] *[Help 1]* > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException > > [*ERROR*] > > [*ERROR*] After correcting the problems, you can resume the build wit

Re: Passing variables from dataset to workflow in Coordinator

> I hope I'm missing something. > > Thank you for your help! > > Cheers, > Lars > -- *Peter Cseh* | Software Engineer, Cloudera Search cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://t

Re: org.apache.hadoop.security.AccessControlException:Client cannot authenticate via:[TOKEN,KERBEROS]

s problem? > > thinks. > I am looking forward to your reply！ > -- *Peter Cseh* | Software Engineer, Cloudera Search cloudera.com <https://www.cloudera.com> [image: Cloudera] <https://www.cloudera.com/> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:

Re: Oozie(5.2.0)get jhs delegation token occured exception in updateCredentials

rties(credentialsProperties, > CredentialsProviderFactory.HDFS); > > addHadoopCredentialProperties(credentialsProperties, > CredentialsProviderFactory.YARN); > > //addHadoopCredentialProperties(credentialsProperties, > CredentialsProviderFactory.JHS); > } > ... -- *Peter Cseh* | Software

Re: RPC response exceeds max length

Srinivasan wrote: > Team, > > Any idea what could be causing this error on a simple Hive action in Oozie? > > JA009: RPC response exceeds maximum data length > > Stack trace attached. > > Thanks, > Aravind > > > -- *Peter Cseh* | Software Engine

Re: recovering from el expression failure

t; > > > > > > > > > However the exists el expression can throw an exception when the > namenode > > > is not whiteliisted in oozie. > > > > > > Unfortunately the whitelist is not under my control and when this > > exception > > > is raised