Re: HadoopDataOutputStream maybe does not expose enough methods of org.apache.hadoop.fs.FSDataOutputStream
I think that is a very good idea. Originally, we wrapped the Hadoop FS classes for convenience (they were changing, we wanted to keep the system independent of Hadoop), but these are no longer relevant reasons, in my opinion. Let's start with your proposal and see if we can actually get rid of the wrapping in a way that is friendly to existing users. Would you open an issue for this? Greetings, Stephan On Wed, Aug 26, 2015 at 6:23 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Hi, I’ve noticed that when you use org.apache.flink.core.fs.FileSystem to write into a hdfs file, calling org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.create(), it returns a HadoopDataOutputStream that wraps a org.apache.hadoop.fs.FSDataOutputStream (under its org.apache.hadoop.hdfs.client .HdfsDataOutputStream wrappper). However, FSDataOutputStream exposes many methods like flush, getPos etc, but HadoopDataOutputStream only wraps write close. For instance, flush() calls the default, empty implementation of OutputStream instead of the hadoop one, and that’s confusing. Moreover, because of the restrictive OutputStream interface, hsync() and hflush() are not exposed to Flink ; maybe having a getWrappedStream() would be convenient. (For now, that prevents me from using Flink FileSystem object, I directly use hadoop’s one). Regards, Arnaud -- L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Encountered unregistered class ID
Hi to all, I'm running a job (with Flink 0.10-SNAPSHOT) that reads some parquet-thrift objects and then it performs some joins and I receive the following exception: Caused by: java.io.IOException: Thread 'SortMerger spilling thread' terminated due to an exception: Encountered unregistered class ID: 93 at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784) Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 93 at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119) at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235) at org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71) at org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781) What could be the cause of this? Best, Flavio
Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails.
Nice. More configuration options :) On Wed, Aug 26, 2015 at 5:58 PM, Robert Metzger rmetz...@apache.org wrote: Therefore, my change will include a configuration option to set a custom location for the file. On Wed, Aug 26, 2015 at 5:55 PM, Maximilian Michels m...@apache.org wrote: The only problem with writing the temp is that it will be gone after a restart. While this is not important for PIDs because the system has been restarted anyways, this can actually be a problem if you want to resume a YARN cluster after you have restarted your system. On Wed, Aug 26, 2015 at 3:34 PM, Robert Metzger rmetz...@apache.org wrote: Yep. I think the start-*.sh scripts are also writing the PID to tmp. On Wed, Aug 26, 2015 at 3:30 PM, Maximilian Michels m...@apache.org wrote: Can't we write the file to the system's temp directory or the user home? IMHO this is more standard practice for these type of session information. On Wed, Aug 26, 2015 at 3:25 PM, Robert Metzger rmetz...@apache.org wrote: Great ;) Not yet, but you are the second user to request this. I think I'll put the file somewhere else now. On Wed, Aug 26, 2015 at 3:19 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Ooops… Seems it was rather a write problem on the conf dir… Sorry, it works! BTW, it’s not really nice to have an application write the configuration dir ; it’s often a root protected directory in usr/lib/flink. Is there a parameter to put that file elsewhere ? De : Robert Metzger [mailto:rmetz...@apache.org] Envoyé : mercredi 26 août 2015 14:42 À : user@flink.apache.org Objet : Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails. Hi Arnaud, usually, you don't have to manually specify the JobManager address manually with the -m argument, because it is reading it from the conf/.yarn-session.properties file. Give me a few minutes to reproduce the issue. On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Hi, Using last nightly build, it seems that if you call yarn-session.sh with -nm option to give a nice application name, then you cannot submit a job with flink run without specify the ever changing -m jobManager address since it does not find it any longer. Regards, Arnaud L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails.
Therefore, my change will include a configuration option to set a custom location for the file. On Wed, Aug 26, 2015 at 5:55 PM, Maximilian Michels m...@apache.org wrote: The only problem with writing the temp is that it will be gone after a restart. While this is not important for PIDs because the system has been restarted anyways, this can actually be a problem if you want to resume a YARN cluster after you have restarted your system. On Wed, Aug 26, 2015 at 3:34 PM, Robert Metzger rmetz...@apache.org wrote: Yep. I think the start-*.sh scripts are also writing the PID to tmp. On Wed, Aug 26, 2015 at 3:30 PM, Maximilian Michels m...@apache.org wrote: Can't we write the file to the system's temp directory or the user home? IMHO this is more standard practice for these type of session information. On Wed, Aug 26, 2015 at 3:25 PM, Robert Metzger rmetz...@apache.org wrote: Great ;) Not yet, but you are the second user to request this. I think I'll put the file somewhere else now. On Wed, Aug 26, 2015 at 3:19 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Ooops… Seems it was rather a write problem on the conf dir… Sorry, it works! BTW, it’s not really nice to have an application write the configuration dir ; it’s often a root protected directory in usr/lib/flink. Is there a parameter to put that file elsewhere ? De : Robert Metzger [mailto:rmetz...@apache.org] Envoyé : mercredi 26 août 2015 14:42 À : user@flink.apache.org Objet : Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails. Hi Arnaud, usually, you don't have to manually specify the JobManager address manually with the -m argument, because it is reading it from the conf/.yarn-session.properties file. Give me a few minutes to reproduce the issue. On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Hi, Using last nightly build, it seems that if you call yarn-session.sh with -nm option to give a nice application name, then you cannot submit a job with flink run without specify the ever changing -m jobManager address since it does not find it any longer. Regards, Arnaud L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
HadoopDataOutputStream maybe does not expose enough methods of org.apache.hadoop.fs.FSDataOutputStream
Hi, I’ve noticed that when you use org.apache.flink.core.fs.FileSystem to write into a hdfs file, calling org.apache.flink.runtime.fs.hdfs.HadoopFileSystem.create(), it returns a HadoopDataOutputStream that wraps a org.apache.hadoop.fs.FSDataOutputStream (under its org.apache.hadoop.hdfs.client .HdfsDataOutputStream wrappper). However, FSDataOutputStream exposes many methods like flush, getPos etc, but HadoopDataOutputStream only wraps write close. For instance, flush() calls the default, empty implementation of OutputStream instead of the hadoop one, and that’s confusing. Moreover, because of the restrictive OutputStream interface, hsync() and hflush() are not exposed to Flink ; maybe having a getWrappedStream() would be convenient. (For now, that prevents me from using Flink FileSystem object, I directly use hadoop’s one). Regards, Arnaud L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: Encountered unregistered class ID
Hi Flavio! That exception means that the Kryo serializers are not in sync. The writers have registered types that the readers do not know. Two possible reasons that I can think of from the top of my head: 1) Do you manually register types? Are you registering new types in the middle of your program? 2) Flink may not be passing the Kryo registration properly down to the sort reader. Greetings, Stephan On Wed, Aug 26, 2015 at 9:04 AM, Flavio Pompermaier pomperma...@okkam.it wrote: Hi to all, I'm running a job (with Flink 0.10-SNAPSHOT) that reads some parquet-thrift objects and then it performs some joins and I receive the following exception: Caused by: java.io.IOException: Thread 'SortMerger spilling thread' terminated due to an exception: Encountered unregistered class ID: 93 at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784) Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 93 at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119) at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235) at org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71) at org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781) What could be the cause of this? Best, Flavio
RE: Application-specific loggers configuration
Hi ! Yes, we’re starting our job with “flink run --jobmanager yarn-cluster” So it’s perfect, we’ll use your fix and, when it’s out, we’ll switch to flink 0.9.1. B.R. From: Aljoscha Krettek [mailto:aljos...@apache.org] Sent: mardi 25 août 2015 19:25 To: user@flink.apache.org Subject: Re: Application-specific loggers configuration Hi Gwenhaël, are you using the one-yarn-cluster-per-job mode of Flink? I.e., you are starting your Flink job with (from the doc): flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 4096 ./examples/flink-java-examples-0.10-SNAPSHOT-WordCount.jar If you are, then this is almost possible on the current version of Flink. What you have to do is copy the conf directory of Flink to a separate directory that is specific to your job. There you make your modifications to the log configuration etc. Then, when you start your job you do this instead: export FLINK_CONF_DIR=/path/to/my/conf flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 4096 ./examples/flink-java-examples-0.10-SNAPSHOT-WordCount.jar You can easily put this into your startup script, of course. I said almost possible because this requires a small fix in bin/flink. Around line 130 this line: FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf needs to be replaced by this line: if [ -z $FLINK_CONF_DIR ]; then FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf; fi (We will fix this in the upcoming version and the 0.9.1 bugfix release.) Does this help? Let us know if you are not using the one-yarn-cluster-per-job mode, then we'll have to try to find another solution. Best, Aljoscha On Tue, 25 Aug 2015 at 16:22 Gwenhael Pasquiers gwenhael.pasqui...@ericsson.commailto:gwenhael.pasqui...@ericsson.com wrote: Hi, We’re developing the first of (we hope) many flink streaming app. We’d like to package the logging configuration (log4j) together with the jar. Meaning, different application will probably have different logging configuration (ex: to different logstash ports) … Is there a way to “override” the many log4j properties files that are in flink/conf./*.properties ? In our environment, the flink binaries would be on the PATH, and our apps would be : - Jar file - App configuration files - Log configuration files - Startup script B.R. Gwenhaël PASQUIERS
Re: Encountered unregistered class ID
I'm running my job from my Eclipse and I don't register any Kryo class in the env. On Wed, Aug 26, 2015 at 10:34 AM, Stephan Ewen se...@apache.org wrote: Hi Flavio! That exception means that the Kryo serializers are not in sync. The writers have registered types that the readers do not know. Two possible reasons that I can think of from the top of my head: 1) Do you manually register types? Are you registering new types in the middle of your program? 2) Flink may not be passing the Kryo registration properly down to the sort reader. Greetings, Stephan On Wed, Aug 26, 2015 at 9:04 AM, Flavio Pompermaier pomperma...@okkam.it wrote: Hi to all, I'm running a job (with Flink 0.10-SNAPSHOT) that reads some parquet-thrift objects and then it performs some joins and I receive the following exception: Caused by: java.io.IOException: Thread 'SortMerger spilling thread' terminated due to an exception: Encountered unregistered class ID: 93 at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784) Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 93 at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119) at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235) at org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71) at org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781) What could be the cause of this? Best, Flavio
RE: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails.
Ooops… Seems it was rather a write problem on the conf dir… Sorry, it works! BTW, it’s not really nice to have an application write the configuration dir ; it’s often a root protected directory in usr/lib/flink. Is there a parameter to put that file elsewhere ? De : Robert Metzger [mailto:rmetz...@apache.org] Envoyé : mercredi 26 août 2015 14:42 À : user@flink.apache.org Objet : Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails. Hi Arnaud, usually, you don't have to manually specify the JobManager address manually with the -m argument, because it is reading it from the conf/.yarn-session.properties file. Give me a few minutes to reproduce the issue. On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud al...@bouyguestelecom.frmailto:al...@bouyguestelecom.fr wrote: Hi, Using last nightly build, it seems that if you call yarn-session.sh with -nm option to give a nice application name, then you cannot submit a job with flink run without specify the ever changing -m jobManager address since it does not find it any longer. Regards, Arnaud L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails.
Hi Arnaud, I think my answer to Gwenhaël could also be helpful to you: are you using the one-yarn-cluster-per-job mode of Flink? I.e., you are starting your Flink job with (from the doc): flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 4096 ./examples/flink-java-examples-0.10-SNAPSHOT-WordCount.jar If you are, then this is almost possible on the current version of Flink. What you have to do is copy the conf directory of Flink to a separate directory that is specific to your job. There you make your modifications to the log configuration etc. Then, when you start your job you do this instead: export FLINK_CONF_DIR=/path/to/my/conf flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 4096 ./examples/flink-java-examples-0.10-SNAPSHOT-WordCount.jar You can easily put this into your startup script, of course. I said almost possible because this requires a small fix in bin/flink. Around line 130 this line: FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf needs to be replaced by this line: if [ -z $FLINK_CONF_DIR ]; then FLINK_CONF_DIR=$FLINK_ROOT_DIR_MANGLED/conf; fi (We will fix this in the upcoming version and the 0.9.1 bugfix release.) Does this help? Let us know if you are not using the one-yarn-cluster-per-job mode, then we'll have to try to find another solution. Forget the stuff about one-yarn-cluster-per-job but the other stuff should be right what you want. I hope it helps. Best, Aljoscha On Wed, 26 Aug 2015 at 15:19 LINZ, Arnaud al...@bouyguestelecom.fr wrote: Ooops… Seems it was rather a write problem on the conf dir… Sorry, it works! BTW, it’s not really nice to have an application write the configuration dir ; it’s often a root protected directory in usr/lib/flink. Is there a parameter to put that file elsewhere ? *De :* Robert Metzger [mailto:rmetz...@apache.org] *Envoyé :* mercredi 26 août 2015 14:42 *À :* user@flink.apache.org *Objet :* Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails. Hi Arnaud, usually, you don't have to manually specify the JobManager address manually with the -m argument, because it is reading it from the conf/.yarn-session.properties file. Give me a few minutes to reproduce the issue. On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Hi, Using last nightly build, it seems that if you call yarn-session.sh with -nm option to give a nice application name, then you cannot submit a job with flink run without specify the ever changing -m jobManager address since it does not find it any longer. Regards, Arnaud -- L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails.
Great ;) Not yet, but you are the second user to request this. I think I'll put the file somewhere else now. On Wed, Aug 26, 2015 at 3:19 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Ooops… Seems it was rather a write problem on the conf dir… Sorry, it works! BTW, it’s not really nice to have an application write the configuration dir ; it’s often a root protected directory in usr/lib/flink. Is there a parameter to put that file elsewhere ? *De :* Robert Metzger [mailto:rmetz...@apache.org] *Envoyé :* mercredi 26 août 2015 14:42 *À :* user@flink.apache.org *Objet :* Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails. Hi Arnaud, usually, you don't have to manually specify the JobManager address manually with the -m argument, because it is reading it from the conf/.yarn-session.properties file. Give me a few minutes to reproduce the issue. On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Hi, Using last nightly build, it seems that if you call yarn-session.sh with -nm option to give a nice application name, then you cannot submit a job with flink run without specify the ever changing -m jobManager address since it does not find it any longer. Regards, Arnaud -- L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails.
Can't we write the file to the system's temp directory or the user home? IMHO this is more standard practice for these type of session information. On Wed, Aug 26, 2015 at 3:25 PM, Robert Metzger rmetz...@apache.org wrote: Great ;) Not yet, but you are the second user to request this. I think I'll put the file somewhere else now. On Wed, Aug 26, 2015 at 3:19 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Ooops… Seems it was rather a write problem on the conf dir… Sorry, it works! BTW, it’s not really nice to have an application write the configuration dir ; it’s often a root protected directory in usr/lib/flink. Is there a parameter to put that file elsewhere ? De : Robert Metzger [mailto:rmetz...@apache.org] Envoyé : mercredi 26 août 2015 14:42 À : user@flink.apache.org Objet : Re: [0.10-SNAPSHOT ] When naming yarn application (yarn-session -nm), flink run without -m fails. Hi Arnaud, usually, you don't have to manually specify the JobManager address manually with the -m argument, because it is reading it from the conf/.yarn-session.properties file. Give me a few minutes to reproduce the issue. On Wed, Aug 26, 2015 at 2:39 PM, LINZ, Arnaud al...@bouyguestelecom.fr wrote: Hi, Using last nightly build, it seems that if you call yarn-session.sh with -nm option to give a nice application name, then you cannot submit a job with flink run without specify the ever changing -m jobManager address since it does not find it any longer. Regards, Arnaud L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
Re: New contributor tasks
Additionally, If you have any questions about contributing, please send a mail to dev mailing list. Regards, Chiwan Park On Aug 27, 2015, at 2:11 PM, Chiwan Park chiwanp...@apache.org wrote: Hi Naveen, There is a guide document [1] about contribution in homepage. Please read first before contributing. Maybe a document of coding guidelines [2] would be helpful to you. You can find some issues [3] to start contributing to Flink in JIRA. The issues are labeled as `starter`, `newbie`, or `easyfix`. Happy contributing! Regards, Chiwan Park [1] http://flink.apache.org/how-to-contribute.html [2] http://flink.apache.org/coding-guidelines.html [3] https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20starter%20ORDER%20BY%20priority%20DESC On Aug 27, 2015, at 2:06 PM, Naveen Madhire vmadh...@umail.iu.edu wrote: Hi, I've setup Flink on my local linux machine and ran few examples as well. Also setup the Intellij IDE for the coding environment. Can anyone please let me know if there are any beginner tasks which I can take a look for contributing to Apache Flink codebase. I am comfortable in Java and Scala programming. Please let me know. Thanks, Naveen
New contributor tasks
Hi, I've setup Flink on my local linux machine and ran few examples as well. Also setup the Intellij IDE for the coding environment. Can anyone please let me know if there are any beginner tasks which I can take a look for contributing to Apache Flink codebase. I am comfortable in Java and Scala programming. Please let me know. Thanks, Naveen
Re: New contributor tasks
Hi Naveen, There is a guide document [1] about contribution in homepage. Please read first before contributing. Maybe a document of coding guidelines [2] would be helpful to you. You can find some issues [3] to start contributing to Flink in JIRA. The issues are labeled as `starter`, `newbie`, or `easyfix`. Happy contributing! Regards, Chiwan Park [1] http://flink.apache.org/how-to-contribute.html [2] http://flink.apache.org/coding-guidelines.html [3] https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20starter%20ORDER%20BY%20priority%20DESC On Aug 27, 2015, at 2:06 PM, Naveen Madhire vmadh...@umail.iu.edu wrote: Hi, I've setup Flink on my local linux machine and ran few examples as well. Also setup the Intellij IDE for the coding environment. Can anyone please let me know if there are any beginner tasks which I can take a look for contributing to Apache Flink codebase. I am comfortable in Java and Scala programming. Please let me know. Thanks, Naveen
