Hi Chesnay,
I checked the JobManager logs - it's not there either.
On Wed, Oct 21, 2020 at 3:51 PM Chesnay Schepler <[email protected]> wrote:
> The main method is executed in the JobManager process and never reaches
> the TaskExecutors (only the individual functions do).
> As such you have to take a peek into the JobManager logs.
>
> On 10/21/2020 11:37 AM, Manas Kale wrote:
>
> Hi,
> I have the following pattern:
>
> public static void main(String[] args) {
>
> // Get the exec environment. This could be a cluster or a mini-cluster
> used for local development. StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> // Make the Flink runtime use event time as time metric.
> env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
> // Generate a watermark every WATERMARK_PERIODICITY ms.
> env.getConfig().setAutoWatermarkInterval(WATERMARK_PERIODICITY);
>
> Config.readProperties();
>
> }
>
> class Config {
>
> private final static Logger LOGGER = LoggerFactory.getLogger(Config.class);
>
> // Populates variables above with values read from config file.public static
> void readProperties() throws Exception {
> Properties prop = new Properties();
>
> String propFileLocation = System.getenv("FLINK_CONFIG_LOCATION");
> if (propFileLocation == null) {
> System.err.println("Properties file pointer env variable
> FLINK_CONFIG_LOCATION missing!");
> System.exit(1);
> }
> FileInputStream is = null;
> try {
> is = new FileInputStream(new File(propFileLocation));
>
> } catch (Exception e) {
> System.err.println("File " + propFileLocation + " not found!");
> System.exit(1);
> }
>
> prop.load(is);
>
> * LOGGER.info("......"); // prints content read from property file*
>
> }
>
> }
>
>
> When I run this program as a minicluster, I am able to see the
> LOGGER.info() being printed in my console.
> However, when I submit this job as a JAR to a flink cluster, the Config
> class's LOGGER.info()* line above is never printed in the
> taskmanager's logs!* I don't understand why this is happening because
> log statements from other operators are definitely being printed in the
> log files on the cluster. What am I doing wrong?
>
> My log4j.properties file is:
>
> log4j.rootLogger=INFO, console,
> fileAppenderlog4j.appender.console=org.apache.log4j.ConsoleAppenderlog4j.appender.console.layout=org.apache.log4j.PatternLayoutlog4j.appender.console.layout.ConversionPattern=%d{HH:mm:ss,SSS}
> %-5p %-60c %x -
> %m%nlog4j.appender.fileAppender=org.apache.log4j.RollingFileAppenderlog4j.appender.fileAppender.layout=org.apache.log4j.PatternLayoutlog4j.appender.fileAppender.layout.ConversionPattern=%d{HH:mm:ss,SSS}
> %-5p %-60c %x -
> %m%nlog4j.appender.fileAppender.File=dataProcessingEngine.loglog4j.appender.fileAppender.policies.type
> = Policieslog4j.appender.fileAppender.policies.size.type =
> SizeBasedTriggeringPolicylog4j.appender.fileAppender.policies.size.size=10MBlog4j.appender.fileAppender.strategy.type
> = DefaultRolloverStrategylog4j.appender.fileAppender.strategy.max = 5
>
>
> Thank you,
> Manas Kale
>
>
>
>
>
>
