flink 1.7.2 YARN Session????????????????????

2020-04-13 文章 Chief 
??
??flink 1.7.2??YARN Session??Hadoop 2.7.3??hdfs 
namenode??haHADOOP_HOME,YARN_CONF_DIR??HADOOP_CONF_DIR??HADOOP_CLASSPATHflink_conf.yamlfs.hdfs.hadoopconf


2020-04-10 19:12:02,908 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- Connecting to ResourceManager 
akka.tcp://flink@trusfortpoc1:23584/user/resourcemanager()
2020-04-10 19:12:02,909 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Cannot serve slot request, no ResourceManager connected. Adding as 
pending request [SlotRequestId{0feacbb4fe16c8c7a70249f1396565d0}]
2020-04-10 19:12:02,911 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- Resolved ResourceManager address, beginning 
registration
2020-04-10 19:12:02,911 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- Registration at ResourceManager attempt 1 
(timeout=100ms)
2020-04-10 19:12:02,912 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Cannot serve slot request, no ResourceManager connected. Adding as 
pending request [SlotRequestId{35ad2384e9cd0efd30b43f5302db24b6}]
2020-04-10 19:12:02,913 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Registering job manager 
0...@akka.tcp://flink@trusfortpoc1:23584/user/jobmanager_0
 for job 24691b33c18d7ad73b1f52edb3d68ae4.
2020-04-10 19:12:02,917 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Registered job manager 
0...@akka.tcp://flink@trusfortpoc1:23584/user/jobmanager_0
 for job 24691b33c18d7ad73b1f52edb3d68ae4.
2020-04-10 19:12:02,919 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- JobManager successfully registered at 
ResourceManager, leader id: .
2020-04-10 19:12:02,919 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Requesting new slot [SlotRequestId{35ad2384e9cd0efd30b43f5302db24b6}] 
and profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, 
directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource 
manager.
2020-04-10 19:12:02,920 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Request slot with profile 
ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0, 
nativeMemoryInMB=0, networkMemoryInMB=0} for job 
24691b33c18d7ad73b1f52edb3d68ae4 with allocation id 
AllocationID{5a12237c7f2bd8b1cc760ddcbab5a1c0}.
2020-04-10 19:12:02,921 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Requesting new slot [SlotRequestId{0feacbb4fe16c8c7a70249f1396565d0}] 
and profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, 
directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource 
manager.
2020-04-10 19:12:02,924 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Requesting new TaskExecutor 
container with resources

flink 1.7.2 YARN Session????????????????????

2020-04-13 文章 Chief 
??
??flink 1.7.2??YARN Session??Hadoop 2.7.3??hdfs 
namenode??haHADOOP_HOME,YARN_CONF_DIR??HADOOP_CONF_DIR??HADOOP_CLASSPATHflink_conf.yamlfs.hdfs.hadoopconf


2020-04-10 19:12:02,908 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- Connecting to ResourceManager 
akka.tcp://flink@trusfortpoc1:23584/user/resourcemanager()
2020-04-10 19:12:02,909 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Cannot serve slot request, no ResourceManager connected. Adding as 
pending request [SlotRequestId{0feacbb4fe16c8c7a70249f1396565d0}]
2020-04-10 19:12:02,911 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- Resolved ResourceManager address, beginning 
registration
2020-04-10 19:12:02,911 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- Registration at ResourceManager attempt 1 
(timeout=100ms)
2020-04-10 19:12:02,912 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Cannot serve slot request, no ResourceManager connected. Adding as 
pending request [SlotRequestId{35ad2384e9cd0efd30b43f5302db24b6}]
2020-04-10 19:12:02,913 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Registering job manager 
0...@akka.tcp://flink@trusfortpoc1:23584/user/jobmanager_0
 for job 24691b33c18d7ad73b1f52edb3d68ae4.
2020-04-10 19:12:02,917 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Registered job manager 
0...@akka.tcp://flink@trusfortpoc1:23584/user/jobmanager_0
 for job 24691b33c18d7ad73b1f52edb3d68ae4.
2020-04-10 19:12:02,919 INFO 
org.apache.flink.runtime.jobmaster.JobMaster 
- JobManager successfully registered at 
ResourceManager, leader id: .
2020-04-10 19:12:02,919 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Requesting new slot [SlotRequestId{35ad2384e9cd0efd30b43f5302db24b6}] 
and profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, 
directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource 
manager.
2020-04-10 19:12:02,920 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Request slot with profile 
ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0, 
nativeMemoryInMB=0, networkMemoryInMB=0} for job 
24691b33c18d7ad73b1f52edb3d68ae4 with allocation id 
AllocationID{5a12237c7f2bd8b1cc760ddcbab5a1c0}.
2020-04-10 19:12:02,921 INFO 
org.apache.flink.runtime.jobmaster.slotpool.SlotPool
 - Requesting new slot [SlotRequestId{0feacbb4fe16c8c7a70249f1396565d0}] 
and profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, 
directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource 
manager.
2020-04-10 19:12:02,924 INFO 
org.apache.flink.yarn.YarnResourceManager 
 - Requesting new TaskExecutor 
container with resources

?????? flink 1.7.2 YARN Session????????????????????

2020-04-15 文章 Chief 
hi Yangze Guo
??checkpoint??namenode
 ha??nameservicecheckpoint




----
??:"Yangze Guo"https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/yarn_setup.html

Best,
Yangze Guo

On Mon, Apr 13, 2020 at 10:52 PM Chief

?????? ????flink sql 1.10 source????????????????????

2020-03-25 文章 Chief 
hi Kurt Young
hive??13??web ui 
source??150source




----
??:"Kurt Young"

?????? ????flink sql 1.10 source????????????????????

2020-03-25 文章 Chief 
hiJun Zhang






----
??:"Jun Zhang"

?????? ????flink sql 1.10 source????????????????????

2020-03-25 文章 Chief 
hi Zhenghua Gao
??


----
??:"Zhenghua Gao"

flink sql 1.10.0 ??hive GenericUDF????

2020-03-19 文章 Chief 
hi all??
?? hive ??flink sql 
1.10.0flink 
hive??GenericUDF??flink??
current_timestamp()
??hive??

   ??flink sql 
org.apache.flink.table.api.ValidationException: SQL validation failed. 
java.lang.reflect.InvocationTargetException
at 
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130)
at 
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertSqlInsert(SqlToOperationConverter.java:342)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:142)
at 
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
at 
org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlUpdate(TableEnvironmentImpl.java:484)
at 
com.trusfort.twinkling.sql.util.SqlSubmit.callInsertInto(SqlSubmit.java:55)
at 
com.trusfort.twinkling.sql.util.SqlSubmit.callCommand(SqlSubmit.java:27)
at 
com.trusfort.twinkling.sql.batch.template.BatchSlidingWindowSqlTemplate.main(BatchSlidingWindowSqlTemplate.java:60)
at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at 
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
at 
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
at 
org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
at 
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
at 
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
at 
java.security.AccessController.doPrivileged(Native Method)
at 
javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at 
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at 
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
Caused by: java.lang.RuntimeException: 
java.lang.reflect.InvocationTargetException
at 
org.apache.flink.table.planner.functions.utils.HiveFunctionUtils.invokeGetResultType(HiveFunctionUtils.java:77)
at 
org.apache.flink.table.planner.functions.utils.HiveScalarSqlFunction.lambda$createReturnTypeInference$0(HiveScalarSqlFunction.java:83)
at 
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:470)
at 
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:437)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:303)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
at 
org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
at 
org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:237)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
at 
org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
at 
org.apache.calcite.sql.type.InferTypes.lambda$static$0(InferTypes.java:46)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1865)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1853)
at

flink sql 1.10.0 ??hive GenericUDF????????????????????????????????????????

2020-03-19 文章 Chief 
hi all??
?? hive ??flink sql 
1.10.0flink 
hive??GenericUDF??flink??
current_timestamp()
??hive
select 
from_unixtime(unix_timestamp(current_timestamp()),'MMddHHmmss');
   ??flink sql 
org.apache.flink.table.api.ValidationException: SQL validation failed. 
java.lang.reflect.InvocationTargetException
at 
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130)
at 
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertSqlInsert(SqlToOperationConverter.java:342)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:142)
at 
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
at 
org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlUpdate(TableEnvironmentImpl.java:484)
at 
com.trusfort.twinkling.sql.util.SqlSubmit.callInsertInto(SqlSubmit.java:55)
at 
com.trusfort.twinkling.sql.util.SqlSubmit.callCommand(SqlSubmit.java:27)
at 
com.trusfort.twinkling.sql.batch.template.BatchSlidingWindowSqlTemplate.main(BatchSlidingWindowSqlTemplate.java:60)
at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at 
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
at 
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
at 
org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
at 
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
at 
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
at 
java.security.AccessController.doPrivileged(Native Method)
at 
javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at 
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at 
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
Caused by: java.lang.RuntimeException: 
java.lang.reflect.InvocationTargetException
at 
org.apache.flink.table.planner.functions.utils.HiveFunctionUtils.invokeGetResultType(HiveFunctionUtils.java:77)
at 
org.apache.flink.table.planner.functions.utils.HiveScalarSqlFunction.lambda$createReturnTypeInference$0(HiveScalarSqlFunction.java:83)
at 
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:470)
at 
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:437)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:303)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
at 
org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
at 
org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:237)
at 
org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
at 
org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
at 
org.apache.calcite.sql.type.InferTypes.lambda$static$0(InferTypes.java:46)
at 
org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1865)
at

????flink sql 1.10 source????????????????????

2020-03-24 文章 Chief 
hi all??
??flink sqlhivehive150sql 
client??10??source??150web 
ui