Re: Re: failed when job graph change
Hi i adding a new sink that was not exists in the graph 1. stop with save point 2. run the the new graph with the new sink operator ( from save point) in this case the job stuck in initializing forever because cant complete transaction ( on the new operator , kafka topic) i dont understand how change the uid will help its a new operator with new uid nick בתאריך יום ד׳, 24 בינו׳ 2024 ב-19:30 מאת Feng Jin < jinfeng1...@gmail.com>: > Hi nick > > If you want to modify the sink operator , I think you can modify the uid > of the operator to avoid restoring the state that does not belong to it. > > > Best, > Feng > > > On Thu, Jan 25, 2024 at 1:19 AM nick toker > wrote: > >> hi >> >> i didn't found anything in the log >> but i found that it happened when i add a new sink operator >> and because i work with checkpoints the flink can't finish the >> transaction ( the new topic in kafka not part of the transaction before i >> added the new sink operator) >> >> so i must cancel the job to make it work >> >> How can I solve this issue? >> >> >> nick >> >> בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-10:27 מאת Xuyang <xyzhong...@163.com >> >: >> >>> Hi, >>> Can you attach the log about the exception when job failed? >>> >>> -- >>> Best！ >>> Xuyang >>> >>> >>> 在 2023-12-04 15:56:04，"nick toker" 写道： >>> >>> Hi >>> >>> restart the job it's ok and i do that , but i must cancel the job and >>> submit a new one and i dont want the data from the state >>> forget to mention that i use the parameter "-allowNonRestoredState" >>> >>> >>> my steps: >>> 1. stop the job with savepoint >>> 2. run the updated job ( update job graph) from savepoint >>> >>> expect it to run >>> >>> currently the result is the the job fail >>> >>> nick >>> >>> >>> >>> בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-8:41 מאת Xuyang <xyzhong...@163.com >>> >: >>> Hi, nick. > using savepoint i must cancel the job to be able run the new graph Do you mean that you need cancel and start the job using the new flink job graph in 1.17.1, and in the past, it was able to make the changes to the new operator effective without restarting the job? I think in order for the new job graph to take effect, it is necessary to restart the job. -- Best！ Xuyang At 2023-12-03 21:49:23, "nick toker" wrote: Hi when i add or remove an operator in the job graph , using savepoint i must cancel the job to be able run the new graph e.g. by adding or removing operator (like new sink target) it was working in the past i using flink 1.17.1 1. is it a known bug? if so when planned to be fix 2. do i need to do something to make it work? nick
Re: Re: failed when job graph change
Hi nick If you want to modify the sink operator , I think you can modify the uid of the operator to avoid restoring the state that does not belong to it. Best, Feng On Thu, Jan 25, 2024 at 1:19 AM nick toker wrote: > hi > > i didn't found anything in the log > but i found that it happened when i add a new sink operator > and because i work with checkpoints the flink can't finish the > transaction ( the new topic in kafka not part of the transaction before i > added the new sink operator) > > so i must cancel the job to make it work > > How can I solve this issue? > > > nick > > בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-10:27 מאת Xuyang <xyzhong...@163.com > >: > >> Hi, >> Can you attach the log about the exception when job failed? >> >> -- >> Best！ >> Xuyang >> >> >> 在 2023-12-04 15:56:04，"nick toker" 写道： >> >> Hi >> >> restart the job it's ok and i do that , but i must cancel the job and >> submit a new one and i dont want the data from the state >> forget to mention that i use the parameter "-allowNonRestoredState" >> >> >> my steps: >> 1. stop the job with savepoint >> 2. run the updated job ( update job graph) from savepoint >> >> expect it to run >> >> currently the result is the the job fail >> >> nick >> >> >> >> בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-8:41 מאת Xuyang <xyzhong...@163.com >> >: >> >>> Hi, nick. >>> >>> > using savepoint i must cancel the job to be able run the new graph >>> >>> Do you mean that you need cancel and start the job using the new flink >>> job graph in 1.17.1, >>> and in the past, it was able to make the changes to the new operator >>> effective without restarting the job? >>> >>> I think in order for the new job graph to take effect, it is necessary >>> to restart the job. >>> >>> -- >>> Best！ >>> Xuyang >>> >>> >>> At 2023-12-03 21:49:23, "nick toker" wrote: >>> >>> Hi >>> >>> when i add or remove an operator in the job graph , using savepoint i >>> must cancel the job to be able run the new graph >>> >>> e.g. by adding or removing operator (like new sink target) >>> it was working in the past >>> i using flink 1.17.1 >>> >>> 1. is it a known bug? if so when planned to be fix >>> >>> 2. do i need to do something to make it work? >>> >>> >>> nick >>> >>>
Re: Re: failed when job graph change
hi i didn't found anything in the log but i found that it happened when i add a new sink operator and because i work with checkpoints the flink can't finish the transaction ( the new topic in kafka not part of the transaction before i added the new sink operator) so i must cancel the job to make it work How can I solve this issue? nick בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-10:27 מאת Xuyang <xyzhong...@163.com>: > Hi, > Can you attach the log about the exception when job failed? > > -- > Best！ > Xuyang > > > 在 2023-12-04 15:56:04，"nick toker" 写道： > > Hi > > restart the job it's ok and i do that , but i must cancel the job and > submit a new one and i dont want the data from the state > forget to mention that i use the parameter "-allowNonRestoredState" > > > my steps: > 1. stop the job with savepoint > 2. run the updated job ( update job graph) from savepoint > > expect it to run > > currently the result is the the job fail > > nick > > > > בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-8:41 מאת Xuyang <xyzhong...@163.com > >: > >> Hi, nick. >> >> > using savepoint i must cancel the job to be able run the new graph >> >> Do you mean that you need cancel and start the job using the new flink >> job graph in 1.17.1, >> and in the past, it was able to make the changes to the new operator >> effective without restarting the job? >> >> I think in order for the new job graph to take effect, it is necessary to >> restart the job. >> >> -- >> Best！ >> Xuyang >> >> >> At 2023-12-03 21:49:23, "nick toker" wrote: >> >> Hi >> >> when i add or remove an operator in the job graph , using savepoint i >> must cancel the job to be able run the new graph >> >> e.g. by adding or removing operator (like new sink target) >> it was working in the past >> i using flink 1.17.1 >> >> 1. is it a known bug? if so when planned to be fix >> >> 2. do i need to do something to make it work? >> >> >> nick >> >>
Re: failed when job graph change
Hi restart the job it's ok and i do that , but i must cancel the job and submit a new one and i dont want the data from the state forget to mention that i use the parameter "-allowNonRestoredState" my steps: 1. stop the job with savepoint 2. run the updated job ( update job graph) from savepoint expect it to run currently the result is the the job fail nick בתאריך יום ב׳, 4 בדצמ׳ 2023 ב-8:41 מאת Xuyang <xyzhong...@163.com>: > Hi, nick. > > > using savepoint i must cancel the job to be able run the new graph > > Do you mean that you need cancel and start the job using the new flink job > graph in 1.17.1, > and in the past, it was able to make the changes to the new operator > effective without restarting the job? > > I think in order for the new job graph to take effect, it is necessary to > restart the job. > > -- > Best！ > Xuyang > > > At 2023-12-03 21:49:23, "nick toker" wrote: > > Hi > > when i add or remove an operator in the job graph , using savepoint i must > cancel the job to be able run the new graph > > e.g. by adding or removing operator (like new sink target) > it was working in the past > i using flink 1.17.1 > > 1. is it a known bug? if so when planned to be fix > > 2. do i need to do something to make it work? > > > nick > >
