Thanks Chesnay for the tip. I'll try to investigate the usage of GlobalJobParameters.
On Wed, Jul 15, 2020 at 2:51 PM Chesnay Schepler <ches...@apache.org> wrote: > The more we strive towards a model where an application can submit > multiple jobs it will become increasingly important to be able to attach > meta data to a job/application to have any idea what is going on. > > But I don't think the PackagedProgram/ProgramDescription is the way to > go; and I'd envision rather something like a meta data object that is > attached to the environment/execute calls. But we have to figure out how > to do this in a way that also works for the SQL APIs. > > What we have done internally is to encode such information in the > GlobalJobParameters which are then available in the WebUI. We have > things like commit IDs encoded into the jar manifest, that we extract at > submission time and put them into the parameters. > My guess would be that such approach can work sufficiently for all > dataset/datastream/table API users. > > On 15/07/2020 14:05, Flavio Pompermaier wrote: > > Ok, it's not a problem for me if the community is not interested in > pushing > > this thing forward. > > When we develop a Job is super useful for us to have the job describing > > itself somehow (what it does and which parameters it requires). > > If this is not in Flink I have to implement it somewhere else but I can't > > think that this is not a common situation. > > However I think I can live with it :D > > > > Best, > > Flavio > > > > On Wed, Jul 15, 2020 at 12:01 PM Aljoscha Krettek <aljos...@apache.org> > > wrote: > > > >> I think no-one is interested to push this personally right now. We would > >> need a champion that is interested and pushes this forward. > >> > >> Best, > >> Aljoscha > >> > >> On Mon, Mar 30, 2020, at 12:45, Flavio Pompermaier wrote: > >>> I would personally like to see a way of describing a Flink job/pipeline > >>> (including its parameters and types) in order to enable better UIs, > then > >>> the important thing is to make things consistent and aligned with the > new > >>> client developments and exploit this new dev sprint to fix such issues. > >>> > >>> On Mon, Mar 30, 2020 at 11:38 AM Aljoscha Krettek <aljos...@apache.org > > > >>> wrote: > >>> > >>>> On 18.03.20 14:45, Flavio Pompermaier wrote: > >>>>> what do you think if we exploit this job-submission sprint to address > >>>> also > >>>>> the problem discussed in > >>>> https://issues.apache.org/jira/browse/FLINK-10862? > >>>> > >>>> That's a good idea! What should we do? It seems that most committers > on > >>>> the issue were in favour of deprecating/removing ProgramDescription. > >>>> >
