Default timeout with no flag may work as well. The main consideration here is whether some api calls may take longer than 60 seconds because of the complexity of the users' Beam pipeline. E.g. Could job_service.Prepare() take longer than 60 seconds if the given Beam pipeline is extremely complex?
Basically if there are cases when the user code may cause the call duration to increase to the point the timeout prevents submitting the app itself then we should consider having a flag. On 2019/08/07 20:13:12, Ahmet Altay wrote: > Could we pick a default timeout value instead of introducing a flag? We use> > 60 seconds as the default timeout for http client [1], we can do the same> > here.> > > [1]> > https://github.com/apache/beam/blob/3a182d64c86ad038692800f5c343659ab0b935b0/sdks/python/apache_beam/internal/http_client.py#L32 > > > > On Wed, Aug 7, 2019 at 11:53 AM enrico canzonieri > > wrote:> > > > Hello,> > >> > > I noticed that the calls to the JobServer from the Python SDK do not have> > > timeouts. If I'm not mistaken that means that the call to pipeline.run ( > > http://pipeline.run/ ) ()> > > could hang forever if the JobServer is not running (or failing to start).> > > E.g.> > > https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/portable_runner.py#L307 > > > > > the call to Prepare() doesn't provide any timeout value and the same> > > applies to other JobServer requests.> > > I was considering adding a --job-server-request-timeout to the> > > PortableOptions> > > > > > class to be used in the JobServer interactions inside probable_runner.py ( > > http://probable_runner.py/ ).> > > Is there any specific reason why the timeout is not currently supported?> > > Does anybody have any objection adding the jobserver timeout? I could> > > volunteer to file a ticket and submit a pr for this.> > >> > > Cheers,> > > Enrico Canzonieri> > >> >
