I used a bounded source:
TextIO.Read.from("./demo.txt”);
Timeout is not configured
I am confused why the waitUntilFinish() is not finished while the spark job is
actually done.
Best regards,
bluejoe
在 2017/5/10 下午7:52，“Jean-Baptiste
Onofré”<[email protected] 代表 [email protected]>
写入:
>Hi,
>
>What's your source (unbounded or bounded) ?
>Did you define a timeout ?
>
>Regards
>JB
>
>On 05/10/2017 01:18 PM, bluejoe wrote:
>> Hi,
>> I create a beam pipeline in a Scala program, using the SparkRunner as runner
>> with sparkMaster=local[4]
>>
>> However, when executing the following code:
>>
>> pipeline.run().waitUntilFinish();
>>
>>
>> The program does not terminate even if the console displays:
>>
>>
>> 17/05/10 17:45:40 INFO SparkContext: Successfully stopped SparkContext
>>
>> 17/05/10 17:45:40 INFO RemoteActorRefProvider$RemotingTerminator: Shutting
>> down
>> remote daemon.
>>
>> 17/05/10 17:45:40 INFO RemoteActorRefProvider$RemotingTerminator: Remote
>> daemon
>> shut down; proceeding with flushing remote transports.
>>
>> 17/05/10 17:45:40 INFO RemoteActorRefProvider$RemotingTerminator: Remoting
>> shut
>> down.
>>
>>
>>
>> If I use pipeline in a right way?
>> Can anybody tell me how to let the program terminate normally?
>>
>> Best regards,
>> bluejoe
>
>--
>Jean-Baptiste Onofré
>[email protected]
>http://blog.nanthrax.net
>Talend - http://www.talend.com
