Re:Re:Re: Re: Re: Cancellation logic in HdfsScanners

Hi Tim, I believe it's a bug and find out a way to reproduce it. Have filed a JIRA: https://issues.apache.org/jira/browse/IMPALA-6423 At 2018-01-17 08:53:44, "Quanlong Huang" wrote: >Thanks, Tim! Let me try to reproduce this scenario on existing scanners. I'll >file a

Re: Re: Re: Cancellation logic in HdfsScanners

I think there is still probably a bug in the existing scanners where they can ignore cancellation under specific conditions. > For non-MT scanners, why don't they just check about > RuntimeState::is_cancelled()? Are there any reasons that they should go ahead until HdfsScanNode::done()? I think

Re:Re: Re: Cancellation logic in HdfsScanners

I'm developing the hdfs orc scanner (IMPALA-5717) and encountered such scenario in test_failpoints.py. The existing scanners can pass this test. I think this might be my own problem so I haven't filed a JIRA yet. Just want to confirm that when setting MT_DOP=0, other scanners won't get into

Re: Re: Cancellation logic in HdfsScanners

Would you be able to file a JIRA? On Tue, Jan 16, 2018 at 3:00 PM, Tim Armstrong wrote: > Looks to me like you found a bug. I think the scanners should be checking > both cancellation conditions, i.e. RuntimeState::is_cancelled_ for MT and > non-MT scanners and

Re: Re: Cancellation logic in HdfsScanners

Looks to me like you found a bug. I think the scanners should be checking both cancellation conditions, i.e. RuntimeState::is_cancelled_ for MT and non-MT scanners and hdfs_scan_node::done_ for non-MT scanners. On Tue, Jan 16, 2018 at 2:48 PM, Quanlong Huang wrote: > Hi