[
https://issues.apache.org/jira/browse/IMPALA-7662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639127#comment-16639127
]
Tim Armstrong commented on IMPALA-7662:
---------------------------------------
Here's the coordinator log:
{noformat}
I1001 07:46:39.096261 11913 impala-server.cc:1060] Registered query
query_id=59468009b1b99760:7fc7cd9f00000000
session_id=6445a8f391197f4d:c3b48a41b6e55d8c
I1001 07:46:39.096377 11913 Frontend.java:1088] Analyzing query: SELECT * from
bad_magic_number
I1001 07:46:39.096596 11913 Frontend.java:1127] Analysis finished.
I1001 07:46:39.096870 16830 impala-beeswax-server.cc:197]
get_results_metadata(): query_id=784d05a0eec24937:26b3a3f300000000
I1001 07:46:39.098851 26968 admission-controller.cc:552] Schedule for
id=59468009b1b99760:7fc7cd9f00000000 in pool_name=default-pool
cluster_mem_needed=32.00 MB PoolConfig: max_requests=-1 max_queued=200
max_mem=-1.00 B
I1001 07:46:39.098872 26968 admission-controller.cc:557] Stats:
agg_num_running=5, agg_num_queued=0, agg_mem_reserved=531.04 MB,
local_host(local_mem_admitted=9.30 GB, num_admitted_running=5, num_queued=0,
backend_mem_reserved=193.51 MB)
I1001 07:46:39.098884 26968 admission-controller.cc:589] Admitted query
id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.098914 26968 coordinator.cc:91] Exec()
query_id=59468009b1b99760:7fc7cd9f00000000 stmt=SELECT * from bad_magic_number
I1001 07:46:39.099181 26968 coordinator.cc:329] starting execution on 2
backends for query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.101203 26958 query-state.cc:482] Executing instance.
instance_id=f14a0994894edbad:393c6b480000000a fragment_idx=1
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=14
I1001 07:46:39.102165 32382 impala-internal-service.cc:49]
ExecQueryFInstances(): query_id=59468009b1b99760:7fc7cd9f00000000
coord=impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22000
#instances=1
I1001 07:46:39.102475 26974 query-state.cc:482] Executing instance.
instance_id=59468009b1b99760:7fc7cd9f00000000 fragment_idx=0
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=15
I1001 07:46:39.110245 26968 coordinator.cc:343] started execution on 2 backends
for query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.115803 26972 query-state.cc:482] Executing instance.
instance_id=f14a0994894edbad:393c6b4800000004 fragment_idx=2
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=16
I1001 07:46:39.116829 26980 query-state.cc:482] Executing instance.
instance_id=f14a0994894edbad:393c6b4800000001 fragment_idx=3
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=17
I1001 07:46:39.131021 26982 query-state.cc:482] Executing instance.
instance_id=f14a0994894edbad:393c6b4800000007 fragment_idx=4
per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=18
I1001 07:46:39.151530 16830 coordinator.cc:456] ExecState: query
id=784d05a0eec24937:26b3a3f300000000 execution completed
I1001 07:46:39.151551 16830 coordinator.cc:581] Coordinator waiting for
backends to finish, 3 remaining. query_id=784d05a0eec24937:26b3a3f300000000
I1001 07:46:39.156179 10036 coordinator.cc:684] Backend completed:
host=impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22000
remaining=3 query_id=784d05a0eec24937:26b3a3f300000000
I1001 07:46:39.156194 10036 coordinator-backend-state.cc:254]
query_id=784d05a0eec24937:26b3a3f300000000: first in-progress backend:
impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22001
I1001 07:46:39.157794 26846 query-state.cc:333] Cancelling fragment instances
as directed by the coordinator. Returned status: Cancelled
I1001 07:46:39.157804 26846 query-state.cc:503] Cancel:
query_id=784d05a0eec24937:26b3a3f300000000
I1001 07:46:39.157812 26846 krpc-data-stream-mgr.cc:325] cancelling all streams
for fragment_instance_id=784d05a0eec24937:26b3a3f300000001
I1001 07:46:39.157819 26846 krpc-data-stream-mgr.cc:325] cancelling all streams
for fragment_instance_id=784d05a0eec24937:26b3a3f300000000
I1001 07:46:39.157897 26846 krpc-data-stream-mgr.cc:294] DeregisterRecvr():
fragment_instance_id=784d05a0eec24937:26b3a3f300000000, node=1
I1001 07:46:39.157995 26846 query-state.cc:490] Instance completed.
instance_id=784d05a0eec24937:26b3a3f300000000 #in-flight=17 status=OK
I1001 07:46:39.229135 23161 coordinator.cc:684] Backend completed:
host=impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22001
remaining=2 query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.229149 23161 coordinator-backend-state.cc:254]
query_id=59468009b1b99760:7fc7cd9f00000000: first in-progress backend:
impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22000
I1001 07:46:39.240075 3922 coordinator.cc:684] Backend completed:
host=impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22000
remaining=1 query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.240239 26974 krpc-data-stream-mgr.cc:294] DeregisterRecvr():
fragment_instance_id=59468009b1b99760:7fc7cd9f00000000, node=1
I1001 07:46:39.240372 26974 query-state.cc:490] Instance completed.
instance_id=59468009b1b99760:7fc7cd9f00000000 #in-flight=16 status=OK
I1001 07:46:39.253808 20164 coordinator.cc:684] Backend completed:
host=impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22001
remaining=2 query_id=784d05a0eec24937:26b3a3f300000000
I1001 07:46:39.253823 20164 coordinator-backend-state.cc:254]
query_id=784d05a0eec24937:26b3a3f300000000: first in-progress backend:
impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22002
I1001 07:46:39.260221 11913 impala-beeswax-server.cc:197]
get_results_metadata(): query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.260561 11913 coordinator.cc:456] ExecState: query
id=59468009b1b99760:7fc7cd9f00000000 execution completed
I1001 07:46:39.260578 11913 coordinator.cc:789] Release admission control
resources for query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.267756 11913 impala-beeswax-server.cc:239] close():
query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.267765 11913 impala-server.cc:1127] UnregisterQuery():
query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.267771 11913 impala-server.cc:1238] Cancel():
query_id=59468009b1b99760:7fc7cd9f00000000
I1001 07:46:39.267787 11913 coordinator.cc:657] CancelBackends()
query_id=59468009b1b99760:7fc7cd9f00000000, tried to cancel 0 backends
I1001 07:46:39.281966 11913 query-exec-mgr.cc:179] ReleaseQueryState(): deleted
query_id=59468009b1b99760:7fc7cd9f00000000
{noformat}
Here's the executor log:
{noformat}
I1001 07:46:39.107010 32375 impala-internal-service.cc:49]
ExecQueryFInstances(): query_id=59468009b1b99760:7fc7cd9f00000000
coord=impala-ec2-centos74-m5-4xlarge-ondemand-02c3.vpc.cloudera.com:22000
#instances=1
I1001 07:46:39.111327 26977 query-state.cc:482] Executing instance.
instance_id=59468009b1b99760:7fc7cd9f00000001 fragment_idx=1
per_fragment_instance_idx=0 coord_state_idx=1 #in-flight=13
I1001 07:46:39.218391 26986 status.cc:73] File
'hdfs://localhost:20500/test-warehouse/bad_magic_number_parquet/bad_magic_number.parquet'
has an invalid version number: XXXX
This could be due to stale metadata. Try running "refresh
functional_parquet.bad_magic_number".
@ 0x199dde2
@ 0x2222366
@ 0x22181c1
@ 0x21a934f
@ 0x219e13d
@ 0x219d5ea
@ 0x219cacf
@ 0x219eeb9
@ 0x1cb3e8d
@ 0x20ee076
@ 0x20f6396
@ 0x20f62ba
@ 0x20f627d
@ 0x33a37d9
@ 0x7f02a4d7be24
@ 0x7f02a4aa934c
I1001 07:46:39.218456 26986 hdfs-scan-node.cc:487] Error preparing scanner for
scan range
hdfs://localhost:20500/test-warehouse/bad_magic_number_parquet/bad_magic_number.parquet(0:241).
I1001 07:46:39.229508 26977 query-state.cc:490] Instance completed.
instance_id=59468009b1b99760:7fc7cd9f00000001 #in-flight=12 status=OK
I1001 07:46:39.231272 26977 query-exec-mgr.cc:179] ReleaseQueryState(): deleted
query_id=59468009b1b99760:7fc7cd9f00000000
{noformat}
> test_parquet reads bad_magic_number.parquet without an error
> ------------------------------------------------------------
>
> Key: IMPALA-7662
> URL: https://issues.apache.org/jira/browse/IMPALA-7662
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Environment: Impala ddef2cb9b14e7f8cf9a68a2a382e10a8e0f91c3d
> exhaustive debug build
> Reporter: Tianyi Wang
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: correctness
>
> {noformat}
> 09:51:41 =================================== FAILURES
> ===================================
> 09:51:41 TestParquet.test_parquet[exec_option: {'batch_size': 0,
> 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen':
> False, 'abort_on_error': 1, 'debug_action':
> 'HDFS_SCANNER_THREAD_CHECK_SOFT_MEM_LIMIT:[email protected]',
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> 09:51:41 [gw5] linux2 -- Python 2.7.5
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/bin/../infra/python/env/bin/python
> 09:51:41 query_test/test_scanners.py:300: in test_parquet
> 09:51:41 self.run_test_case('QueryTest/parquet', vector)
> 09:51:41 common/impala_test_suite.py:423: in run_test_case
> 09:51:41 assert False, "Expected exception: %s" % expected_str
> 09:51:41 E AssertionError: Expected exception: File
> 'hdfs://localhost:20500/test-warehouse/bad_magic_number_parquet/bad_magic_number.parquet'
> has an invalid version number: XXXX
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
