Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19589
to look at the new patch set (#2).
Change subject: IMPALA-11974: Fix lazy list operators for Python 3 compatibility
......................................................................
IMPALA-11974: Fix lazy list operators for Python 3 compatibility
Python 3 changes list operators such as range, map, and filter
to be lazy. Some code that expects the list operators to happen
immediately will fail. e.g.
Python 2:
range(0,5) == [0,1,2,3,4]
True
Python 3:
range(0,5) == [0,1,2,3,4]
False
The fix is to wrap locations with list(). i.e.
Python 3:
list(range(0,5)) == [0,1,2,3,4]
True
Since the base operators are now lazy, Python 3 also removes the
old lazy versions (e.g. xrange, ifilter, izip, etc). This uses
future's builtins package to convert the code to the Python 3
behavior (i.e. xrange -> future's builtins.range).
Most of the changes were done via these futurize fixes:
- libfuturize.fixes.fix_xrange_with_import
- lib2to3.fixes.fix_map
- lib2to3.fixes.fix_filter
This eliminates the pylint warnings:
- xrange-builtin
- range-builtin-not-iterating
- map-builtin-not-iterating
- zip-builtin-not-iterating
- filter-builtin-not-iterating
- reduce-builtin
- deprecated-itertools-function
Testing:
- Ran core job
Change-Id: Ic7c082711f8eff451a1b5c085e97461c327edb5f
---
M bin/banned_py3k_warnings.txt
M bin/generate_minidump_collection_testdata.py
M bin/get_code_size.py
M bin/load-data.py
M bin/run-workload.py
M bin/single_node_perf_run.py
M bin/start-impala-cluster.py
M testdata/bin/generate-schema-statements.py
M testdata/bin/generate-test-vectors.py
M testdata/bin/load_nested.py
M testdata/bin/random_avro_schema.py
M testdata/bin/rewrite-iceberg-metadata.py
M testdata/common/cgroups.py
M testdata/common/text_delims_table.py
M testdata/common/widetable.py
M tests/authorization/test_ranger.py
M tests/beeswax/impala_beeswax.py
M tests/benchmark/plugins/vtune_plugin.py
M tests/benchmark/report_benchmark_results.py
M tests/common/environ.py
M tests/common/impala_cluster.py
M tests/common/impala_test_suite.py
M tests/common/kudu_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_result_verifier.py
M tests/comparison/cluster.py
M tests/comparison/data_generator.py
M tests/comparison/data_generator_mapred_common.py
M tests/comparison/db_connection.py
M tests/comparison/discrepancy_searcher.py
M tests/comparison/funcs.py
M tests/comparison/query.py
M tests/comparison/query_generator.py
M tests/comparison/query_profile.py
M tests/comparison/statement_generator.py
M tests/conftest.py
M tests/custom_cluster/test_admission_controller.py
M tests/custom_cluster/test_auto_scaling.py
M tests/custom_cluster/test_blacklist.py
M tests/custom_cluster/test_breakpad.py
M tests/custom_cluster/test_codegen_cache.py
M tests/custom_cluster/test_concurrent_ddls.py
M tests/custom_cluster/test_concurrent_kudu_create.py
M tests/custom_cluster/test_custom_statestore.py
M tests/custom_cluster/test_events_custom_configs.py
M tests/custom_cluster/test_exchange_deferred_batches.py
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_hdfs_fd_caching.py
M tests/custom_cluster/test_incremental_metadata_updates.py
M tests/custom_cluster/test_local_catalog.py
M tests/custom_cluster/test_mem_reservations.py
M tests/custom_cluster/test_metadata_replicas.py
M tests/custom_cluster/test_metastore_service.py
M tests/custom_cluster/test_parquet_max_page_header.py
M tests/custom_cluster/test_preload_table_types.py
M tests/custom_cluster/test_process_failures.py
M tests/custom_cluster/test_query_expiration.py
M tests/custom_cluster/test_query_retries.py
M tests/custom_cluster/test_restart_services.py
M tests/custom_cluster/test_rpc_timeout.py
M tests/custom_cluster/test_scratch_disk.py
M tests/custom_cluster/test_set_and_unset.py
M tests/custom_cluster/test_topic_update_frequency.py
M tests/custom_cluster/test_udf_concurrency.py
M tests/custom_cluster/test_wide_table_operations.py
M tests/failure/test_failpoints.py
M tests/hs2/hs2_test_suite.py
M tests/hs2/test_fetch_first.py
M tests/hs2/test_hs2.py
M tests/metadata/test_compute_stats.py
M tests/metadata/test_ddl.py
M tests/metadata/test_hms_integration.py
M tests/metadata/test_load.py
M tests/metadata/test_recover_partitions.py
M tests/metadata/test_recursive_listing.py
M tests/metadata/test_stats_extrapolation.py
M tests/performance/scheduler.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_avro_schema_resolution.py
M tests/query_test/test_cancellation.py
M tests/query_test/test_cast_with_format.py
M tests/query_test/test_compressed_formats.py
M tests/query_test/test_decimal_casting.py
M tests/query_test/test_decimal_fuzz.py
M tests/query_test/test_exprs.py
M tests/query_test/test_hdfs_caching.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_insert_behaviour.py
M tests/query_test/test_insert_parquet.py
M tests/query_test/test_insert_permutation.py
M tests/query_test/test_kudu.py
M tests/query_test/test_mem_usage_scaling.py
M tests/query_test/test_parquet_bloom_filter.py
M tests/query_test/test_scanners.py
M tests/query_test/test_scanners_fuzz.py
M tests/query_test/test_sort.py
M tests/query_test/test_tpch_queries.py
M tests/shell/test_shell_commandline.py
M tests/statestore/test_statestore.py
M tests/stress/concurrent_select.py
M tests/stress/queries.py
M tests/stress/query_retries_stress_runner.py
M tests/stress/test_acid_stress.py
M tests/stress/test_ddl_stress.py
M tests/stress/test_insert_stress.py
M tests/util/calculation_util.py
M tests/util/concurrent_workload.py
M tests/util/get_parquet_metadata.py
M tests/util/ssh_util.py
M tests/util/test_file_parser.py
110 files changed, 338 insertions(+), 229 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/19589/2
--
To view, visit http://gerrit.cloudera.org:8080/19589
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic7c082711f8eff451a1b5c085e97461c327edb5f
Gerrit-Change-Number: 19589
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
