Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17771 )
Change subject: WiP: IMPALA-10798 : Prototype for JSON reader ...................................................................... Patch Set 25: (40 comments) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-json-scanner.cc File be/src/exec/hdfs-json-scanner.cc: http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-json-scanner.cc@220 PS25, Line 220: std::shared_ptr<arrow::Field> field_a = arrow::field(cvf.name(), ColumnType2ArrowType(cvf.type())); line too long (103 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-json-scanner.cc@262 PS25, Line 262: VLOG_QUERY << " PrintTemplateTuple:::" << PrintTuple(template_tuple, *tuple_desc) << std::endl; line too long (103 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-json-scanner.cc@285 PS25, Line 285: VLOG_QUERY<< "Column in GetNextInternal" << column->type()->ToString() << column->ToString(); line too long (99 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc File be/src/exec/hdfs-scan-node-base.cc: http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@70 PS25, Line 70: "for all reads, regardless of whether the read is local or remote. By default, the " line too long (126 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@71 PS25, Line 71: "IO data cache is only used if the data is expected to be remote. Used by tests."); line too long (125 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@79 PS25, Line 79: " across all Disk I/O threads in HDFS read operations."); line too long (112 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@81 PS25, Line 81: " spent across all Disk I/O threads in HDFS open operations."); line too long (122 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@84 PS25, Line 84: " while it is executing I/O operations on behalf of a scan."); line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@91 PS25, Line 91: "disks accessed by HDFS scan. Each local disk is counted as a disk and each type of" line too long (150 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@92 PS25, Line 92: " remote filesystem (e.g. HDFS remote reads, S3) is counted as a distinct disk."); line too long (148 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@94 PS25, Line 94: " average number of HDFS read threads executing read operations on behalf of this " line too long (161 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@95 PS25, Line 95: "scan. Higher values (i.e. close to the aggregate number of I/O threads across " line too long (158 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@96 PS25, Line 96: "all disks accessed) show that this scan is using a larger proportion of the I/O " line too long (160 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@97 PS25, Line 97: "capacity of the system. Lower values show that either this scan is not I/O bound" line too long (160 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@98 PS25, Line 98: " or that it is getting a small share of the I/O capacity of the system."); line too long (153 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@106 PS25, Line 106: "Use this to determine if the scan got all of the reservation it wanted. Does not " line too long (101 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@107 PS25, Line 107: "include subsequent reservation increases done by scanner implementation " line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@129 PS25, Line 129: "threads spent waiting for I/O. This value can be compared to the value of " line too long (128 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@130 PS25, Line 130: "ScannerThreadsTotalWallClockTime of MT_DOP = 0 scan nodes or otherwise compared " line too long (134 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@131 PS25, Line 131: "to the total time reported for MT_DOP > 0 scan nodes. High values show that " line too long (130 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@132 PS25, Line 132: "scanner threads are spending significant time waiting for I/O instead of " line too long (127 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@133 PS25, Line 133: "processing data. Note that this includes the time when the thread is runnable " line too long (132 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@137 PS25, Line 137: "Each sample in the counter is the size of a single column that is scanned by the " line too long (101 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@141 PS25, Line 141: "Each sample in the counter is the size of a single column that is scanned by the " line too long (101 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@353 PS25, Line 353: VLOG_QUERY<< hdfs_table_->partition_descriptors().size() << " " << hdfs_table_->DebugString() << " " << shared_state_.use_mt_scan_node_ << " " << instance_ctx_pbs.size() << " " << tnode_->hdfs_scan_node << GetStackTrace(); line too long (233 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@751 PS25, Line 751: metadata->partition_id, FilterStats::FILES_KEY, filter_ctxs, file, state)) { line too long (93 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1236 PS25, Line 1236: "Read $0 of data across network that was expected to be local. Block locality " line too long (144 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1237 PS25, Line 1237: "metadata for table '$1.$2' may be stale. This only affects query performance " line too long (144 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1238 PS25, Line 1238: "and not result correctness. One of the common causes for this warning is HDFS " line too long (145 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1239 PS25, Line 1239: "rebalancer moving some of the file's blocks. If the issue persists, consider " line too long (144 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1240 PS25, Line 1240: "running \"INVALIDATE METADATA `$1`.`$2`\".", line too long (110 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1241 PS25, Line 1241: PrettyPrinter::Print(unexpected_remote_bytes_->value(), TUnit::BYTES), line too long (135 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/be/src/exec/hdfs-scan-node-base.cc@1242 PS25, Line 1242: hdfs_table_->database(), hdfs_table_->name()))); line too long (113 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/bin/bootstrap_toolchain.py File bin/bootstrap_toolchain.py: http://gerrit.cloudera.org:8080/#/c/17771/25/bin/bootstrap_toolchain.py@489 PS25, Line 489: " flake8: E501 line too long (98 > 90 characters) http://gerrit.cloudera.org:8080/#/c/17771/25/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java: http://gerrit.cloudera.org:8080/#/c/17771/25/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@458 PS25, Line 458: line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/17771/25/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1405 PS25, Line 1405: 0, fileDesc.getFileLength(), partition.getId(), fileDesc.getFileLength(), line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1406 PS25, Line 1406: fileDesc.getFileCompression().toThrift(), fileDesc.getModificationTime(), line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1410 PS25, Line 1410: currentOffset, currentLength, partition.getId(), fileDesc.getFileLength(), line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@1411 PS25, Line 1411: fileDesc.getFileCompression().toThrift(), fileDesc.getModificationTime(), line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/17771/25/tests/query_test/test_tpch_queries.py File tests/query_test/test_tpch_queries.py: http://gerrit.cloudera.org:8080/#/c/17771/25/tests/query_test/test_tpch_queries.py@39 PS25, Line 39: s flake8: E501 line too long (96 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/17771 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If79364a421d862d0d837f9be694911e388d4d629 Gerrit-Change-Number: 17771 Gerrit-PatchSet: 25 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Comment-Date: Sun, 26 Mar 2023 16:28:22 +0000 Gerrit-HasComments: Yes
