Ismaël Mejía created BEAM-11027:
-----------------------------------
Summary: ZetaSQL Nexmark run takes too long because of excessive
logging
Key: BEAM-11027
URL: https://issues.apache.org/jira/browse/BEAM-11027
Project: Beam
Issue Type: Test
Components: dsl-sql-zetasql, testing-nexmark
Reporter: Ismaël Mejía
Assignee: Ismaël Mejía
The execution of the Nexmark ZetaSQL tests takes too long and produces a
gigantic log file > 600MB fulls of debug information of grpc when run on the
Spark runner
{quote}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND HEADERS: streamId=39861
headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0 endStream=true}}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND WINDOW_UPDATE: streamId=0
windowSizeIncrement=151}}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] OUTBOUND HEADERS: streamId=39867
headers=GrpcHttp2OutboundHeaders[:authority: 0.0.0.0:0, :path:
/zetasql.local_service.ZetaSqlLocalService/Evaluate, :method: POST, :scheme:
http, content-type: application/grpc, te: trailers, user-agent:
grpc-java-netty/1.18.0, grpc-accept-encoding: gzip] streamDependency=0
weight=16 exclusive=false padding=0 endStream=false}}{quote}
This can be reproduced by doing:
./gradlew :sdks:java:testing:nexmark:run -Pnexmark.runner=":runners:spark"
-Pnexmark.args=" --runner=SparkRunner --streaming=false --suite=SMOKE
--queryLanguage=zetaSql --manageResources=false --monitorJobs=true
--enforceEncodability=true --enforceImmutability=true"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
