[
https://issues.apache.org/jira/browse/BEAM-11027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-11027:
--------------------------------
Description:
The execution of the Nexmark ZetaSQL tests takes too long and produces a
gigantic log file > 600MB fulls of debug information of grpc when run on the
Spark runner
{quote}{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND HEADERS:
streamId=39861 headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0
endStream=true}}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND WINDOW_UPDATE: streamId=0
windowSizeIncrement=151}}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] OUTBOUND HEADERS: streamId=39867
headers=GrpcHttp2OutboundHeaders[:authority: 0.0.0.0:0, :path:
/zetasql.local_service.ZetaSqlLocalService/Evaluate, :method: POST, :scheme:
http, content-type: application/grpc, te: trailers, user-agent:
grpc-java-netty/1.18.0, grpc-accept-encoding: gzip] streamDependency=0
weight=16 exclusive=false padding=0 endStream=false}}
{quote}
The full run in the CI takes 2h30 because of this
https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Spark_PR/52/
This can be reproduced by doing:
./gradlew :sdks:java:testing:nexmark:run -Pnexmark.runner=":runners:spark"
-Pnexmark.args=" --runner=SparkRunner --streaming=false --suite=SMOKE
--queryLanguage=zetaSql --manageResources=false --monitorJobs=true
--enforceEncodability=true --enforceImmutability=true"
was:
The execution of the Nexmark ZetaSQL tests takes too long and produces a
gigantic log file > 600MB fulls of debug information of grpc when run on the
Spark runner
{quote}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND HEADERS: streamId=39861
headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0 endStream=true}}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND WINDOW_UPDATE: streamId=0
windowSizeIncrement=151}}
{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id: 0x555bd369,
L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] OUTBOUND HEADERS: streamId=39867
headers=GrpcHttp2OutboundHeaders[:authority: 0.0.0.0:0, :path:
/zetasql.local_service.ZetaSqlLocalService/Evaluate, :method: POST, :scheme:
http, content-type: application/grpc, te: trailers, user-agent:
grpc-java-netty/1.18.0, grpc-accept-encoding: gzip] streamDependency=0
weight=16 exclusive=false padding=0 endStream=false}}{quote}
This can be reproduced by doing:
./gradlew :sdks:java:testing:nexmark:run -Pnexmark.runner=":runners:spark"
-Pnexmark.args=" --runner=SparkRunner --streaming=false --suite=SMOKE
--queryLanguage=zetaSql --manageResources=false --monitorJobs=true
--enforceEncodability=true --enforceImmutability=true"
> ZetaSQL Nexmark run takes too long because of excessive logging
> ---------------------------------------------------------------
>
> Key: BEAM-11027
> URL: https://issues.apache.org/jira/browse/BEAM-11027
> Project: Beam
> Issue Type: Test
> Components: dsl-sql-zetasql, testing-nexmark
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: P2
>
> The execution of the Nexmark ZetaSQL tests takes too long and produces a
> gigantic log file > 600MB fulls of debug information of grpc when run on the
> Spark runner
> {quote}{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
> 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND HEADERS:
> streamId=39861 headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0
> endStream=true}}
> {{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
> 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND WINDOW_UPDATE:
> streamId=0 windowSizeIncrement=151}}
> {{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
> 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] OUTBOUND HEADERS:
> streamId=39867 headers=GrpcHttp2OutboundHeaders[:authority: 0.0.0.0:0, :path:
> /zetasql.local_service.ZetaSqlLocalService/Evaluate, :method: POST, :scheme:
> http, content-type: application/grpc, te: trailers, user-agent:
> grpc-java-netty/1.18.0, grpc-accept-encoding: gzip] streamDependency=0
> weight=16 exclusive=false padding=0 endStream=false}}
> {quote}
> The full run in the CI takes 2h30 because of this
> https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Spark_PR/52/
> This can be reproduced by doing:
>
> ./gradlew :sdks:java:testing:nexmark:run -Pnexmark.runner=":runners:spark"
> -Pnexmark.args=" --runner=SparkRunner --streaming=false --suite=SMOKE
> --queryLanguage=zetaSql --manageResources=false --monitorJobs=true
> --enforceEncodability=true --enforceImmutability=true"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
