[
https://issues.apache.org/jira/browse/BEAM-11027?focusedWorklogId=496204&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496204
]
ASF GitHub Bot logged work on BEAM-11027:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Oct/20 21:32
Start Date: 06/Oct/20 21:32
Worklog Time Spent: 10m
Work Description: iemejia removed a comment on pull request #13023:
URL: https://github.com/apache/beam/pull/13023#issuecomment-704564202
Run Spark ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 496204)
Time Spent: 0.5h (was: 20m)
> ZetaSQL Nexmark run takes too long because of excessive logging
> ---------------------------------------------------------------
>
> Key: BEAM-11027
> URL: https://issues.apache.org/jira/browse/BEAM-11027
> Project: Beam
> Issue Type: Test
> Components: dsl-sql-zetasql, testing-nexmark
> Reporter: Ismaël Mejía
> Assignee: Ismaël Mejía
> Priority: P2
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The execution of the Nexmark ZetaSQL tests takes too long and produces a
> gigantic log file > 600MB fulls of debug information of grpc when run on the
> Spark runner
> {quote}{{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
> 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND HEADERS:
> streamId=39861 headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0
> endStream=true}}
> {{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
> 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] INBOUND WINDOW_UPDATE:
> streamId=0 windowSizeIncrement=151}}
> {{DEBUG com.google.zetasql.io.grpc.netty.NettyClientHandler: [id:
> 0x555bd369, L:0.0.0.0/0.0.0.0:0 - R:0.0.0.0/0.0.0.0:0] OUTBOUND HEADERS:
> streamId=39867 headers=GrpcHttp2OutboundHeaders[:authority: 0.0.0.0:0, :path:
> /zetasql.local_service.ZetaSqlLocalService/Evaluate, :method: POST, :scheme:
> http, content-type: application/grpc, te: trailers, user-agent:
> grpc-java-netty/1.18.0, grpc-accept-encoding: gzip] streamDependency=0
> weight=16 exclusive=false padding=0 endStream=false}}
> {quote}
> The full run in the CI takes 2h30 because of this
> https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Spark_PR/52/
> This can be reproduced by doing:
>
> ./gradlew :sdks:java:testing:nexmark:run -Pnexmark.runner=":runners:spark"
> -Pnexmark.args=" --runner=SparkRunner --streaming=false --suite=SMOKE
> --queryLanguage=zetaSql --manageResources=false --monitorJobs=true
> --enforceEncodability=true --enforceImmutability=true"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
