[
https://issues.apache.org/jira/browse/FLINK-32392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736984#comment-17736984
]
Qingsheng Ren commented on FLINK-32392:
---------------------------------------
Some investigations:
* The case only happens on Microsoft-hosted agents, whose limit on disk space
is 10GB.
* The local Maven repo (.m2) is cached across pipeline runs, which is ~15GB
currently, and it kept growing in the past year.
I think the root cause is that the cached local Maven repo has a lot of
outdated dependencies and never get cleared. I created a PR to invalidate the
cache per year so that the size of local Maven repo won't grow indefinitely.
> Several jobs failed on AZP with No space left on device
> -------------------------------------------------------
>
> Key: FLINK-32392
> URL: https://issues.apache.org/jira/browse/FLINK-32392
> Project: Flink
> Issue Type: Bug
> Components: Test Infrastructure
> Affects Versions: 1.18.0, 1.16.3, 1.17.2
> Reporter: Sergey Nuyanzin
> Priority: Critical
> Labels: pull-request-available, test-stability
>
> This Build failed with no space left
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=50162&view=logs&j=585d8b77-fa33-51bc-8163-03e54ba9ce5b
> {noformat}
> ##[error]Unhandled exception. System.IO.IOException: No space left on device
> : '/home/vsts/agents/3.220.5/_diag/Worker_20230619-021757-utc.log'
> at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle,
> ReadOnlySpan`1 buffer, Int64 fileOffset)
> at System.IO.Strategies.BufferedFileStreamStrategy.FlushWrite()
> at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
> at System.Diagnostics.TextWriterTraceListener.Flush()
> at
> Microsoft.VisualStudio.Services.Agent.HostTraceListener.WriteHeader(String
> source, TraceEventType eventType, Int32 id) in
> /home/vsts/work/1/s/src/Microsoft.VisualStudio.Services.Agent/HostTraceListener.cs:line
> 151
> at
> Microsoft.VisualStudio.Services.Agent.HostTraceListener.TraceEvent(TraceEventCache
> eventCache, String source, TraceEventType eventType, Int32 id, String
> message) in
> /home/vsts/work/1/s/src/Microsoft.VisualStudio.Services.Agent/HostTraceListener.cs:line
> 81
> at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType,
> Int32 id, String message)
> at
> Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ProcessExitedHandler(Object
> sender, EventArgs e) in
> /home/vsts/work/1/s/src/Agent.Sdk/ProcessInvoker.cs:line 496
> at System.Diagnostics.Process.OnExited()
> at System.Diagnostics.Process.RaiseOnExited()
> at System.Diagnostics.Process.CompletionCallback(Object waitHandleContext,
> Boolean wasSignaled)
> at
> System.Threading._ThreadPoolWaitOrTimerCallback.WaitOrTimerCallback_Context_f(Object
> state)
> at System.Threading.ExecutionContext.RunInternal(ExecutionContext
> executionContext, ContextCallback callback, Object state)
> --- End of stack trace from previous location ---
> at
> System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(_ThreadPoolWaitOrTimerCallback
> helper, Boolean timedOut)
> at System.Threading.PortableThreadPool.CompleteWait(RegisteredWaitHandle
> handle, Boolean timedOut)
> at System.Threading.ThreadPoolWorkQueue.Dispatch()
> at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
> ,##[error]The hosted runner encountered an error while running your job.
> (Error Type: Failure).
> {noformat}
> for 1.16, 1.17 it happens while 'Upload artifacts to S3'
> for 1.18 while 'Deploy maven snapshot'
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
