[
https://issues.apache.org/jira/browse/YUNIKORN-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu-Lin Chen resolved YUNIKORN-2559.
-----------------------------------
Fix Version/s: 1.6.0
Resolution: Fixed
Merge to master.
> DATA RACE: EventStore.Store() and Context.PublishEvents()
> ----------------------------------------------------------
>
> Key: YUNIKORN-2559
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2559
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - common
> Reporter: Yu-Lin Chen
> Assignee: Yu-Lin Chen
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
> Attachments: shim-racing-log.txt
>
>
> How to reproduce:
> # In shim, update core version to the latest version
> (v0.0.0-20240415111844-72540e2b277f)
> # go mod tidy
> # Run 'make test > shim-racing-log.txt'
>
> {code:java}
> WARNING: DATA RACE
> Write at 0x00c003882008 by goroutine 59:
> github.com/apache/yunikorn-core/pkg/events.(*EventStore).Store()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_store.go:59
> +0x1aa
>
> github.com/apache/yunikorn-core/pkg/events.(*EventSystemImpl).StartServiceWithPublisher.func2()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_system.go:194
> +0x167Previous read at 0x00c003882008 by goroutine 60:
> github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).PublishEvents()
> /home/chenyulin0719/yunikorn/yunikorn-k8shim/pkg/cache/context.go:1176
> +0x97
> github.com/apache/yunikorn-k8shim/pkg/cache.(*AsyncRMCallback).SendEvent()
>
> /home/chenyulin0719/yunikorn/yunikorn-k8shim/pkg/cache/scheduler_callback.go:235
> +0xec
>
> github.com/apache/yunikorn-core/pkg/events.(*EventPublisher).StartService.func1()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_publisher.go:60
> +0x27dGoroutine 59 (running) created at:
>
> github.com/apache/yunikorn-core/pkg/events.(*EventSystemImpl).StartServiceWithPublisher()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_system.go:183
> +0x287
> github.com/apache/yunikorn-core/pkg/events.(*EventSystemImpl).StartService()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_system.go:166
> +0x2b
>
> github.com/apache/yunikorn-core/pkg/entrypoint.startAllServicesWithParameters()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/entrypoint/entrypoint.go:80
> +0x9b
> github.com/apache/yunikorn-core/pkg/entrypoint.StartAllServices()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/entrypoint/entrypoint.go:43
> +0x59
> github.com/apache/yunikorn-k8shim/pkg/shim.(*MockScheduler).init()
>
> /home/chenyulin0719/yunikorn/yunikorn-k8shim/pkg/shim/scheduler_mock_test.go:63
> +0xad
> github.com/apache/yunikorn-k8shim/pkg/shim.TestApplicationScheduling()
>
> /home/chenyulin0719/yunikorn/yunikorn-k8shim/pkg/shim/scheduler_test.go:60
> +0x8c
> testing.tRunner()
> /usr/local/go/src/testing/testing.go:1689 +0x21e
> testing.(*T).Run.gowrap1()
> /usr/local/go/src/testing/testing.go:1742 +0x44Goroutine 60 (running)
> created at:
> github.com/apache/yunikorn-core/pkg/events.(*EventPublisher).StartService()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_publisher.go:50
> +0xc4
>
> github.com/apache/yunikorn-core/pkg/events.(*EventSystemImpl).StartServiceWithPublisher()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_system.go:203
> +0x2b8
> github.com/apache/yunikorn-core/pkg/events.(*EventSystemImpl).StartService()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/events/event_system.go:166
> +0x2b
>
> github.com/apache/yunikorn-core/pkg/entrypoint.startAllServicesWithParameters()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/entrypoint/entrypoint.go:80
> +0x9b
> github.com/apache/yunikorn-core/pkg/entrypoint.StartAllServices()
>
> /home/chenyulin0719/go/pkg/mod/github.com/apache/[email protected]/pkg/entrypoint/entrypoint.go:43
> +0x59
> github.com/apache/yunikorn-k8shim/pkg/shim.(*MockScheduler).init()
>
> /home/chenyulin0719/yunikorn/yunikorn-k8shim/pkg/shim/scheduler_mock_test.go:63
> +0xad
> github.com/apache/yunikorn-k8shim/pkg/shim.TestApplicationScheduling()
>
> /home/chenyulin0719/yunikorn/yunikorn-k8shim/pkg/shim/scheduler_test.go:60
> +0x8c
> testing.tRunner()
> /usr/local/go/src/testing/testing.go:1689 +0x21e
> testing.(*T).Run.gowrap1()
> /usr/local/go/src/testing/testing.go:1742 +0x44
> ================== {code}
> {*}Root cause{*}:
> EventStore.events were read/write by 2 different goroutine.
> * Goroutine 1: EventPublisher.StartService() call sp.store.CollectEvents(),
> return reference of es.events[:es.idx]
> * Goroutine 2: EventSystemImpl.StartServiceWithPublisher() call
> ec.Store.Store(event) to update es.events[es.idx] = event
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
