Re: [osg-submissions] Multi-threaded usage of textures

[Anish Thomas]

Hi Ulrich,

The updating of a texture's images happen before the image's modified-count is 
updated by the dirtying thread. This means that even if a third thread updates 
the image in the mean-time, this new update doesn't get missed from being 
uploaded to VRAM as the texture-upload happens *after* the modified count is 
read into the texture from its image. Therefore the updated texture in VRAM 
should reflect the 2nd update as well even if the modified-count is only from 
the 1st update. The only drawback I see is that a redundant texture upload 
could occur in a future texture apply().

Of course, since textures were not meant for multi-threaded updates, the 
application I'm working on uses a central texture container which prevents more 
than one DBPager thread from updating the same texture simultaneously. The fix 
was only to also synchronize the RenderThread too.

Cheers,
Anish


Ulrich Hertlein wrote:
> Hi Anish,
> 
> Looking at the fix I don't think this works in all cases, it only makes it 
> less likely to
> occur.
> 
> The check and the set are still done non-atomically, so another thread could 
> be scheduled
> inbetween and one update would be lost.
> 
> My preferred solution would be to use std::atomic with exchange for the 
> modifiedCount, but
> I don't know if that's available across all compilers that OSG supports.  
> It's been in
> since C++11 though so I would hope it's safe to use.
> 
> Cheers,
> /ulrich
> 
> On 19/10/17 07:31, Anish Thomas wrote:
> 
> > Hi,
> > 
> > I've attached the original and the fixed versions of the texture-handling 
> > files based on the head of the 3.2 branch.
> > 
> > Thank you!
> > 
> > Cheers,
> > Anish
> > 
> 
> 
> ___
> osg-submissions mailing list
> 
> http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org
> 
>  --
> Post generated by Mail2Forum


--
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72251#72251





___
osg-submissions mailing list
osg-submissions@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org

Re: [osg-submissions] Multi-threaded usage of textures

[Ulrich Hertlein]

Hi Anish,

Looking at the fix I don't think this works in all cases, it only makes it less 
likely to
occur.

The check and the set are still done non-atomically, so another thread could be 
scheduled
inbetween and one update would be lost.

My preferred solution would be to use std::atomic with exchange for the 
modifiedCount, but
I don't know if that's available across all compilers that OSG supports.  It's 
been in
since C++11 though so I would hope it's safe to use.

Cheers,
/ulrich

On 19/10/17 07:31, Anish Thomas wrote:
> Hi,
> 
> I've attached the original and the fixed versions of the texture-handling 
> files based on the head of the 3.2 branch.
> 
> Thank you!
> 
> Cheers,
> Anish



signature.asc
Description: OpenPGP digital signature
___
osg-submissions mailing list
osg-submissions@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org

Re: [osg-submissions] Multi-threaded usage of textures

[Robert Osfield]

H Anish,

Thanks for the fixes, I've merged them with the OpenSceneGraph-3.2 branch
and then applied similar changes to OpenSceneGraph-3.4 branch and master, I
couldn't apply the changes directly to these as the code base has changed a
little too much to make a direct merge easy.

Could you test out which versions you can and let me know if things are now
working.

Thanks,
Robert.

On 19 October 2017 at 06:31, Anish Thomas  wrote:

> Hi,
>
> I've attached the original and the fixed versions of the texture-handling
> files based on the head of the 3.2 branch.
>
> Thank you!
>
> Cheers,
> Anish
>
> --
> Read this topic online here:
> http://forum.openscenegraph.org/viewtopic.php?p=72199#72199
>
>
>
>
> ___
> osg-submissions mailing list
> osg-submissions@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-
> submissions-openscenegraph.org
>
>
___
osg-submissions mailing list
osg-submissions@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org

Re: [osg-submissions] Multi-threaded usage of textures

[Eric Sokolowsky]

This could apply to a problem I have been having in my own application.
Please post your fix once you have it.

On Oct 17, 2017 5:01 AM, "Anish Thomas"  wrote:

> Hi Robert,
>
> I can submit a fix then.
> Thanks for the quick reply.
>
> Cheers,
> Anish
>
>
> robertosfield wrote:
> > Hi Anish,
> >
> >
> >
> > I think you are correct, the way to handle concurrrent reading/updates
> of an osg::Image is to take a local copy of the
> osg::Image::getModifedCount() prior to reading from the image and then use
> this value.  This would still have the danger of the image data being
> updated at the same time as it's being read but at least a texture update
> wouldn't be missed.
> >
> >
> > Robert.
> >
> >
> >
> > On 17 October 2017 at 09:38, Anish Thomas < ()> wrote:
> >
> > > Hi,
> > >
> > > While modifying texture-atlases in DBPager threads we noticed an issue
> of thread-synchronization.
> > >
> > > Code like the snippet below (From Texture2D.cpp):
> > >
> > > else if (_image.valid() && getModifiedCount(contextID) !=
> _image->getModifiedCount())
> > > {
> > > applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
> > > _textureWidth, _textureHeight,
> _internalFormat, _numMipmapLevels);
> > >
> > > // update the modified tag to show that it is up to date.
> > > getModifiedCount(contextID) = _image->getModifiedCount();
> > >
> > > }
> > >
> > > If another thread (like a DBPager thread) modifies the image of a
> texture (in this case a texture-atlas which gets more and more images added
> to it while it is also being applied on geometry), the modified count
> changes made by one or more dirty() calls can get missed.
> > >
> > > If the image got dirtied before the RenderThread tried to apply this
> texture, code-execution enters the if() described above. If the texture
> upload takes too long and the image is dirtied again, the effect of the 2nd
> dirty can be missed when the modified count of the image is put into the
> texture.
> > >
> > > A simple solution would be to use a local variable to store the
> image's modified-count, and apply it to the texture. The overhead of a
> possible redundant texture-upload on the next apply would be down-side. But
> texture-atlases would support multi-threaded updates better.
> > >
> > > Thoughts?
> > >
> > > Thank you!
> > >
> > > Cheers,
> > > Anish
> > >
> > > --
> > > Read this topic online here:
> > > http://forum.openscenegraph.org/viewtopic.php?p=72188#72188 (
> http://forum.openscenegraph.org/viewtopic.php?p=72188#72188)
> > >
> > >
> > >
> > >
> > >
> > > ___
> > > osg-submissions mailing list
> > >  ()
> > > http://lists.openscenegraph.org/listinfo.cgi/osg-
> submissions-openscenegraph.org (http://lists.openscenegraph.
> org/listinfo.cgi/osg-submissions-openscenegraph.org)
> > >
> >
> >
> >  --
> > Post generated by Mail2Forum
>
>
> --
> Read this topic online here:
> http://forum.openscenegraph.org/viewtopic.php?p=72191#72191
>
>
>
>
>
> ___
> osg-submissions mailing list
> osg-submissions@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-
> submissions-openscenegraph.org
>
___
osg-submissions mailing list
osg-submissions@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org

Re: [osg-submissions] Multi-threaded usage of textures

[Anish Thomas]

Hi Robert,

I can submit a fix then.
Thanks for the quick reply.

Cheers,
Anish


robertosfield wrote:
> Hi Anish,
> 
> 
> 
> I think you are correct, the way to handle concurrrent reading/updates of an 
> osg::Image is to take a local copy of the osg::Image::getModifedCount() prior 
> to reading from the image and then use this value.  This would still have the 
> danger of the image data being updated at the same time as it's being read 
> but at least a texture update wouldn't be missed.
> 
> 
> Robert.
> 
> 
> 
> On 17 October 2017 at 09:38, Anish Thomas < ()> wrote:
> 
> > Hi,
> > 
> > While modifying texture-atlases in DBPager threads we noticed an issue of 
> > thread-synchronization.
> > 
> > Code like the snippet below (From Texture2D.cpp):
> > 
> > else if (_image.valid() && getModifiedCount(contextID) != 
> > _image->getModifiedCount())
> > {
> >             applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
> >                                     _textureWidth, _textureHeight, 
> > _internalFormat, _numMipmapLevels);
> > 
> >             // update the modified tag to show that it is up to date.
> >             getModifiedCount(contextID) = _image->getModifiedCount();
> > 
> > }
> > 
> > If another thread (like a DBPager thread) modifies the image of a texture 
> > (in this case a texture-atlas which gets more and more images added to it 
> > while it is also being applied on geometry), the modified count changes 
> > made by one or more dirty() calls can get missed.
> > 
> > If the image got dirtied before the RenderThread tried to apply this 
> > texture, code-execution enters the if() described above. If the texture 
> > upload takes too long and the image is dirtied again, the effect of the 2nd 
> > dirty can be missed when the modified count of the image is put into the 
> > texture.
> > 
> > A simple solution would be to use a local variable to store the image's 
> > modified-count, and apply it to the texture. The overhead of a possible 
> > redundant texture-upload on the next apply would be down-side. But 
> > texture-atlases would support multi-threaded updates better.
> > 
> > Thoughts?
> > 
> > Thank you!
> > 
> > Cheers,
> > Anish
> > 
> > --
> > Read this topic online here:
> > http://forum.openscenegraph.org/viewtopic.php?p=72188#72188 
> > (http://forum.openscenegraph.org/viewtopic.php?p=72188#72188)
> > 
> > 
> > 
> > 
> > 
> > ___
> > osg-submissions mailing list
> >  ()
> > http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org
> >  
> > (http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org)
> > 
> 
> 
>  --
> Post generated by Mail2Forum


--
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=72191#72191





___
osg-submissions mailing list
osg-submissions@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org

Re: [osg-submissions] Multi-threaded usage of textures

[Robert Osfield]

Hi Anish,

I think you are correct, the way to handle concurrrent reading/updates of
an osg::Image is to take a local copy of the osg::Image::getModifedCount()
prior to reading from the image and then use this value.  This would still
have the danger of the image data being updated at the same time as it's
being read but at least a texture update wouldn't be missed.

Robert.

On 17 October 2017 at 09:38, Anish Thomas  wrote:

> Hi,
>
> While modifying texture-atlases in DBPager threads we noticed an issue of
> thread-synchronization.
>
> Code like the snippet below (From Texture2D.cpp):
>
> else if (_image.valid() && getModifiedCount(contextID) !=
> _image->getModifiedCount())
> {
> applyTexImage2D_subload(state,GL_TEXTURE_2D,_image.get(),
> _textureWidth, _textureHeight,
> _internalFormat, _numMipmapLevels);
>
> // update the modified tag to show that it is up to date.
> getModifiedCount(contextID) = _image->getModifiedCount();
>
> }
>
> If another thread (like a DBPager thread) modifies the image of a texture
> (in this case a texture-atlas which gets more and more images added to it
> while it is also being applied on geometry), the modified count changes
> made by one or more dirty() calls can get missed.
>
> If the image got dirtied before the RenderThread tried to apply this
> texture, code-execution enters the if() described above. If the texture
> upload takes too long and the image is dirtied again, the effect of the 2nd
> dirty can be missed when the modified count of the image is put into the
> texture.
>
> A simple solution would be to use a local variable to store the image's
> modified-count, and apply it to the texture. The overhead of a possible
> redundant texture-upload on the next apply would be down-side. But
> texture-atlases would support multi-threaded updates better.
>
> Thoughts?
>
> Thank you!
>
> Cheers,
> Anish
>
> --
> Read this topic online here:
> http://forum.openscenegraph.org/viewtopic.php?p=72188#72188
>
>
>
>
>
> ___
> osg-submissions mailing list
> osg-submissions@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-
> submissions-openscenegraph.org
>
___
osg-submissions mailing list
osg-submissions@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org