[jira] [Commented] (AVRO-2202) Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when combined with musl libc static builds
[ https://issues.apache.org/jira/browse/AVRO-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811094#comment-16811094 ] Hudson commented on AVRO-2202: -- SUCCESS: Integrated in Jenkins build AvroJava #643 (See [https://builds.apache.org/job/AvroJava/643/]) [AVRO-2202] Make sure -g is not removed for debug builds, only add (dkulp: [https://github.com/apache/avro/commit/9839e2a67373a5cfd700182a15f949822ce3df99]) * (edit) lang/c++/CMakeLists.txt * (edit) lang/c++/build.sh > Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when > combined with musl libc static builds > > > Key: AVRO-2202 > URL: https://issues.apache.org/jira/browse/AVRO-2202 > Project: Apache Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.8.2 > Environment: OS: Gentoo Linux > Compiler: Gcc 6.4.0 targeting musl libc 1.19 with stdlibc++ in a static build >Reporter: Josh Scoggins >Assignee: Daniel Kulp >Priority: Major > Fix For: 1.9.0 > > > Observed with master and version 1.8.2. Inside of the C++ implementation's > CMakeLists.txt the lines 56-60 were added sometime in 2016 which activate > features inside of libstdc++ to help find errors. This change has two side > effects: > # -g is eliminated from the build so debugging gdb becomes harder as source > position data has been removed > # The safety containers that implicitly wrap all standard containers somehow > (I can't figure out exactly how) cause memory corruption when setMetadata is > called (in version 1.8.2) or when a Writer object is destroyed (on master). > Since our software uses musl libc with libstdc++ I believe that it is an > interaction with these debugging features which cause memory corruption (as I > said, I have no clue _why_ it is happening, only that these features _cause_ > it :() > Previously, we were using 1.7.7 which did not have these lines in its > CMakeLists.txt and thus did not exhibit any sort of memory corruption. I was > able to work around this by removing lines 56-60 from CMakeLists.txt so our > static libs could be built and our unit tests not crash by writing to memory > it did not own. There should be a toggle or something similar to turn off > these features for debug builds as they are for developers of avro only. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2202) Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when combined with musl libc static builds
[ https://issues.apache.org/jira/browse/AVRO-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811050#comment-16811050 ] ASF subversion and git services commented on AVRO-2202: --- Commit 9839e2a67373a5cfd700182a15f949822ce3df99 in avro's branch refs/heads/master from Daniel Kulp [ https://gitbox.apache.org/repos/asf?p=avro.git;h=9839e2a ] [AVRO-2202] Make sure -g is not removed for debug builds, only add GLIBCXX_DEBUG flags for our test builds > Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when > combined with musl libc static builds > > > Key: AVRO-2202 > URL: https://issues.apache.org/jira/browse/AVRO-2202 > Project: Apache Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.8.2 > Environment: OS: Gentoo Linux > Compiler: Gcc 6.4.0 targeting musl libc 1.19 with stdlibc++ in a static build >Reporter: Josh Scoggins >Assignee: Thiruvalluvan M. G. >Priority: Major > Fix For: 1.9.0 > > > Observed with master and version 1.8.2. Inside of the C++ implementation's > CMakeLists.txt the lines 56-60 were added sometime in 2016 which activate > features inside of libstdc++ to help find errors. This change has two side > effects: > # -g is eliminated from the build so debugging gdb becomes harder as source > position data has been removed > # The safety containers that implicitly wrap all standard containers somehow > (I can't figure out exactly how) cause memory corruption when setMetadata is > called (in version 1.8.2) or when a Writer object is destroyed (on master). > Since our software uses musl libc with libstdc++ I believe that it is an > interaction with these debugging features which cause memory corruption (as I > said, I have no clue _why_ it is happening, only that these features _cause_ > it :() > Previously, we were using 1.7.7 which did not have these lines in its > CMakeLists.txt and thus did not exhibit any sort of memory corruption. I was > able to work around this by removing lines 56-60 from CMakeLists.txt so our > static libs could be built and our unit tests not crash by writing to memory > it did not own. There should be a toggle or something similar to turn off > these features for debug builds as they are for developers of avro only. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AVRO-2202) Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when combined with musl libc static builds
[
https://issues.apache.org/jira/browse/AVRO-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562161#comment-16562161
]
Josh Scoggins commented on AVRO-2202:
-
Well the corruption is the same each time we ran our unit tests. In fact, it is
a simple empty case where we just construct the object empty and then save it
to disk. Our structure is recursive so that may also contribute to this issue.
Here is the sample code snippet that _should_ exhibit the issue.
{code:c++}
template
void saveObject(const T& object, const std::string& path, byte* schemaData,
unsigned schemaLen) {
avro::ValidSchema cpxSchema = avro::compileJsonSchemaFromMemory(schemaData,
schemaLen);
avro::DataFileWriter writer(file.c_str(), cpxSchema, 16 * 1024, NULL_CODEC);
// in 1.8.2 the construction contains setMetadata which causes the issue
writer.write(object);
writer.flush();
writer.close();
// at the end of the function during the destruction of writer is where the
crash happens on master
}{code}
> Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when
> combined with musl libc static builds
>
>
> Key: AVRO-2202
> URL: https://issues.apache.org/jira/browse/AVRO-2202
> Project: Avro
> Issue Type: Bug
> Components: c++
>Affects Versions: 1.8.2
> Environment: OS: Gentoo Linux
> Compiler: Gcc 6.4.0 targeting musl libc 1.19 with stdlibc++ in a static build
>Reporter: Josh Scoggins
>Priority: Major
>
> Observed with master and version 1.8.2. Inside of the C++ implementation's
> CMakeLists.txt the lines 56-60 were added sometime in 2016 which activate
> features inside of libstdc++ to help find errors. This change has two side
> effects:
> # -g is eliminated from the build so debugging gdb becomes harder as source
> position data has been removed
> # The safety containers that implicitly wrap all standard containers somehow
> (I can't figure out exactly how) cause memory corruption when setMetadata is
> called (in version 1.8.2) or when a Writer object is destroyed (on master).
> Since our software uses musl libc with libstdc++ I believe that it is an
> interaction with these debugging features which cause memory corruption (as I
> said, I have no clue _why_ it is happening, only that these features _cause_
> it :()
> Previously, we were using 1.7.7 which did not have these lines in its
> CMakeLists.txt and thus did not exhibit any sort of memory corruption. I was
> able to work around this by removing lines 56-60 from CMakeLists.txt so our
> static libs could be built and our unit tests not crash by writing to memory
> it did not own. There should be a toggle or something similar to turn off
> these features for debug builds as they are for developers of avro only.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AVRO-2202) Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when combined with musl libc static builds
[ https://issues.apache.org/jira/browse/AVRO-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561152#comment-16561152 ] Thiruvalluvan M. G. commented on AVRO-2202: --- I suppose the memory corruption problem you mention is not deterministic. If it is, will it be possible to post sample code snippet that exhibits the problem? > Use of -fstack-protector or -D_GLIBCXX_DEBUG causes memory corruption when > combined with musl libc static builds > > > Key: AVRO-2202 > URL: https://issues.apache.org/jira/browse/AVRO-2202 > Project: Avro > Issue Type: Bug > Components: c++ >Affects Versions: 1.8.2 > Environment: OS: Gentoo Linux > Compiler: Gcc 6.4.0 targeting musl libc 1.19 with stdlibc++ in a static build >Reporter: Josh Scoggins >Priority: Major > > Observed with master and version 1.8.2. Inside of the C++ implementation's > CMakeLists.txt the lines 56-60 were added sometime in 2016 which activate > features inside of libstdc++ to help find errors. This change has two side > effects: > # -g is eliminated from the build so debugging gdb becomes harder as source > position data has been removed > # The safety containers that implicitly wrap all standard containers somehow > (I can't figure out exactly how) cause memory corruption when setMetadata is > called (in version 1.8.2) or when a Writer object is destroyed (on master). > Since our software uses musl libc with libstdc++ I believe that it is an > interaction with these debugging features which cause memory corruption (as I > said, I have no clue _why_ it is happening, only that these features _cause_ > it :() > Previously, we were using 1.7.7 which did not have these lines in its > CMakeLists.txt and thus did not exhibit any sort of memory corruption. I was > able to work around this by removing lines 56-60 from CMakeLists.txt so our > static libs could be built and our unit tests not crash by writing to memory > it did not own. There should be a toggle or something similar to turn off > these features for debug builds as they are for developers of avro only. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
