[jira] [Commented] (DAFFODIL-1878) Parse and unparse files/sec values vary significantly using compiled and saved parsers
[ https://issues.apache.org/jira/browse/DAFFODIL-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355618#comment-16355618 ] Michael Beckerle commented on DAFFODIL-1878: Ok, so multi-thread interactions may be at work here. Are there variations like this in runtime if we run only 1 thread? If not then we have a strong hint. Can we get that data point too? > Parse and unparse files/sec values vary significantly using compiled and > saved parsers > -- > > Key: DAFFODIL-1878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1878 > Project: Daffodil > Issue Type: Bug > Components: General >Affects Versions: 2.1.0 >Reporter: Dave Thompson >Assignee: Steve Lawrence >Priority: Major > Fix For: 2.2.0 > > Attachments: Performance report for 01-19-2018.docx > > > After updating the nightly scripts to use the pre-compiled/saved parsers the > result show that for some tests there are significant performance value > differences between using runtime compiled parsers and the saved parsers. > Attached is the report email. The Previous Val column is from non-saved > parser run. Curr Val column is from previously compiled/saved parsers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1878) Parse and unparse files/sec values vary significantly using compiled and saved parsers
[ https://issues.apache.org/jira/browse/DAFFODIL-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355603#comment-16355603 ] Dave Thompson commented on DAFFODIL-1878: - Additional information: Noticed that there were significant variances in the jpeg_10t_1m_1000 test between nightly runs, even when there was no change to daffodil when using saved parser. Looked at the raw csv data file and found that the average files/sec varied greatly between test iterations (e.g. 30 files/sec to 145 files/sec). Executed the test (ten iterations) numerous times varying the number files processed. Found that as the files processed was lowered the files/sec performance lowered as did the difference between the max and min files/sec lowered but remained significant. Executed with 5 threads and got similar results, lower files/sec and lower but still excessive difference. The difference when executing the test using the runtime compiled parser/schema was usually between 2 – 5 files/sec. > Parse and unparse files/sec values vary significantly using compiled and > saved parsers > -- > > Key: DAFFODIL-1878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1878 > Project: Daffodil > Issue Type: Bug > Components: General >Affects Versions: 2.1.0 >Reporter: Dave Thompson >Assignee: Steve Lawrence >Priority: Major > Fix For: 2.2.0 > > Attachments: Performance report for 01-19-2018.docx > > > After updating the nightly scripts to use the pre-compiled/saved parsers the > result show that for some tests there are significant performance value > differences between using runtime compiled parsers and the saved parsers. > Attached is the report email. The Previous Val column is from non-saved > parser run. Curr Val column is from previously compiled/saved parsers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-1878) Parse and unparse files/sec values vary significantly using compiled and saved parsers
[ https://issues.apache.org/jira/browse/DAFFODIL-1878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339710#comment-16339710 ] Steve Lawrence commented on DAFFODIL-1878: -- I was able to put a hack together that allowed all the transient parameters to be set to null and thus all them to be garbage collected. Confirmed that the only references to compilation data structures in runtime objects are in the *RuntimeData and *DPathCompileInfo objects. Unfortunately, after allowing them to be garbage collected performance still remained the same. So the extra memory usage does not appear to be the cause of the slowdown. > Parse and unparse files/sec values vary significantly using compiled and > saved parsers > -- > > Key: DAFFODIL-1878 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1878 > Project: Daffodil > Issue Type: Bug > Components: General >Affects Versions: 2.1.0 >Reporter: Dave Thompson >Assignee: Steve Lawrence >Priority: Major > Fix For: 2.1.0 > > Attachments: Performance report for 01-19-2018.docx > > > After updating the nightly scripts to use the pre-compiled/saved parsers the > result show that for some tests there are significant performance value > differences between using runtime compiled parsers and the saved parsers. > Attached is the report email. The Previous Val column is from non-saved > parser run. Curr Val column is from previously compiled/saved parsers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
