[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails
[ https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258237#comment-13258237 ] Karl Wright edited comment on CONNECTORS-347 at 4/20/12 1:27 PM: - A postgresql run is as follows: {code} testdb=# select dochash,docid from jobqueue; dochash | docid --+- EE6C0ED002CE9303183FEFEF111D6598203F6295 | C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\testdir 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\test2.txt 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\test1.txt 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\testdir\seconddir 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\testdir\test3.txt (6 rows) testdb=# select parentidhash, childidhash from intrinsiclink; parentidhash | childidhash --+-- EE6C0ED002CE9303183FEFEF111D6598203F6295 | 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | EE6C0ED002CE9303183FEFEF111D6598203F6295 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | EE6C0ED002CE9303183FEFEF111D6598203F6295 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | EE6C0ED002CE9303183FEFEF111D6598203F6295 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 7013D738BB77ED47763449997384046B20B77832 | 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 (7 rows) testdb=# select parentidhash, distance from hopcount; parentidhash | distance --+-- EE6C0ED002CE9303183FEFEF111D6598203F6295 |0 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 |1 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 |1 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 |1 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 |2 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 |2 7013D738BB77ED47763449997384046B20B77832 |3 (7 rows) testdb=# {code} was (Author: kwri...@metacarta.com): A postgresql run is as follows: {code} testdb=# select dochash,docid from jobqueue; dochash | docid --+- EE6C0ED002CE9303183FEFEF111D6598203F6295 | C:\wip\mcf\trunk\tests\filesystem\te st-postgresql-output\testdata E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | C:\wip\mcf\trunk\tests\filesystem\te st-postgresql-output\testdata\testdir 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | C:\wip\mcf\trunk\tests\filesystem\te st-postgresql-output\testdata\test2.txt 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | C:\wip\mcf\trunk\tests\filesystem\te st-postgresql-output\testdata\test1.txt 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | C:\wip\mcf\trunk\tests\filesystem\te st-postgresql-output\testdata\testdir\seconddir 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | C:\wip\mcf\trunk\tests\filesystem\te st-postgresql-output\testdata\testdir\test3.txt (6 rows) testdb=# select parentidhash, childidhash from intrinsiclink; parentidhash | childidhash --+- - EE6C0ED002CE9303183FEFEF111D6598203F6295 | 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | EE6C0ED002CE9303183FEFEF111D6598203F 6295 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | EE6C0ED002CE9303183FEFEF111D6598203F 6295 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | EE6C0ED002CE9303183FEFEF111D6598203F 6295 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | E7DE0B8D3F153CECFD3C57B4D83E59073EC4 DD60 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | E7DE0B8D3F153CECFD3C57B4D83E59073EC4 DD60 7013D738BB77ED47763449997384046B20B77832 | 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4 C666 (7 rows) testdb=# select parentidhash, distance from hopcount; parentidhash | distance --+-- EE6C0ED002CE9303183FEFEF111D6598203F6295 |0 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 |1 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 |1 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 |1 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 |2 4ED8AFCC917B604C1EE4C83458DB9B7265967F40
[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails
[ https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258202#comment-13258202 ] Karl Wright edited comment on CONNECTORS-347 at 4/20/12 12:58 PM: -- Here's a postmortem for a run that succeeded: {code} mysql> select parentidhash,distance from hopcount; +--+--+ | parentidhash | distance | +--+--+ | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |0 | | 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 |3 | | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 |2 | | B622AAB99E7BFF749E2C93E01009C3E02F90693C |2 | | 9F6F59234CC36DF341184FAEC934BEEE516E1E60 |1 | | CD31925B98C4B481F6523030A1C7C9D9DF0CC080 |1 | | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |1 | +--+--+ 7 rows in set (0.00 sec) mysql> select parentidhash,childidhash from intrinsiclink; +--+--+ | parentidhash | childidhash | +--+--+ | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | | | 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | | 9F6F59234CC36DF341184FAEC934BEEE516E1E60 | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | | B622AAB99E7BFF749E2C93E01009C3E02F90693C | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | | CD31925B98C4B481F6523030A1C7C9D9DF0CC080 | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | +--+--+ 7 rows in set (0.00 sec) mysql> select dochash,docid from jobqueue; +--++ | dochash | docid | +--++ | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata | | B622AAB99E7BFF749E2C93E01009C3E02F90693C | C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\testdir\test3.txt | | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\testdir\seconddir | | CD31925B98C4B481F6523030A1C7C9D9DF0CC080 | C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\test1.txt | | 9F6F59234CC36DF341184FAEC934BEEE516E1E60 | C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\test2.txt | | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\testdir | +--++ 6 rows in set (0.00 sec) {code} Even in this one, the jobqueue table is inconsistent with the hopcount and intrinsiclink tables. They should all be consistent. was (Author: kwri...@metacarta.com): Here's a postmortem for a run that succeeded: {code} mysql> select parentidhash,distance from hopcount; +--+--+ | parentidhash | distance | +--+--+ | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |0 | | 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 |3 | | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 |2 | | B622AAB99E7BFF749E2C93E01009C3E02F90693C |2 | | 9F6F59234CC36DF341184FAEC934BEEE516E1E60 |1 | | CD31925B98C4B481F6523030A1C7C9D9DF0CC080 |1 | | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |1 | +--+--+ 7 rows in set (0.00 sec) mysql> select parentidhash,childidhash from intrinsiclink; +--+--+ | parentidhash | childidhash | +--+--+ | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | | 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | | | 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 | 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | | 9F6F59234CC36DF341184FAEC934BEEE516E1E60 | 5432CFFCE5729BF1F3E6
[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails
[ https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257899#comment-13257899 ] Karl Wright edited comment on CONNECTORS-347 at 4/20/12 1:42 AM: - Just saw it again; clearly it is "real". My working hypothesis is that the counts returned by count() in the SQL statements are in fact not synchronized with the transactions, so maybe the count can "run behind". The context in which this happens is an initial crawl of six documents, with a hopcount filter. If the filter is not working the count would be 7; with a working filter, it's 6. But how can we then get 4? was (Author: kwri...@metacarta.com): Just saw it again; clearly it is "real". My working hypothesis is that the counts returned by count(x) in the SQL statements are in fact not synchronized with the transactions, so maybe the count can "run behind". The context in which this happens is an initial crawl of six documents, with a hopcount filter. If the filter is not working the count would be 7; with a working filter, it's 6. But how can we then get 4? > MySQL HopCount test sometimes fails > --- > > Key: CONNECTORS-347 > URL: https://issues.apache.org/jira/browse/CONNECTORS-347 > Project: ManifoldCF > Issue Type: Bug > Components: Framework core >Affects Versions: ManifoldCF 0.5, ManifoldCF 0.6 >Reporter: Karl Wright >Assignee: Karl Wright > Fix For: ManifoldCF 0.6 > > > The HopcountMySQLIT test sometimes fails complaining that it is expecting to > see 6 documents but only sees 4. > {code} > [junit] org.apache.manifoldcf.core.interfaces.ManifoldCFException: Wrong > number of documents processed - expected 6, saw 4 > [junit] at > org.apache.manifoldcf.filesystem_tests.HopcountMySQLIT.hopcountCheck(HopcountMySQLIT.java:152) > {code} > This does not happen on every run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails
[ https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257899#comment-13257899 ] Karl Wright edited comment on CONNECTORS-347 at 4/19/12 11:23 PM: -- Just saw it again; clearly it is "real". My working hypothesis is that the counts returned by count(x) in the SQL statements are in fact not synchronized with the transactions, so maybe the count can "run behind". The context in which this happens is an initial crawl of six documents, with a hopcount filter. If the filter is not working the count would be 7; with a working filter, it's 6. But how can we then get 4? was (Author: kwri...@metacarta.com): Just saw it again; clearly it is "real". > MySQL HopCount test sometimes fails > --- > > Key: CONNECTORS-347 > URL: https://issues.apache.org/jira/browse/CONNECTORS-347 > Project: ManifoldCF > Issue Type: Bug > Components: Framework core >Affects Versions: ManifoldCF 0.5 >Reporter: Karl Wright >Assignee: Karl Wright > Fix For: ManifoldCF next > > > The HopcountMySQLIT test sometimes fails complaining that it is expecting to > see 6 documents but only sees 4. > {code} > [junit] org.apache.manifoldcf.core.interfaces.ManifoldCFException: Wrong > number of documents processed - expected 6, saw 4 > [junit] at > org.apache.manifoldcf.filesystem_tests.HopcountMySQLIT.hopcountCheck(HopcountMySQLIT.java:152) > {code} > This does not happen on every run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira