[
https://issues.apache.org/jira/browse/ASTERIXDB-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475341#comment-15475341
]
Taewoo Kim edited comment on ASTERIXDB-1487 at 9/8/16 11:26 PM:
----------------------------------------------------------------
What's the main reason here?
was (Author: wangsaeu):
What's the main issue here?
> Fuzzy select-join on inverted index poses inconsistent results.
> ---------------------------------------------------------------
>
> Key: ASTERIXDB-1487
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1487
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: AsterixDB
> Environment: MAC 4 cores, 8GB memory. The current master till
> 3/17/2016.
> Reporter: Wenhai
> Assignee: Wenhai
> Priority: Critical
> Attachments: csx-small-multi-id.txt, dblp-small-multi-id.txt
>
>
> As shown in below. After we switching the two "for" branches of the fuzzy
> join over a select, the results are consistent.
> Schema
> {noformat}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type DBLPNestedType as closed {
> id: int64,
> dblpid: string,
> title: string,
> authors: string,
> misc: string
> }
> create type DBLPType as closed {
> nested: DBLPNestedType
> }
> create type CSXNestedType as closed {
> id: int64,
> csxid: string,
> title: string,
> authors: string,
> misc: string
> }
> create type CSXType as closed {
> nested: CSXNestedType
> }
> create dataset DBLPtmp(DBLPNestedType) primary key id;
> create dataset CSXtmp(CSXNestedType) primary key id;
> create dataset DBLP(DBLPType) primary key nested.id;
> create dataset CSX(CSXType) primary key nested.id;
> use dataverse test;
> load dataset DBLPtmp
> using localfs
> (("path"="asterix_nc1://data/dblp-small/dblp-small-multi-id.txt"),("format"="delimited-text"),("delimiter"=":"),("quote"="\u0000"))
> pre-sorted;
> load dataset CSXtmp
> using localfs
> (("path"="asterix_nc1://data/pub-small/csx-small-multi-id.txt"),("format"="delimited-text"),("delimiter"=":"),("quote"="\u0000"));
> insert into dataset DBLP(
> for $x in dataset DBLPtmp
> return {
> "nested": $x
> }
> );
> insert into dataset CSX(
> for $x in dataset CSXtmp
> return {
> "nested": $x
> }
> );
> {noformat}
> Indexes
> {noformat}
> create index keyword_index on DBLP(nested.title) type keyword;
> create index keyword_indexdbauhors on DBLP(nested.authors) type keyword;
> create index keyword_indexcsxauthors on CSX(nested.authors) type keyword;
> {noformat}
> The following query
> {noformat}
> use dataverse test;
> set simthresholds '.1'
> let $s := count(
> for $o in dataset DBLP
> for $t in dataset CSX
> where contains($o.nested.title, "System") and word-tokens($o.nested.authors)
> ~= word-tokens($t.nested.authors)
> return $o
> )
> return $s
> {noformat}
> will return 28, while the query
> {noformat}
> use dataverse test;
> set simthresholds '.1'
> let $s := count(
> for $t in dataset CSX
> for $o in dataset DBLP
> where contains($o.nested.title, "System") and word-tokens($o.nested.authors)
> ~= word-tokens($t.nested.authors)
> return $o
> )
> return $s
> {noformat}
> will return 3 or pose a error in a big dataset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
