[jira] [Commented] (ASTERIXDB-1451) Upsert: Open Index test fails with duplicate rows in result when VBC page size is reduced

2016-09-12 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485309#comment-15485309
 ] 

ASF subversion and git services commented on ASTERIXDB-1451:


Commit 534d589297cc4fadbd61b4de70af5c406299814c in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=534d589 ]

ASTERIXDB-1451: Remove Record Casting for insert/delete/upsert

This change includes the following:
- Introduce cast function in case of delete operation
  after the primary index to ensure types are passed
  correctly to enforced indexes.
- Introduce cast function in case of upsert operation
  before old secondary keys extraction to ensure types
  are passed correctly to enforced indexes.
- Replace all record casts with open field casts for
  insert/delete/upsert operations.
- Sonar-Qube fixes.

Change-Id: I6a80105798ea1c86a6a0eb69a79b9573b54931b7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1146
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Upsert: Open Index test fails with duplicate rows in result when VBC page 
> size is reduced
> -
>
> Key: ASTERIXDB-1451
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1451
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Michael Blow
>Assignee: Murtadha Hubail
>
> To repro:
> - configure storage.memorycomponent.pagesize to 8k, and increase 
> storage.memorycomponent.numpages to 24
> - run asterix-app runtime tests
> - observe failure in open-index test with duplicated rows as shown below.
> $ diff -du 
> src/test/resources/runtimets/results/upsert/open-index/open-index.1.adm 
> rttest/results/upsert/open-index.adm
> --- src/test/resources/runtimets/results/upsert/open-index/open-index.1.adm   
> 2016-04-27 20:40:58.0 -0700
> +++ rttest/results/upsert/open-index.adm  2016-05-16 19:08:57.0 
> -0700
> @@ -1099,11 +1099,15 @@
>  { "o_orderkey": 5927, "o_custkey": 116 }
>  { "o_orderkey": 5952, "o_custkey": 148 }
>  { "o_orderkey": 5955, "o_custkey": 94 }
> +{ "o_orderkey": 5955, "o_custkey": 94 }
> +{ "o_orderkey": 5957, "o_custkey": 89 }
>  { "o_orderkey": 5957, "o_custkey": 89 }
>  { "o_orderkey": 5958, "o_custkey": 115 }
>  { "o_orderkey": 5984, "o_custkey": 70 }
>  { "o_orderkey": 5985, "o_custkey": 143 }
>  { "o_orderkey": 5986, "o_custkey": 115 }
> +{ "o_orderkey": 5986, "o_custkey": 115 }
> +{ "o_orderkey": 5987, "o_custkey": 64 }
>  { "o_orderkey": 5987, "o_custkey": 64 }
>  { "o_orderkey": 10986, "o_custkey": 115 }
>  { "o_orderkey": 10987, "o_custkey": 64 }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1451) Upsert: Open Index test fails with duplicate rows in result when VBC page size is reduced

2016-08-12 Thread Murtadha Hubail (JIRA) 

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419091#comment-15419091
 ] 

Murtadha Hubail commented on ASTERIXDB-1451:




I have been investigating this issue for some time. It is not related to the 
VBC page size nor upsert. When any record, with an enforced index of a type 
other than INT64, is written to a disk component, a delete operation with an 
antimatter tuple for that same record will not hide the record written in the 
disk component of the enforced index. Therefore, duplicate records will be 
returned in any query using the enforced index.

I haven't found the root cause yet, but I believe it is related to the variable 
type propagation during query compilation. I will discuss it with [~buyingyi] 
and [~alamoudi].

The issue can be easily reproduced by the following:

{code}
drop dataverse test if exists;
create dataverse test;
use dataverse test;

create type OrderOpenType as open {
  o_orderkey: int64
}

create dataset OrdersOpen(OrderOpenType)
primary key o_orderkey;

insert into dataset OrdersOpen (
  {"o_orderkey": 1,
  "o_custkey": 1,}
)

create index idx_Orders_Custkey on OrdersOpen(o_custkey:int32) enforced;

for $o in dataset('OrdersOpen')
where
 $o.o_custkey >=-1
return {
  "o_orderkey": $o.o_orderkey
}
{code}

> Upsert: Open Index test fails with duplicate rows in result when VBC page 
> size is reduced
> -
>
> Key: ASTERIXDB-1451
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1451
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Michael Blow
>Assignee: Murtadha Hubail
>
> To repro:
> - configure storage.memorycomponent.pagesize to 8k, and increase 
> storage.memorycomponent.numpages to 24
> - run asterix-app runtime tests
> - observe failure in open-index test with duplicated rows as shown below.
> $ diff -du 
> src/test/resources/runtimets/results/upsert/open-index/open-index.1.adm 
> rttest/results/upsert/open-index.adm
> --- src/test/resources/runtimets/results/upsert/open-index/open-index.1.adm   
> 2016-04-27 20:40:58.0 -0700
> +++ rttest/results/upsert/open-index.adm  2016-05-16 19:08:57.0 
> -0700
> @@ -1099,11 +1099,15 @@
>  { "o_orderkey": 5927, "o_custkey": 116 }
>  { "o_orderkey": 5952, "o_custkey": 148 }
>  { "o_orderkey": 5955, "o_custkey": 94 }
> +{ "o_orderkey": 5955, "o_custkey": 94 }
> +{ "o_orderkey": 5957, "o_custkey": 89 }
>  { "o_orderkey": 5957, "o_custkey": 89 }
>  { "o_orderkey": 5958, "o_custkey": 115 }
>  { "o_orderkey": 5984, "o_custkey": 70 }
>  { "o_orderkey": 5985, "o_custkey": 143 }
>  { "o_orderkey": 5986, "o_custkey": 115 }
> +{ "o_orderkey": 5986, "o_custkey": 115 }
> +{ "o_orderkey": 5987, "o_custkey": 64 }
>  { "o_orderkey": 5987, "o_custkey": 64 }
>  { "o_orderkey": 10986, "o_custkey": 115 }
>  { "o_orderkey": 10987, "o_custkey": 64 }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)