[jira] [Commented] (ASTERIXDB-2176) Deletion doesn't work on the RTree index.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182129#comment-17182129
 ] 

ASF subversion and git services commented on ASTERIXDB-2176:


Commit ae092dd7cb3eec160950703edbf2967ba2b31e34 in asterixdb's branch 
refs/heads/master from Ian Maxon
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=ae092dd ]

[ASTERIXDB-2176][RT] Flexible python pathing

- Allow python path and args to be specified in config

Change-Id: I6f75b29be59090250b2ff9992279187babd97fff
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7423
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Michael Blow 


> Deletion doesn't work on the RTree index.
> -
>
> Key: ASTERIXDB-2176
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2176
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Chen Luo
>Priority: Major
>
> This is a simplified version of "upsert/primary-secondary-tree" AQL test case.
> spatialData.json file
> { "id": 12, "point": point("6.0,3.0") }
> moreSpatialData.json file
> {"id": 12, "point": point("4.1,7.0")}
> DDL: 
> {code}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type MyRecord as closed {
>   id: int64,
>   point: point
> }
> create dataset UpsertTo(MyRecord)
>  primary key id;
> create dataset UpsertFrom(MyRecord)
>  primary key id;
> create index rtree_index_point on UpsertTo(point) type rtree;
> {code}
> DML
> {code}
> load dataset UpsertTo
> using localfs
> (("path"="asterix_nc1://data/spatial/spatialData.json"),("format"="adm"));
> load dataset UpsertFrom
> using localfs
> (("path"="asterix_nc1://data/spatial/moreSpatialData.json"),("format"="adm"));
> upsert into dataset UpsertTo(
> for $x in dataset UpsertFrom
> return $x
> );
> for $o in dataset('UpsertTo')
> where spatial-intersect($o.point, 
> create-polygon([4.0,1.0,4.0,4.0,12.0,4.0,12.0,1.0]))
> order by $o.id
> return $o;
> {code}
> This DML returns the new record correctly. But, the issue is that the indexed 
> value in RTree has not been updated.
> When searching the rtree_index_point index, the searcher sees the previous 
> value - point("6.0,3.0"), not the new value - point("4.1,7.0"). So, this 
> record will be fetched from the primary index. However, the primary index 
> search returns the updated value and the select() verifies the value. It 
> returns true by coincidence because the new value satisfies the 
> spatial-intersect() condition.
> The secondary index search should see the updated value, not the previous 
> value. And this is an issue for the index-only plan case since it only uses 
> the value from a secondary index.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ASTERIXDB-2176) Deletion doesn't work on the RTree index.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165832#comment-17165832
 ] 

ASF subversion and git services commented on ASTERIXDB-2176:


Commit 7b959961116754fb80b754e63dba5d20ef8797ae in asterixdb's branch 
refs/heads/master from Ian Maxon
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=7b95996 ]

[ASTERIXDB-2176][RT] Improved Python IPC

- use msgpack serialization over tcp loopback
- convert directly from ADM binary format to msg

Change-Id: I5cbbc367944b489aee651ea050e74990dcf65521
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6883
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Dmitry Lychagin 


> Deletion doesn't work on the RTree index.
> -
>
> Key: ASTERIXDB-2176
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2176
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Chen Luo
>Priority: Major
>
> This is a simplified version of "upsert/primary-secondary-tree" AQL test case.
> spatialData.json file
> { "id": 12, "point": point("6.0,3.0") }
> moreSpatialData.json file
> {"id": 12, "point": point("4.1,7.0")}
> DDL: 
> {code}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type MyRecord as closed {
>   id: int64,
>   point: point
> }
> create dataset UpsertTo(MyRecord)
>  primary key id;
> create dataset UpsertFrom(MyRecord)
>  primary key id;
> create index rtree_index_point on UpsertTo(point) type rtree;
> {code}
> DML
> {code}
> load dataset UpsertTo
> using localfs
> (("path"="asterix_nc1://data/spatial/spatialData.json"),("format"="adm"));
> load dataset UpsertFrom
> using localfs
> (("path"="asterix_nc1://data/spatial/moreSpatialData.json"),("format"="adm"));
> upsert into dataset UpsertTo(
> for $x in dataset UpsertFrom
> return $x
> );
> for $o in dataset('UpsertTo')
> where spatial-intersect($o.point, 
> create-polygon([4.0,1.0,4.0,4.0,12.0,4.0,12.0,1.0]))
> order by $o.id
> return $o;
> {code}
> This DML returns the new record correctly. But, the issue is that the indexed 
> value in RTree has not been updated.
> When searching the rtree_index_point index, the searcher sees the previous 
> value - point("6.0,3.0"), not the new value - point("4.1,7.0"). So, this 
> record will be fetched from the primary index. However, the primary index 
> search returns the updated value and the select() verifies the value. It 
> returns true by coincidence because the new value satisfies the 
> spatial-intersect() condition.
> The secondary index search should see the updated value, not the previous 
> value. And this is an issue for the index-only plan case since it only uses 
> the value from a secondary index.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ASTERIXDB-2176) Deletion doesn't work on the RTree index.

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099380#comment-17099380
 ] 

ASF subversion and git services commented on ASTERIXDB-2176:


Commit a28ddb731dfc242b0391a38564213fd4701a2955 in asterixdb's branch 
refs/heads/master from Ian Maxon
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=a28ddb7 ]

[ASTERIXDB-2176] Python UDFs

- user model changes: yes
- storage format changes: no
- interface changes: yes

Details:
- Allow UDFs to be shiv-packaged python modules with their
  dependencies
- Use pyro for python RPC
- Maven build for pyro package into server
- Remove JObject spatial types
- Simpler conversion from java primitive/standard types and
  collections
  to JObject equivalents

Change-Id: Ibea23a2e9308132f343d80eff04ede9a235aa021
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5526
Contrib: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Michael Blow 


> Deletion doesn't work on the RTree index.
> -
>
> Key: ASTERIXDB-2176
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2176
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Chen Luo
>Priority: Major
>
> This is a simplified version of "upsert/primary-secondary-tree" AQL test case.
> spatialData.json file
> { "id": 12, "point": point("6.0,3.0") }
> moreSpatialData.json file
> {"id": 12, "point": point("4.1,7.0")}
> DDL: 
> {code}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type MyRecord as closed {
>   id: int64,
>   point: point
> }
> create dataset UpsertTo(MyRecord)
>  primary key id;
> create dataset UpsertFrom(MyRecord)
>  primary key id;
> create index rtree_index_point on UpsertTo(point) type rtree;
> {code}
> DML
> {code}
> load dataset UpsertTo
> using localfs
> (("path"="asterix_nc1://data/spatial/spatialData.json"),("format"="adm"));
> load dataset UpsertFrom
> using localfs
> (("path"="asterix_nc1://data/spatial/moreSpatialData.json"),("format"="adm"));
> upsert into dataset UpsertTo(
> for $x in dataset UpsertFrom
> return $x
> );
> for $o in dataset('UpsertTo')
> where spatial-intersect($o.point, 
> create-polygon([4.0,1.0,4.0,4.0,12.0,4.0,12.0,1.0]))
> order by $o.id
> return $o;
> {code}
> This DML returns the new record correctly. But, the issue is that the indexed 
> value in RTree has not been updated.
> When searching the rtree_index_point index, the searcher sees the previous 
> value - point("6.0,3.0"), not the new value - point("4.1,7.0"). So, this 
> record will be fetched from the primary index. However, the primary index 
> search returns the updated value and the select() verifies the value. It 
> returns true by coincidence because the new value satisfies the 
> spatial-intersect() condition.
> The secondary index search should see the updated value, not the previous 
> value. And this is an issue for the index-only plan case since it only uses 
> the value from a secondary index.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ASTERIXDB-2176) Deletion doesn't work on the RTree index.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273559#comment-16273559
 ] 

ASF subversion and git services commented on ASTERIXDB-2176:


Commit 50de6669f24d376398d6977a363ac4223897c579 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=50de666 ]

[ASTERIXDB-2176] Fix deletion on LSMRTree

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- The deletion on LSMRTree is not working on master, because initially
we insert anti-matter tuples into the memory BTree, and during flush
these tuples are copied to the RTree again. However, we forgot to
set these tuples from the in-memory BTree as anti-matter tuples. This
patch fixes this.
- Also modifies the test case to cover this case.

Change-Id: I3d9417e56f06044f585e019089004efd2b2af3b7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2188
Sonar-Qube: Jenkins 
Reviewed-by: Taewoo Kim 
Tested-by: Jenkins 
Integration-Tests: Jenkins 


> Deletion doesn't work on the RTree index.
> -
>
> Key: ASTERIXDB-2176
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2176
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Chen Luo
>
> This is a simplified version of "upsert/primary-secondary-tree" AQL test case.
> spatialData.json file
> { "id": 12, "point": point("6.0,3.0") }
> moreSpatialData.json file
> {"id": 12, "point": point("4.1,7.0")}
> DDL: 
> {code}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type MyRecord as closed {
>   id: int64,
>   point: point
> }
> create dataset UpsertTo(MyRecord)
>  primary key id;
> create dataset UpsertFrom(MyRecord)
>  primary key id;
> create index rtree_index_point on UpsertTo(point) type rtree;
> {code}
> DML
> {code}
> load dataset UpsertTo
> using localfs
> (("path"="asterix_nc1://data/spatial/spatialData.json"),("format"="adm"));
> load dataset UpsertFrom
> using localfs
> (("path"="asterix_nc1://data/spatial/moreSpatialData.json"),("format"="adm"));
> upsert into dataset UpsertTo(
> for $x in dataset UpsertFrom
> return $x
> );
> for $o in dataset('UpsertTo')
> where spatial-intersect($o.point, 
> create-polygon([4.0,1.0,4.0,4.0,12.0,4.0,12.0,1.0]))
> order by $o.id
> return $o;
> {code}
> This DML returns the new record correctly. But, the issue is that the indexed 
> value in RTree has not been updated.
> When searching the rtree_index_point index, the searcher sees the previous 
> value - point("6.0,3.0"), not the new value - point("4.1,7.0"). So, this 
> record will be fetched from the primary index. However, the primary index 
> search returns the updated value and the select() verifies the value. It 
> returns true by coincidence because the new value satisfies the 
> spatial-intersect() condition.
> The secondary index search should see the updated value, not the previous 
> value. And this is an issue for the index-only plan case since it only uses 
> the value from a secondary index.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2176) Deletion doesn't work on the RTree index.

[Taewoo Kim (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265520#comment-16265520
 ] 

Taewoo Kim commented on ASTERIXDB-2176:
---

Adding one more point:

A sqlpp execution test case: "scan-delete-rtree-secondary-index" conducts the 
following:

(1) creates a dataset 
(2) loads 21 records. (id = 1 ~ 21)
(3) creates a RTree index
(4) deletes 10 records (id > 10) 
(5) query

On the master branch, an RTree search reveals id: 12 and 20. These records are 
filtered by the primary-index search since it cannot find the record 12 and 20. 
So it's not seen. Overall, I think deletion on RTree doesn't work now. And it 
is hidden because of the primary-index search.

> Deletion doesn't work on the RTree index.
> -
>
> Key: ASTERIXDB-2176
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2176
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>
> This is a simplified version of "upsert/primary-secondary-tree" AQL test case.
> spatialData.json file
> { "id": 12, "point": point("6.0,3.0") }
> moreSpatialData.json file
> {"id": 12, "point": point("4.1,7.0")}
> DDL: 
> {code}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type MyRecord as closed {
>   id: int64,
>   point: point
> }
> create dataset UpsertTo(MyRecord)
>  primary key id;
> create dataset UpsertFrom(MyRecord)
>  primary key id;
> create index rtree_index_point on UpsertTo(point) type rtree;
> {code}
> DML
> {code}
> load dataset UpsertTo
> using localfs
> (("path"="asterix_nc1://data/spatial/spatialData.json"),("format"="adm"));
> load dataset UpsertFrom
> using localfs
> (("path"="asterix_nc1://data/spatial/moreSpatialData.json"),("format"="adm"));
> upsert into dataset UpsertTo(
> for $x in dataset UpsertFrom
> return $x
> );
> for $o in dataset('UpsertTo')
> where spatial-intersect($o.point, 
> create-polygon([4.0,1.0,4.0,4.0,12.0,4.0,12.0,1.0]))
> order by $o.id
> return $o;
> {code}
> This DML returns the new record correctly. But, the issue is that the indexed 
> value in RTree has not been updated.
> When searching the rtree_index_point index, the searcher sees the previous 
> value - point("6.0,3.0"), not the new value - point("4.1,7.0"). So, this 
> record will be fetched from the primary index. However, the primary index 
> search returns the updated value and the select() verifies the value. It 
> returns true by coincidence because the new value satisfies the 
> spatial-intersect() condition.
> The secondary index search should see the updated value, not the previous 
> value. And this is an issue for the index-only plan case since it only uses 
> the value from a secondary index.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)