[jira] [Commented] (ASTERIXDB-2018) Http requests hang forever if client closed connection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115050#comment-16115050
 ] 

ASF subversion and git services commented on ASTERIXDB-2018:


Commit f94f63d3f8e52cdb099cce365b9fb053050969cf in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=f94f63d ]

[ASTERIXDB-2018][NET] Add HTTP close channel listener

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
Add a listener on HTTP channel to notify waiters when the
channel is closed.

Change-Id: I08e4890880a70d42e3e181366a5b0252102d0ea2
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1917
Reviewed-by: abdullah alamoudi 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 


> Http requests hang forever if client closed connection
> --
>
> Key: ASTERIXDB-2018
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2018
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: NET - Network
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> If a client closes the HTTP request before the response is sent, the request 
> thread hangs forever in ChunkedNettyOutputStream.ensureWritable



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2019) ArithmeticException encountered when the cluster doesn't have any partitions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115051#comment-16115051
 ] 

ASF subversion and git services commented on ASTERIXDB-2019:


Commit e0d8e5078f90823e8dd51052317a7da1c08cc9f9 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=e0d8e50 ]

[ASTERIXDB-2019][CLUS] Update cluster state on partitions changes

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Set the cluster to UNUSABLE when no partitions are registered
- Update cluster state after partitions register/de-register
- Reject unregistered nodes queries on CC
- Avoid NPE when trying to send to a node that was de-registered

Change-Id: I7d11733a1dcd86136e157d80517bff4abcfc776b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1918
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> ArithmeticException encountered when the cluster doesn't have any partitions
> 
>
> Key: ASTERIXDB-2019
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2019
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> When the cluster doesn't have any partitions it is still in the ACTIVE 
> state!! and when it receives a request, the following exception is 
> encountered:
> {code:java}
> 2017-07-31T02:49:28.801-07:00 INFO CBAS.translator.QueryTranslator 
> [QueryTranslator] / by zero
> java.lang.ArithmeticException: / by zero
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.JobBuilder.(JobBuilder.java:87)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:58)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)
>  ~[algebricks-compiler-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:333)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.rewriteCompileQuery(QueryTranslator.java:1834)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.lambda$handleQuery$1(QueryTranslator.java:2307)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.createAndRunJob(QueryTranslator.java:2407)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.deliverResult(QueryTranslator.java:2347)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.handleQuery(QueryTranslator.java:2319)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:370)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.message.ExecuteStatementRequestMessage.lambda$handle$0(ExecuteStatementRequestMessage.java:127)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_121]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [?:1.8.0_121]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [?:1.8.0_121]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [?:1.8.0_121]
>   at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> 2017-07-31T02:49:28.802-07:00 ERRO CBAS.apache.asterix [QueryTranslator] 
> Unexpected exception
> java.lang.ArithmeticException: / by zero
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.JobBuilder.(JobBuilder.java:87)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:58)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)
>  ~[algebricks-compiler-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> 

[jira] [Commented] (ASTERIXDB-2006) Exclusive Modify Lock doesn't cover Modify lock

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105748#comment-16105748
 ] 

ASF subversion and git services commented on ASTERIXDB-2006:


Commit b5d4f56ae427351a282af4c8d740d8b91d21f80f in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=b5d4f56 ]

[ASTERIXDB-2006][TX] Fix metadata lock containment rules

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Locks with mode exclusive modify cover all modify modes

Change-Id: Ib0ecbaed86370707d560b7d0c3e6933c198aab41
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1907
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Reviewed-by: Till Westmann 
Integration-Tests: Jenkins 


> Exclusive Modify Lock doesn't cover Modify lock
> ---
>
> Key: ASTERIXDB-2006
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2006
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: TX - Transactions
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> When acquiring a metadata lock, the caller passes the requested mode. For 
> example READ and WRITE.
> When the lock has already been acquired, the lock manager checks if the 
> acquired mode covers the requested mode. For example, WRITE covers READ but 
> not the other way around.
> EXCLUSIVE_MODIFY_LOCK covers MODIFY lock as well but that was not implemented 
> correctly in the metadata lock manager.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2004) TestExecutor sometimes write test results outside target

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105460#comment-16105460
 ] 

ASF subversion and git services commented on ASTERIXDB-2004:


Commit 58be3a8e40caf430982e26a58284f9c6df41836e in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=58be3a8 ]

[ASTERIXDB-2004][TEST] Prevent Tests from writing outside target

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Some tests access data and queries outside the module. When that
  happens, the base path can contain ../ which can lead to results
  being written outside target. This change removes all ../ from
  the path of the actual results.

Change-Id: If100c33780fa436ddb2a8e64f3901251156f5524
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1905
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> TestExecutor sometimes write test results outside target
> 
>
> Key: ASTERIXDB-2004
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2004
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: TEST - Test Framework
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2007) CCApplication.stop() throws NullPointerException when start() fails before appCtx is initialized

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106594#comment-16106594
 ] 

ASF subversion and git services commented on ASTERIXDB-2007:


Commit 18b8323eef11544719af8119677f64fc0c234935 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=18b8323 ]

[ASTERIXDB-2007][CLUS] Ensure AppCtx created before stopping Active Handler

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- if application.start() fails before creation of the application
  context, then application.stop() will throw a null pointer
  exception. This change checks that appCtx is not null before
  dispatching calls on the object.

Change-Id: I898b6192fd91054129ed11d20c7ffe21116408c2
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1908
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
BAD: Jenkins 
Reviewed-by: Murtadha Hubail 


> CCApplication.stop() throws NullPointerException when start() fails before 
> appCtx is initialized
> 
>
> Key: ASTERIXDB-2007
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2007
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1897) NPE in complex group-by query

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119306#comment-16119306
 ] 

ASF subversion and git services commented on ASTERIXDB-1897:


Commit bf1aea2acdc094aa34626e15679eaacf7b5a28de in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=bf1aea2 ]

[ASTERIXDB-1897][COMP][RT] Fix complex group-by.

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Fix type computer for numeric aggregations;
- Fix error reporting for SubplanRuntimeFactory;
- Add a negative test query.

Change-Id: Iebb393820a8edd0c54d80248b2a33c77d4f6fd7b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1923
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Till Westmann 


> NPE in complex group-by query
> -
>
> Key: ASTERIXDB-1897
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1897
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Yingyi Bu
>Assignee: Yingyi Bu
>
> The DDLs and query are as follows:
> {noformat}
> DROP DATAVERSE gby IF EXISTS;
> CREATE DATAVERSE gby;
> USE gby;
> CREATE TYPE PolicyType AS {
> id: UUID
> }
> CREATE DATASET policies(PolicyType) PRIMARY KEY id AUTOGENERATED;
> INSERT INTO policies
> (
>   [ {
> "policyno": "C123",
> "state": "CA",
> "zipcode": "96008",
> "make": "Honda",
> "accidents": [
>   {
> "year": "2015",
> "cost": 5000
>   },
>   {
> "year": "2016",
> "cost": 8000
>   },
>   {
> "year": "2016",
> "cost": 6000
>   }
> ]
>   },
>   {
> "policyno": "C124",
> "state": "CA",
> "zipcode": "96853",
> "make": "Ford",
> "accidents": [
>   {
> "year": "2015",
> "cost": 5000
>   },
>   {
> "year": "2015",
> "cost": 8000
>   },
>   {
> "year": "2016",
> "cost": 6000
>   }
> ]
>   },
>   {
> "policyno": "A123",
> "state": "AZ",
> "zipcode": "86008",
> "make": "Honda",
> "accidents": [
> {
>   "year": "2015",
>   "cost": 5000
> },
> {
>   "year": "2016",
>   "cost": 8000
> },
> {
>   "year": "2016",
>   "cost": 6000
> }
> ]
>   },
>   {
> "policyno": "A124",
> "state": "AZ",
> "zipcode": "86853",
> "make": "Ford",
> "accidents": [
> {
>   "year": "2015",
>   "cost": 5000
> },
> {
>   "year": "2016",
>   "cost": 8000
> },
> {
>   "year": "2016",
>   "cost": 6000
> }
> ]
>   },
>   {
> "policyno": "U123",
> "state": "UT",
> "zipcode": "66008",
> "make": "Honda",
> "accidents": [
> {
>   "year": "2015",
>   "cost": 5000
> },
> {
>   "year": "2016",
>   "cost": 8000
> },
> {
>   "year": "2016",
>   "cost": 6000
> }
> ]
>   },
>   {
> "policyno": "U124",
> "state": "UT",
> "zipcode": "66853",
> "make": "Ford",
> "accidents": [ ]
>   },
>   {
> "policyno": "U125",
> "state": "UT",
> "zipcode": "66853",
> "make": "Ford"
>   } ]
> );
> FROM policies p
> GROUP BY state GROUP AS g
> SELECT state,
>(
>  FROM g
>  SELECT VALUE SUM(
>   (
> FROM g.p.accidents a
> WHERE a.year = "2016"
> SELECT VALUE COUNT(*)
>)[0]
> )
>)[0]  / (COUNT(*) * 1.0 ) AS risk
> ORDER BY risk DESC
> LIMIT 5;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2029) Remove this from dataset not found errors

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119458#comment-16119458
 ] 

ASF subversion and git services commented on ASTERIXDB-2029:


Commit 6dde58257cdd56de5e85010636bf1493e8fceb81 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=6dde582 ]

[ASTERIXDB-2029][COMP]Remove "this" from dataset errors

- user model changes: improves error message
- storage format changes: no
- interface changes: no

details:
- Error message for drop dataset operation had the word "this"
  as part of the error message. It was removed

Change-Id: Ifc74edb44aef8bfac3cb73fb8192bf6c353a66de
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1928
Reviewed-by: abdullah alamoudi 
Integration-Tests: abdullah alamoudi 
Tested-by: abdullah alamoudi 


> Remove this from dataset not found errors
> -
>
> Key: ASTERIXDB-2029
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2029
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: COMP - Compiler
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2008) ClusterStateManager.removePending adds nodes to pending removal even if they are not part of the cluster

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118223#comment-16118223
 ] 

ASF subversion and git services commented on ASTERIXDB-2008:


Commit 07075667c39d1b44939dfb83389de2626b21d8b7 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=0707566 ]

[ASTERIXDB-2008][CLUS] Only add pending removal if node known

[ASTERIXDB-2023][ING] Introduce Enums instead of using bytes

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Only nodes which are known to cluster manager are added
to the list of nodes pending removal. Other nodes are ignored
- Enums introduced:
  - ActiveEvent.Kind
  - ActivePartitionMessage.Event
- Remove AdapterRuntimeManager
- Remove AdapterExecutor

Change-Id: I7044896559798426c04a3f46861bc5335b25d140
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1921
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> ClusterStateManager.removePending adds nodes to pending removal even if they 
> are not part of the cluster
> 
>
> Key: ASTERIXDB-2008
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2008
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2023) Introduce Enums instead of using bytes for ActiveEvents and Messages

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118224#comment-16118224
 ] 

ASF subversion and git services commented on ASTERIXDB-2023:


Commit 07075667c39d1b44939dfb83389de2626b21d8b7 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=0707566 ]

[ASTERIXDB-2008][CLUS] Only add pending removal if node known

[ASTERIXDB-2023][ING] Introduce Enums instead of using bytes

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Only nodes which are known to cluster manager are added
to the list of nodes pending removal. Other nodes are ignored
- Enums introduced:
  - ActiveEvent.Kind
  - ActivePartitionMessage.Event
- Remove AdapterRuntimeManager
- Remove AdapterExecutor

Change-Id: I7044896559798426c04a3f46861bc5335b25d140
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1921
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> Introduce Enums instead of using bytes for ActiveEvents and Messages
> 
>
> Key: ASTERIXDB-2023
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2023
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: ING - Ingestion
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Some classes such as ActiveEvent uses a byte member to indicates type. While 
> this is not incorrect, it introduces room for mistakes. In addition, logging 
> becomes harder to follow if the byte value is logged.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1706) Sporadic integration test failure

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120608#comment-16120608
 ] 

ASF subversion and git services commented on ASTERIXDB-1706:


Commit a6d9b093a3aa0344a18897691e74260befdacf8f in asterixdb's branch 
refs/heads/master from [~imaxon]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=a6d9b09 ]

[ASTERIXDB-1706][API] Additional logging for debug

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
This is just another log, similar to the response size one
already merged, to try to nail down the true source of this
issue. The previous logs point to the issue being somewhere
besides result distribution.

Change-Id: Ie377e531688ecf715ef822ed0fb027ee124fc976
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1929
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Murtadha Hubail 


> Sporadic integration test failure
> -
>
> Key: ASTERIXDB-1706
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1706
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: *DB - AsterixDB
>Reporter: Till
>Assignee: Ian Maxon
>
> We do have some sporadic wrong results in the integration tests, e.g. 
> https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/org.apache.asterix$asterix-installer/1005/testReport/junit/org.apache.asterix.installer.test/ManagixExecutionIT/test_1223_/
> {noformat}
> Oct 23, 2016 5:07:42 PM org.apache.asterix.test.aql.TestExecutor executeTest
> INFO: Starting [TEST]: temporal/overlap_bins ... 
> Expected results file: 
> ../asterix-app/src/test/resources/runtimets/results/temporal/overlap_bins/overlap_bins.1.adm
> Actual results file: 
> target/ittest/results/temporal/overlap_bins/overlap_bins.1.adm
> testFile 
> ../asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.3.query.aql
>  raised an exception: org.apache.asterix.test.base.ComparisonException: 
> Result for 
> ../asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.3.query.aql
>  changed at line 1:
> < { "timebins": [ interval(time("17:00:00.000Z"), time("17:30:00.000Z")), 
> interval(time("17:30:00.000Z"), time("18:00:00.000Z")), 
> interval(time("18:00:00.000Z"), time("18:30:00.000Z")), 
> interval(time("18:30:00.000Z"), time("19:00:00.000Z")) ], "datebins": [ 
> interval(date("1980-01-01"), date("1990-01-01")), 
> interval(date("1990-01-01"), date("2000-01-01")), 
> interval(date("2000-01-01"), date("2010-01-01")), 
> interval(date("2010-01-01"), date("2020-01-01")) ], "datetimebins": [ 
> interval(datetime("1800-01-01T00:00:00.000Z"), 
> datetime("1900-01-01T00:00:00.000Z")), 
> interval(datetime("1900-01-01T00:00:00.000Z"), 
> datetime("2000-01-01T00:00:00.000Z")), 
> interval(datetime("2000-01-01T00:00:00.000Z"), 
> datetime("2100-01-01T00:00:00.000Z")) ] }
> > 
> org.apache.asterix.test.base.ComparisonException: Result for 
> ../asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.3.query.aql
>  changed at line 1:
> < { "timebins": [ interval(time("17:00:00.000Z"), time("17:30:00.000Z")), 
> interval(time("17:30:00.000Z"), time("18:00:00.000Z")), 
> interval(time("18:00:00.000Z"), time("18:30:00.000Z")), 
> interval(time("18:30:00.000Z"), time("19:00:00.000Z")) ], "datebins": [ 
> interval(date("1980-01-01"), date("1990-01-01")), 
> interval(date("1990-01-01"), date("2000-01-01")), 
> interval(date("2000-01-01"), date("2010-01-01")), 
> interval(date("2010-01-01"), date("2020-01-01")) ], "datetimebins": [ 
> interval(datetime("1800-01-01T00:00:00.000Z"), 
> datetime("1900-01-01T00:00:00.000Z")), 
> interval(datetime("1900-01-01T00:00:00.000Z"), 
> datetime("2000-01-01T00:00:00.000Z")), 
> interval(datetime("2000-01-01T00:00:00.000Z"), 
> datetime("2100-01-01T00:00:00.000Z")) ] }
> > 
>   at 
> org.apache.asterix.test.aql.TestExecutor.throwLineChanged(TestExecutor.java:203)
>   at 
> org.apache.asterix.test.aql.TestExecutor.runScriptAndCompareWithResult(TestExecutor.java:152)
>   at 
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:775)
>   at 
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:1003)
>   at 
> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:666)
>   at 
> org.apache.asterix.installer.test.AbstractExecutionIT.test(AbstractExecutionIT.java:161)
>   at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at 

[jira] [Commented] (ASTERIXDB-2019) ArithmeticException encountered when the cluster doesn't have any partitions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115923#comment-16115923
 ] 

ASF subversion and git services commented on ASTERIXDB-2019:


Commit 06478601cb1cb34363a38f630a0a00e98b782c25 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=0647860 ]

[ASTERIXDB-2019][CLUS] Sync getting cluster state

user model changes: no
storage format changes: no
interface changes: no

Details:
- Leave it to the caller when to refresh the cluster state
  after register/deregister of cluster partitions.
- Synchronize cluster state to avoid getting invalid state
  during partitions reg/dereg

Change-Id: I2bc5f86cedeb4728ccbb9811a36a4655a7786246
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1920
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> ArithmeticException encountered when the cluster doesn't have any partitions
> 
>
> Key: ASTERIXDB-2019
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2019
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> When the cluster doesn't have any partitions it is still in the ACTIVE 
> state!! and when it receives a request, the following exception is 
> encountered:
> {code:java}
> 2017-07-31T02:49:28.801-07:00 INFO CBAS.translator.QueryTranslator 
> [QueryTranslator] / by zero
> java.lang.ArithmeticException: / by zero
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.JobBuilder.(JobBuilder.java:87)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:58)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)
>  ~[algebricks-compiler-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:333)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.rewriteCompileQuery(QueryTranslator.java:1834)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.lambda$handleQuery$1(QueryTranslator.java:2307)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.createAndRunJob(QueryTranslator.java:2407)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.deliverResult(QueryTranslator.java:2347)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.handleQuery(QueryTranslator.java:2319)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:370)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> org.apache.asterix.app.message.ExecuteStatementRequestMessage.lambda$handle$0(ExecuteStatementRequestMessage.java:127)
>  ~[asterix-app-0.9.2-SNAPSHOT.jar:0.9.2-SNAPSHOT]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_121]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [?:1.8.0_121]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [?:1.8.0_121]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [?:1.8.0_121]
>   at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> 2017-07-31T02:49:28.802-07:00 ERRO CBAS.apache.asterix [QueryTranslator] 
> Unexpected exception
> java.lang.ArithmeticException: / by zero
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.JobBuilder.(JobBuilder.java:87)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:58)
>  ~[algebricks-core-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)
>  ~[algebricks-compiler-0.3.2-SNAPSHOT.jar:0.3.2-SNAPSHOT]
>   at 
> org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:333)
>  

[jira] [Commented] (ASTERIXDB-2024) Generic issue for test/build/infra improvements

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115929#comment-16115929
 ] 

ASF subversion and git services commented on ASTERIXDB-2024:


Commit 68f7e43d884ed7ad47a4cc923a6f5518425ac26b in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=68f7e43 ]

[ASTERIXDB-2024][TEST] Test fwk comment patterns

Update the test framework to match comment patterns only at head of
line, to avoid truncating other instances of these characters i.e.
http://foobar.com -> http:

Change-Id: I4c317ccee242bd5fa2f5ec5bed85a69f0a47cf11
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1919
Reviewed-by: Michael Blow 
Integration-Tests: Michael Blow 
Tested-by: Michael Blow 


> Generic issue for test/build/infra improvements
> ---
>
> Key: ASTERIXDB-2024
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2024
> Project: Apache AsterixDB
>  Issue Type: Task
>  Components: OTH - Other, TEST - Test Framework
>Reporter: Michael Blow
>Assignee: Michael Blow
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2039) Log HttpServer direct memory budget on creation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133235#comment-16133235
 ] 

ASF subversion and git services commented on ASTERIXDB-2039:


Commit b879175e8373736e7de13bb63e924d875af74581 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=b879175 ]

[ASTERIXDB-2039][OTH] Log Http Server direct memory budget

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Calculate mem budget = number of executors x max high watermark.
- Log the calculated budget.

Change-Id: I4a324f10db52e77c7e69ca4246b9d84b4479f25d
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1941
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> Log HttpServer direct memory budget on creation
> ---
>
> Key: ASTERIXDB-2039
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2039
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2045) Failed flush shouldn't wait for lagging merge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131361#comment-16131361
 ] 

ASF subversion and git services commented on ASTERIXDB-2045:


Commit fcd89f5a92a80b2b69d99845471856a5909be9a8 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=fcd89f5 ]

[ASTERIXDB-2045][STO] Do Not Wait For Lagging Merge on Failed Flush

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Failed flush operations shouldn't wait for lagging merges
  since they won't add any new disk components.
- Use logger to log exceptions.

Change-Id: I915e993a76d5c692a276b1d7f3426a25f910cf46
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1947
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Failed flush shouldn't wait for lagging merge
> -
>
> Key: ASTERIXDB-2045
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2045
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> A failed flush shouldn't wait for lagging merge since it is not going to add 
> more disk components.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2044) Listify in subqueries

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131526#comment-16131526
 ] 

ASF subversion and git services commented on ASTERIXDB-2044:


Commit 5170fb212aa0e2c1e93327553fc38b6acd22c73b in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=5170fb2 ]

[ASTERIXDB-2044][COMP] Eliminate listify for complex group-by

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Fix EliminateSubplanWithInputCardinalityOneRule to handle recursive
  subplans;
- Fix various places that assumes the nested plans inside a group-by
  operator cannot be empty;
- Added regression tests.

Change-Id: Ida9aa8d89a89f90256e54c8c1806af9b4a162d21
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1946
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Yingyi Bu 


> Listify in subqueries
> -
>
> Key: ASTERIXDB-2044
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2044
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: COMP - Compiler
>Reporter: Yingyi Bu
>Assignee: Yingyi Bu
>
> The following query will result in unnecessary listifies in the optimized 
> query plan.
> {noformat}
> DROP  DATAVERSE tpch IF EXISTS;
> CREATE  dataverse tpch;
> USE tpch;
> CREATE TYPE LineItemType AS CLOSED {
>   l_orderkey : integer,
>   l_partkey : integer,
>   l_suppkey : integer,
>   l_linenumber : integer,
>   l_quantity : double,
>   l_extendedprice : double,
>   l_discount : double,
>   l_tax : double,
>   l_returnflag : string,
>   l_linestatus : string,
>   l_shipdate : string,
>   l_commitdate : string,
>   l_receiptdate : string,
>   l_shipinstruct : string,
>   l_shipmode : string,
>   l_comment : string
> }
> CREATE DATASET LineItem(LineItemType) PRIMARY KEY l_orderkey,l_linenumber;
> SELECT l_returnflag AS l_returnflag,
>l_linestatus AS l_linestatus,
>coll_count(cheap) AS count_cheaps,
>coll_count(expensive) AS count_expensives
> FROM LineItem AS l
> /* +hash */
> GROUP BY l.l_returnflag AS l_returnflag,l.l_linestatus AS l_linestatus
> GROUP AS g
> LET cheap = (
>   SELECT ELEMENT m
>   FROM (FROM g SELECT VALUE l) AS m
>   WHERE m.l_discount > 0.05
> ),
> expensive = (
>   SELECT ELEMENT m
>   FROM (FROM g SELECT VALUE l) AS m
>   WHERE m.l_discount <= 0.05
> )
> ORDER BY l_returnflag,l_linestatus
> ;
> {noformat}
> {noformat}
> distribute result [$$31]
> -- DISTRIBUTE_RESULT  |PARTITIONED|
>   exchange
>   -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
> project ([$$31])
> -- STREAM_PROJECT  |PARTITIONED|
>   assign [$$31] <- [{"l_returnflag": $$l_returnflag, "l_linestatus": 
> $$l_linestatus, "count_cheaps": $$36, "count_expensives": $$37}]
>   -- ASSIGN  |PARTITIONED|
> exchange
> -- SORT_MERGE_EXCHANGE [$$l_returnflag(ASC), $$l_linestatus(ASC) ]  
> |PARTITIONED|
>   project ([$$l_returnflag, $$l_linestatus, $$36, $$37])
>   -- STREAM_PROJECT  |PARTITIONED|
> subplan {
>   aggregate [$$37] <- [agg-count($$m)]
>   -- AGGREGATE  |LOCAL|
> select (le($$39, 0.05))
> -- STREAM_SELECT  |LOCAL|
>   assign [$$39] <- [$$m.getField(6)]
>   -- ASSIGN  |LOCAL|
> unnest $$m <- scan-collection($$24)
> -- UNNEST  |LOCAL|
>   subplan {
> aggregate [$$24] <- [listify($$23)]
> -- AGGREGATE  |LOCAL|
>   assign [$$23] <- [$$g.getField(0)]
>   -- ASSIGN  |LOCAL|
> unnest $$g <- 
> scan-collection($$15)
> -- UNNEST  |LOCAL|
>   nested tuple source
>   -- NESTED_TUPLE_SOURCE  |LOCAL|
>  }
>   -- SUBPLAN  |LOCAL|
> nested tuple source
> -- NESTED_TUPLE_SOURCE  |LOCAL|
>}
> -- SUBPLAN  |PARTITIONED|
>   subplan {
> aggregate [$$36] <- [agg-count($$m)]
> -- AGGREGATE  |LOCAL|
>   select (gt($$38, 0.05))
>   -- STREAM_SELECT  |LOCAL|
>  

[jira] [Commented] (ASTERIXDB-2049) Start Feed hangs indefinitely

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134294#comment-16134294
 ] 

ASF subversion and git services commented on ASTERIXDB-2049:


Commit 7f62b205c031b283641292521a94b11233f57ef6 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=7f62b20 ]

[ASTERIXDB-2049][ING] Improve logging for Active messages

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- With each active message, log the runtime which sent it.

Change-Id: I9663277cadf922b7e0ec224b056357b0f24bbb05
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1951
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> Start Feed hangs indefinitely
> -
>
> Key: ASTERIXDB-2049
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2049
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: ING - Ingestion
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1901) UDF fails at constant parameter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007479#comment-16007479
 ] 

ASF subversion and git services commented on ASTERIXDB-1901:


Commit bacf0c595eb0172522ce88a2600a6deaaafbc98b in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=bacf0c5 ]

ASTERIXDB-1901 Fix IntroduceDynamicTypeCastForExternalFunctionRule

1. Instead of pattern matching, now we will inspect every paramter of
   UDF. If there is a type mismatch, a cast function will be added.
2. Fixed the issue that type casting only applies to the first argument.
3. Added test case for this.

Change-Id: I6f44b2460ae3322fc52451e7939b6b5e711790a7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1730
Reviewed-by: Yingyi Bu 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 


> UDF fails at constant parameter
> ---
>
> Key: ASTERIXDB-1901
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1901
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>Assignee: Xikui Wang
>
> If the declared datatype for UDF input is a open type, UDF will fail when 
> given constant record (which has a closed datatype implicitly)
> Execute following query with default lib installed can reproduce this issue:
> {noformat}
> drop dataverse externallibtest if exists;
> create dataverse externallibtest;
> use dataverse externallibtest;
> create type TextType if not exists as open {
> id: int32,
> text: string
> };
> /* This will fail */
> let $i:=testlib#toUpper({"id":1, "text":"lower text"})
> return $i
> /* This works */
> let $i:={"id":1, "text":"lower text"}
> return testlib#toUpper($i)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1789) ArrayIndexOutOfBound in hybrid hash join

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006688#comment-16006688
 ] 

ASF subversion and git services commented on ASTERIXDB-1789:


Commit 7a0ba78d3bc03e064f8a9909daa50822a47003aa in asterixdb's branch 
refs/heads/master from [~wangsaeu]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=7a0ba78 ]

ASTERIXDB-1789: fix an accessor that is used by Hash Table

 - Fix an accesor that is used by SerializableHashTable class.
   This accessor class is required to calculate the original
   hash value of a tuple when compacting the hash table.

Change-Id: I74b6de8f189e841c16735dbd63f13892bbd61c20
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1734
Reviewed-by: Yingyi Bu 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 


> ArrayIndexOutOfBound in hybrid hash join
> 
>
> Key: ASTERIXDB-1789
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1789
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Yingyi Bu
>Assignee: Taewoo Kim
>Priority: Critical
>
> I have a two node cluster, where each node with one iodevice.
> The data is tpch query at scale 0.4.
> The query is q7:
> {noformat}
> USE tpch;
> WITH q7_volume_shipping_tmp AS
> (
> SELECT n1.n_name AS supp_nation,
>n2.n_name AS cust_nation,
>n1.n_nationkey AS s_nationkey,
>n2.n_nationkey AS c_nationkey
> FROM  Nation as n1,
>   Nation as n2
> WHERE (n1.n_name='FRANCE' AND n2.n_name='GERMANY') OR 
> (n1.n_name='GERMANY' AND n2.n_name='FRANCE')
> )
> SELECT supp_nation, cust_nation, l_year, sum(volume) AS revenue
> FROM
>   (
> SELECT t.supp_nation, t.cust_nation, GET_YEAR(l3.l_shipdate) AS l_year,
>l3.l_extendedprice * (1 - l3.l_discount) AS volume
> FROM q7_volume_shipping_tmp t JOIN
> (
>SELECT l2.l_shipdate, l2.l_extendedprice, l2.l_discount, 
> l2.c_nationkey, s.s_nationkey
>FROM Supplier s JOIN
>  (
>   SELECT l1.l_shipdate, l1.l_extendedprice, l1.l_discount, 
> l1.l_suppkey, c.c_nationkey
>   FROM Customer c JOIN
>  (
>SELECT l.l_shipdate, l.l_extendedprice, l.l_discount, 
> l.l_suppkey, o.o_custkey
>FROM Orders o
>JOIN LineItem l ON o.o_orderkey = l.l_orderkey AND 
> l.l_shipdate >= '1995-01-01'
> AND l.l_shipdate <= '1996-12-31'
>) l1 ON c.c_custkey = l1.o_custkey
> ) l2 ON s.s_suppkey = l2.l_suppkey
>  ) l3 ON t.c_nationkey = l3.c_nationkey AND t.s_nationkey = 
> l3.s_nationkey
>) shipping
> GROUP BY supp_nation, cust_nation, l_year
> ORDER BY supp_nation, cust_nation, l_year;
> {noformat}
> {noformat}
> org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
> HYR0003: 32768
> at 
> org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:223)
> at 
> org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:50)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: 
> 32768
> at 
> org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
> at org.apache.hyracks.control.nc.Task.run(Task.java:330)
> ... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: 32768
> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:378)
> at org.apache.hyracks.control.nc.Task.run(Task.java:308)
> ... 3 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 32768
> at 
> org.apache.hyracks.data.std.accessors.MurmurHash3BinaryHash.hash(MurmurHash3BinaryHash.java:46)
> at 
> org.apache.asterix.dataflow.data.nontagged.hash.AMurmurHash3BinaryHashFunctionFamily$1.hash(AMurmurHash3BinaryHashFunctionFamily.java:103)
> at 
> org.apache.hyracks.dataflow.common.data.partition.FieldHashPartitionComputerFamily$1.partition(FieldHashPartitionComputerFamily.java:56)
> at 
> org.apache.hyracks.dataflow.std.structures.SerializableHashTable.updateHeaderToContentPointerInHeaderFrame(SerializableHashTable.java:392)
> at 
> 

[jira] [Commented] (ASTERIXDB-1842) start-sample-cluster.sh with user name ending with space.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010689#comment-16010689
 ] 

ASF subversion and git services commented on ASTERIXDB-1842:


Commit d0324c201c919067a904cb1000bb8cbca1f7bf0a in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=d0324c2 ]

Handle Spaces in Path For Sample Cluster

Fixes ASTERIXDB-1842- start-sample-cluster.sh with user name ending with
space.

Change-Id: Id4b17698762e3cb2959152e6f957dad175d0f99b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1747
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> start-sample-cluster.sh with user name ending with space.
> -
>
> Key: ASTERIXDB-1842
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1842
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>Assignee: Michael Blow
>Priority: Minor
>
> One user has an user name ending with an extra space. The script doesn't 
> recognize this in the space detection part with a proper feedback.
> This is under OSX.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1940) Report number of errors

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046789#comment-16046789
 ] 

ASF subversion and git services commented on ASTERIXDB-1940:


Commit c1bd877b183f6b634f0a9d9208ed70f4d4fc6946 in asterixdb's branch 
refs/heads/master from [~tillw]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=c1bd877 ]

ASTERIXDB-1940: Report number of errors

Change-Id: I2d7fbf351cc0fc21f47cf0b9d82a8b72bc5098e9
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1831
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> Report number of errors
> ---
>
> Key: ASTERIXDB-1940
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1940
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: AsterixDB, HTTP API
>Reporter: Till
>Assignee: Till
>
> Add the number of errors that occurred during request processing to the 
> metrics section of the query/service API.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1938) AppendOnlyLinkedMetadataPageManager.put() is called after close(). HYR0012 is thrown

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046945#comment-16046945
 ] 

ASF subversion and git services commented on ASTERIXDB-1938:


Commit 16b57d8e39aa877aa6b4bbb124a261727d46409d in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=16b57d8 ]

ASTERIXDB-1938: Keep empty disk components when merge/flush

During flush/merge, it is possible that the new disk component
is empty. However, the current LSMDiskComponentBulkLoader would
clean up empty disk component. This patch fix this behavior by
introducing an extra parameter cleanupEmptyComponent.

Change-Id: Ic4f7ec41f56a2f9124920d67657f88160634f0e7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1826
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 
Reviewed-by: Yingyi Bu 


> AppendOnlyLinkedMetadataPageManager.put() is called after close(). HYR0012 is 
> thrown
> 
>
> Key: ASTERIXDB-1938
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1938
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: Storage
>Reporter: Dmitry Lychagin
>Assignee: Chen Luo
>
> We're getting this exception:
> rg.apache.hyracks.api.exceptions.HyracksDataException: HYR0012: Invalid 
> attempt to write to a flushed append only metadata page
>   at 
> org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:49)
>   at 
> org.apache.hyracks.storage.am.common.freepage.AppendOnlyLinkedMetadataPageManager.put(AppendOnlyLinkedMetadataPageManager.java:311)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.DiskComponentMetadata.put(DiskComponentMetadata.java:38)
>   at 
> org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.putLSNIntoMetadata(AbstractLSMIOOperationCallback.java:105)
>   at 
> org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.afterOperation(AbstractLSMIOOperationCallback.java:188)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.flush(LSMHarness.java:503)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.flush(LSMTreeIndexAccessor.java:121)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.FlushOperation.call(FlushOperation.java:42)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.FlushOperation.call(FlushOperation.java:30)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> However at that time the AppendOnlyLinkedMetadataPageManager is already 
> closed.
> The stack trace for the close() method invocation is the following:
> at 
> org.apache.hyracks.storage.am.common.freepage.AppendOnlyLinkedMetadataPageManager.close(AppendOnlyLinkedMetadataPageManager.java:216)
>   at 
> org.apache.hyracks.storage.am.common.impls.AbstractTreeIndex.deactivate(AbstractTreeIndex.java:163)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMDiskComponentBulkLoader.cleanupArtifacts(AbstractLSMDiskComponentBulkLoader.java:170)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMDiskComponentBulkLoader.end(AbstractLSMDiskComponentBulkLoader.java:158)
>   at 
> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.flush(LSMBTree.java:357)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.flush(LSMHarness.java:502)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.flush(LSMTreeIndexAccessor.java:121)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.FlushOperation.call(FlushOperation.java:42)
>   at 
> org.apache.hyracks.storage.am.lsm.common.impls.FlushOperation.call(FlushOperation.java:30)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> So AppendOnlyLinkedMetadataPageManager.close() is called from LSMHarness:502 
> (flush), then the next line (LSMHarness:503) calls the callback which tries 
> to write data into the closed page manager. At that point its 
> 

[jira] [Commented] (ASTERIXDB-1945) Fix BufferCache API/Lifecycle

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057692#comment-16057692
 ] 

ASF subversion and git services commented on ASTERIXDB-1945:


Commit 198ffbca2eac15d542ea46ea863cebde1fa29138 in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=198ffbc ]

[ASTERIXDB-1945][STO] Cleanup Buffer Cache API

- user model changes: no
- storage format changes: no
- interface changes: yes
  INcApplicationContext
   - removed IFileMapProvider getFileMapManager();
 to hide FileMapManager from other components;
  IStorageManager
   - IFileMapProvider getFileMapProvider(INCServiceContext ctx);
 to hide FileMapManager from other components;
  IFileHandle
   - added FileReference getFileReference();
 to avoid unnecessary casts;
  IIOManager
   - public void deleteWorkspaceFiles() throws
  HyracksDataException;
 added throws;
  ILSMIndexFileManager
   - void createDirs() throws HyracksDataException;
 added throws;
  IInvertedIndex
   - added void purge() throws HyracksDataException;
 a. InvertedIndexes don't implement the ITreeIndex interface.
 b. when we deactivate a disk component, we need to purge it so
the buffer cache doesn't go through each page.
 c. this need to be revisited, ASTERIXDB-1944
  IFileMapManager
   - int registerFile(FileReference fileRef) throws
HyracksDataException;
 return value added for future reference of the index file
 inside BufferCache or VirtualBufferCache;
   - FileReference unregisterFile(int fileId) throws
HyracksDataException;
 return value added for future refernece of the file;
  IBufferCache
   - int createFile(FileReference fileRef) throws
  HyracksDataException;
 return value added for future reference of the index file
 inside BufferCache or VirtualBufferCache;
   - void deleteFile(int fileId) throws HyracksDataException;
 remove the dirty page flag since there's no dirty page;
   - int openFile(FileReference fileRef) throws
 HyracksDataException;
 return value added for future reference of the index file
 inside BufferCache or VirtualBufferCache;
   - added void deleteFile(FileReference file) throws
   HyracksDataException;
 we used to have this public methods in both BufferCache
 and VirtualBufferCache. Now we lifted it into the interface.
  AbstractLSMIndex
   - removed protected abstract void
 destroyMemoryComponent(ILSMMemoryComponent c)
 throws HyracksDataException;
 It is because turned out when we deactivate, we actually
 destroy them. However, because of the not well defined API,
 double destroy was okay and so we used to do double destroy.

Details:
This change fixes the buffer cache to follow the API such that:
1. createFile creates the file.
2. deleteFile deletes the file.
3. openFile opens the file.
4. closeFile closes the file.
5. creates existing file is not allowed.
6. deletes deleted file is not allowed.
7. open non existing file is not allowed.

In addition, we hide the file map from all other components.

Change-Id: I0a973c2adb2e7fdcbbf18c7b888af3de5f0acc74
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1843
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Fix BufferCache API/Lifecycle
> -
>
> Key: ASTERIXDB-1945
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1945
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Currently, BufferCache has an interesting behavior such as:
> 1. CreateFile doesn't create a file but only create a name-id mapping in 
> memory.
> 2. To get the id of a file, the caller must have access to the map associated 
> with the buffer cache.
> 3. openFile. If create file was called before openFile, then it creates the 
> file. If the file already exists, it deletes it and create a new file.
> This should be fixed and given clear behavior that matches the expected API. 
> All usages of the buffer cache must be fixed as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1944) Align LSM Inverted index and its individual components with other types of indexes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057693#comment-16057693
 ] 

ASF subversion and git services commented on ASTERIXDB-1944:


Commit 198ffbca2eac15d542ea46ea863cebde1fa29138 in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=198ffbc ]

[ASTERIXDB-1945][STO] Cleanup Buffer Cache API

- user model changes: no
- storage format changes: no
- interface changes: yes
  INcApplicationContext
   - removed IFileMapProvider getFileMapManager();
 to hide FileMapManager from other components;
  IStorageManager
   - IFileMapProvider getFileMapProvider(INCServiceContext ctx);
 to hide FileMapManager from other components;
  IFileHandle
   - added FileReference getFileReference();
 to avoid unnecessary casts;
  IIOManager
   - public void deleteWorkspaceFiles() throws
  HyracksDataException;
 added throws;
  ILSMIndexFileManager
   - void createDirs() throws HyracksDataException;
 added throws;
  IInvertedIndex
   - added void purge() throws HyracksDataException;
 a. InvertedIndexes don't implement the ITreeIndex interface.
 b. when we deactivate a disk component, we need to purge it so
the buffer cache doesn't go through each page.
 c. this need to be revisited, ASTERIXDB-1944
  IFileMapManager
   - int registerFile(FileReference fileRef) throws
HyracksDataException;
 return value added for future reference of the index file
 inside BufferCache or VirtualBufferCache;
   - FileReference unregisterFile(int fileId) throws
HyracksDataException;
 return value added for future refernece of the file;
  IBufferCache
   - int createFile(FileReference fileRef) throws
  HyracksDataException;
 return value added for future reference of the index file
 inside BufferCache or VirtualBufferCache;
   - void deleteFile(int fileId) throws HyracksDataException;
 remove the dirty page flag since there's no dirty page;
   - int openFile(FileReference fileRef) throws
 HyracksDataException;
 return value added for future reference of the index file
 inside BufferCache or VirtualBufferCache;
   - added void deleteFile(FileReference file) throws
   HyracksDataException;
 we used to have this public methods in both BufferCache
 and VirtualBufferCache. Now we lifted it into the interface.
  AbstractLSMIndex
   - removed protected abstract void
 destroyMemoryComponent(ILSMMemoryComponent c)
 throws HyracksDataException;
 It is because turned out when we deactivate, we actually
 destroy them. However, because of the not well defined API,
 double destroy was okay and so we used to do double destroy.

Details:
This change fixes the buffer cache to follow the API such that:
1. createFile creates the file.
2. deleteFile deletes the file.
3. openFile opens the file.
4. closeFile closes the file.
5. creates existing file is not allowed.
6. deletes deleted file is not allowed.
7. open non existing file is not allowed.

In addition, we hide the file map from all other components.

Change-Id: I0a973c2adb2e7fdcbbf18c7b888af3de5f0acc74
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1843
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Align LSM Inverted index and its individual components with other types of 
> indexes
> --
>
> Key: ASTERIXDB-1944
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1944
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> The bad design is causing many methods which throws 
> UnsupportedOperationException such the following in LSMInvertedIndex as:
> @Override
> public IInvertedListCursor createInvertedListCursor() {
> throw new UnsupportedOperationException("Cannot create inverted list 
> cursor on lsm inverted index.");
> }
> @Override
> public void openInvertedListCursor(IInvertedListCursor listCursor, 
> ITupleReference searchKey,
> IIndexOperationContext ictx) throws HyracksDataException {
> throw new UnsupportedOperationException("Cannot open inverted list 
> cursor on lsm inverted index.");
> }
> @Override
> public void purge() throws HyracksDataException {
> throw new 

[jira] [Commented] (ASTERIXDB-1943) Make rebalance operation idempotent

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061176#comment-16061176
 ] 

ASF subversion and git services commented on ASTERIXDB-1943:


Commit a32a852bef6d15f0d99b85d548a99d8ab88b4bba in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=a32a852 ]

[ASTERIXDB-1943][API][STO] Make rebalance idempotent.

- user model changes:
  added rebalance cancellation HTTP API.

- storage format changes: no

- interface changes: no

Details:
- add a HTTP API for cancelling a rebalance request;
- clean up leftover states at the beginning of a
  rebalance request;
- add tests for rebalance cancellation.

Change-Id: I0d14a07978e106cd497cc35538fafef318b2fcf7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1821
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Make rebalance operation idempotent
> ---
>
> Key: ASTERIXDB-1943
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1943
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: *DB - AsterixDB
>Reporter: Yingyi Bu
>Assignee: Till
>
> Add a rebalance cancellation API and make the rebalance operation idempotent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1951) Cleanup type constructor functions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060457#comment-16060457
 ] 

ASF subversion and git services commented on ASTERIXDB-1951:


Commit fe00b1af941382c3f47737e9b0d5a9b667365dc7 in asterixdb's branch 
refs/heads/master from [~dlychagin-cb]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=fe00b1a ]

[ASTERIXDB-1951][FUN] Cleanup type constructor functions

- user model changes: yes

1) primitive type constructor functions now accept input values
   of their respective types and return them as is
2) removed type constructor function for 'null' type

- storage format changes: no
- interface changes: no

Change-Id: I4a6627bfcc302b2621df8d7837c15434970b9d3a
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1846
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Yingyi Bu 


> Cleanup type constructor functions
> --
>
> Key: ASTERIXDB-1951
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1951
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: FUN - Functions
>Reporter: Dmitry Lychagin
>
> 1) Currently primitive type constructor functions behavior is inconsistent 
> when input value has the same type as the constructed type.
> Constructor functions for 'string' and 'binary' types allow input value to be 
> 'string'/'binary' and just return it as is, while other functions raise type 
> mismatch exception.
> We should make sure that all constructor functions for primitive types pass 
> through values of their respective types.
> 2) Current implementation for 'null' type constructor function is incorrect. 
> It returns 'missing' instead of 'null'. Moreover it's unclear whether this 
> function is necessary at all, because
> - we already have a keyword for creating 'null's 
> - the function only accepts one string value: 'null', while raising error for 
> all other strings. 
> The proposal is to remove this type constructor function from the function 
> library



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1953) Clean LSM File Managers

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061380#comment-16061380
 ] 

ASF subversion and git services commented on ASTERIXDB-1953:


Commit e0cf970124baed319800d58d107c4bfbe98ae84a in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=e0cf970 ]

[ASTERIXDB-1953][STO] Cleanup LSM File Managers

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- use FileReference instead of String absolute path
- user error codes

Change-Id: I97bab76888790ca282ad9508ce8416f7c7a52fb7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1849
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
BAD: Jenkins 
Reviewed-by: Ian Maxon 


> Clean LSM File Managers
> ---
>
> Key: ASTERIXDB-1953
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1953
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Currently, LSM File managers store all the paths as absolute path Strings. 
> This makes the code bug prone, and requires constructing multiple File 
> objects. and using the IO manager to resolve the io device.
> More over, all the error messages in these classes are outdated and use the 
> old mechanism of creating error messages.
> Instead of using absolute path strings, we should use FileReferences.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1443) No unit test cases for FrameDistributor

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061786#comment-16061786
 ] 

ASF subversion and git services commented on ASTERIXDB-1443:


Commit d1d047b5e4d8a097c6b0c9d49058fc4ffe137cf6 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=d1d047b ]

[ASTERIXDB-1443][FEED] Remove Frame Distributor

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- FrameDistributor and DistributeFeedFrameWriter are not used
  anymore.

Change-Id: I27c1ff99ce797923dd709d181387560e4f9448a5
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1853
Sonar-Qube: Jenkins 
Reviewed-by: Xikui Wang 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 


> No unit test cases for FrameDistributor
> ---
>
> Key: ASTERIXDB-1443
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1443
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> A frame distributor is a special frame. it has a very specialized behavior 
> and it requires its own unit tests to ensure that it works as expected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1944) Align LSM Inverted index and its individual components with other types of indexes

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061926#comment-16061926
 ] 

ASF subversion and git services commented on ASTERIXDB-1944:


Commit 63ab05d2a60c5b18bba3e9e4a350a79abfc728f6 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=63ab05d ]

[ASTERIXDB-1944][STO] Align LSM inverted index with other indexes

- user model changes: no
- storage format changes: no
- interface changes: yes
  IInvertedIndexComponent was introduced to separate LSM inverted
  index from its component and avoid multiple MethodNotSupported
  exceptions

Change-Id: I0fb7b446edfb14de25ecaf99965ae0a7325101c9
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1852
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> Align LSM Inverted index and its individual components with other types of 
> indexes
> --
>
> Key: ASTERIXDB-1944
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1944
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> The bad design is causing many methods which throws 
> UnsupportedOperationException such the following in LSMInvertedIndex as:
> @Override
> public IInvertedListCursor createInvertedListCursor() {
> throw new UnsupportedOperationException("Cannot create inverted list 
> cursor on lsm inverted index.");
> }
> @Override
> public void openInvertedListCursor(IInvertedListCursor listCursor, 
> ITupleReference searchKey,
> IIndexOperationContext ictx) throws HyracksDataException {
> throw new UnsupportedOperationException("Cannot open inverted list 
> cursor on lsm inverted index.");
> }
> @Override
> public void purge() throws HyracksDataException {
> throw new UnsupportedOperationException();
> }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1869) Ensure that size of lock list equals to the number of locked metadata entities.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061927#comment-16061927
 ] 

ASF subversion and git services commented on ASTERIXDB-1869:


Commit d8536d7a108f03c705a9f82c953f7ae296d394e5 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=d8536d7 ]

[ASTERIXDB-1869][MTD][TX] Control Lock size

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Previously, each lock call to the metadata lock manager increment
  the lock counter and create a new Pair object. After
  this change, the lock manager checks if the lock was obtained
  previously.

Change-Id: Icc884949a824d64fcf27e26c350a2bc0fa496141
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1854
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Reviewed-by: Murtadha Hubail 


> Ensure that size of lock list equals to the number of locked metadata 
> entities.
> ---
>
> Key: ASTERIXDB-1869
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1869
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1946) Support Create Secondary Index for Correlated Datasets

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16056027#comment-16056027
 ] 

ASF subversion and git services commented on ASTERIXDB-1946:


Commit a7fa05bb014503609d1b6287fa08b0a296098088 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=a7fa05b ]

[ASTERIXDB-1946][STO][IDX] Create BTreeIndex for Correlated Datasets

Implemented create seconary BTree index for datasets using
correlated merge policy. Instead of creating one component
for the new index, this change creates one component for each
component of the primary index to maintain the correlation.
The current implementation assumes when a secondary index is being
created, the dataset is locked with no modifications.

Change-Id: I2a3435e6720f07bd6a5092d4d9ce42e8d4b7894c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1813
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Yingyi Bu 
BAD: Jenkins 
Integration-Tests: Jenkins 


> Support Create Secondary Index for Correlated Datasets
> --
>
> Key: ASTERIXDB-1946
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1946
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: IDX - Indexes, STO - Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> Currently, when we create a secondary index, the result index would contain 
> only one disk component. This is undesirable for datasets using correlated 
> merge policy. Instead, it would be desirable for the new index to create one 
> disk component for each component of the primary index.
> This issue would implement this behavior for secondary BTree, RTree and 
> InvertedIndex.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1945) Fix BufferCache API/Lifecycle

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054894#comment-16054894
 ] 

ASF subversion and git services commented on ASTERIXDB-1945:


Commit d900046959197958b01505c218d79795807b177f in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=d900046 ]

Revert "ASTERIXDB-1945 [STO] Cleanup Buffer Cache API"

This reverts commit ae3daf6ef3397e583637360dc460c6391e03dc29.

Change-Id: I5e4e23f43a68e82c38fb8d1d7f4c0d01985c3a10
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1842
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 


> Fix BufferCache API/Lifecycle
> -
>
> Key: ASTERIXDB-1945
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1945
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Currently, BufferCache has an interesting behavior such as:
> 1. CreateFile doesn't create a file but only create a name-id mapping in 
> memory.
> 2. To get the id of a file, the caller must have access to the map associated 
> with the buffer cache.
> 3. openFile. If create file was called before openFile, then it creates the 
> file. If the file already exists, it deletes it and create a new file.
> This should be fixed and given clear behavior that matches the expected API. 
> All usages of the buffer cache must be fixed as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1915) Dataset partition files does not uniformly distributed across IO devices

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021951#comment-16021951
 ] 

ASF subversion and git services commented on ASTERIXDB-1915:


Commit 54f7c9f3067be2d5661b074c445c0a32c1b7f1c3 in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=54f7c9f ]

ASTERIXDB-1915: make dataset files uniformly distributed among io devices.

Change-Id: I2dd9e17e96c1d4ef55e29d0a0f8feadf8ce321ed
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1770
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Dataset partition files does not uniformly distributed across IO devices
> 
>
> Key: ASTERIXDB-1915
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1915
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: Storage
>Reporter: Yingyi Bu
>Assignee: Yingyi Bu
>
> When we create a dataset, the partitioned physical files can distribute 
> across iodevices in a non-uniform way,  because there are multiple calling 
> sites for DefaultDeviceComputer.compute(path) for creating each index 
> partition.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1917) FLUSH_LSN for disk components is not correctly set when a NC holds multiple partitions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022298#comment-16022298
 ] 

ASF subversion and git services commented on ASTERIXDB-1917:


Commit 1e51daacb8aacc42184e76fda1f2cd6b0eb2e824 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=1e51daa ]

ASTERIXDB-1917: FLUSH_LSN for disk components is not correctly set

-Fixed a bug that FLUSH_LSN for flushed disk components is not
correctly set (not increasing) when an NC has multiple partitions.
-Added LSMIOOperationCallback unit tests to cover this bug

Change-Id: If438e34f8f612458d81f618eea04c0c72c49a9fe
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1771
Reviewed-by: abdullah alamoudi 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 


> FLUSH_LSN for disk components is not correctly set when a NC holds multiple 
> partitions
> --
>
> Key: ASTERIXDB-1917
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1917
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: Hyracks, Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
> Attachments: asterix-build-configuration-lsm.xml, sample.zip
>
>
> When we flush a memory component of an index, we would set an LSN to the 
> result disk component. The LSN is set as the last operation which modifies 
> that memory component. Thus, given an index, the FLUSH_LSNs of its flushed 
> disk components should be increasing, i.e., later flushed components get 
> larger LSNs.
> However, currently I observed a bug that later flushed disk components get a 
> smaller LSN, which breaks this properly.
> A brief explanation of this bug is as follows. Suppose we have one dataset D, 
> and two partitions on one NC. Suppose D only has a primary index P. Further 
> suppose P1 and P2 are two partitioned indexes for the two partitions on this 
> NC. This implies P1 and P2 share the same PrimaryIndexOperationTracker, and 
> they would always be flushed together.
> The LSN for flushed disk components is ILSMIOOperationCallback. Now suppose 
> an index has two memory components. ILSMIOOperationCallback maintains an 
> array mutableLastLSNs of length 2 to track the FLUSH_LSN for two memory 
> components. Before scheduling each flush operation, 
> PrimaryIndexOperationTracker needs to call ILSMIOOperationCallback to set the 
> FLUSH_LSN.
> Now consider the following scenario (which happens but very rarely). 
> Initially,
> P1.mutableLastLSNs=[0,0]
> P2.mutableLastLSNs=[0,0]
> Suppose dataset D needs to be flushed, and mutableLastLSNs is set as follows:
> P1.mutableLastLSNs=[1, 0]
> P2.mutableLastLSNs=[1, 0]
> Then, suppose the flush operation of P2 is fast, and produces a disk 
> component P2.d1 (P2.d1.LSN = 1). Data continues to come into P2, and it needs 
> to be flushed again. However, P1 is still flushing the first memory 
> component. Then mutableLastLSNs become:
> P1.mutableLastLSNs=[1, 2]
> P2.mutableLastLSNs=[1, 2].
> Surprisingly, the flush operation of P2 is again fast, and produces a disk 
> component P2.d2 (P2.d2.LSN = 2). Still, data continues to come into P2, and 
> it needs to be flushed again. But P1 is still flushing the first memory 
> component. Then mutableLastLSNs become:
> P1.mutableLastLSNs=[3, 2]
> P2.mutableLastLSNs=[3, 2].
> At this time, P1 finishes its first flush operation, and produced the disk 
> component P1.d1 (P1.d1.LSN = 3). This in incorrect, since P1.d1.LSN should be 
> 1, not 3. Its original value is overwritten by the flush request of P2!
> To reproduce this bug, one needs to change the codebase slightly (i.e., 
> LSMBTreeIOOperationCallback). I added a member variable
> {code}
>  private volatile long prevLSN = 0;
> {code}
> and added one check in the getComponentLSN method:
> {code}
>  @Override
> public long getComponentLSN(List diskComponents) 
> throws HyracksDataException {
> if (diskComponents == null) {
> // Implies a flush IO operation. --> moves the flush pointer
> // Flush operation of an LSM index are executed sequentially.
> synchronized (this) {
> long lsn = mutableLastLSNs[readIndex];
> if (!(prevLSN <= lsn)) {
> throw new IllegalStateException();
> }
> prevLSN = lsn;
> return lsn;
> }
> }
> // Get max LSN from the diskComponents. Implies a merge IO operation 
> or Recovery operation.
> long maxLSN = -1L;
> for (ILSMComponent c : diskComponents) {
> BTree btree = 

[jira] [Commented] (ASTERIXDB-1905) Filter doesn't created correct if create index using bulkload

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012939#comment-16012939
 ] 

ASF subversion and git services commented on ASTERIXDB-1905:


Commit 9861cbfb710c7bf1c72a19923ad3413d938c90cd in asterixdb's branch 
refs/heads/master from [~imaxon]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=9861cbf ]

ASTERIXDB-1905: Incorrect filter for post-load sidx

The issue seems to be limited to RTrees. The MBR evaluators
were not accounting for the point MBR optimization as they
were in the compiled load pidx+sidx case

Change-Id: Iea158ad4c29ad4421020a28a72e68637bc538560
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1743
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Jianfeng Jia 


> Filter doesn't created correct if create index using bulkload
> -
>
> Key: ASTERIXDB-1905
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1905
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Jianfeng Jia
>Assignee: Ian Maxon
>Priority: Blocker
>
> It seems we are not passing the correct(or enough) information to the 
> secondary index BulkloadOperator when it is created afterward if the data has 
> been ingested already. During the bulkload, only the secondary keys are sent 
> to the index, while the filter information is pointed to the first value of 
> the next secondary keys, which makes the filter information incorrect. 
> We actually have several test cases for it, e.g., 
> `filters/load-with-secondary-rtree`. However, the test is passed just by 
> chance. Since the filter type tag is *double* (comes from the first key of 
> the secondary keys), and the search query is *datetime* , and the filter 
> satisfies function will only call the rawComparator by comparing the first 
> byte, which always matches *index.filterValue < query.filterValue* .  And we 
> happened to choose the *$m.send-time < datetime("2012-11-20T10:10:00.000Z")* 
> query. If we choose the *>* relation, then nothing is returned. 
> I mark it *blocking* because first it's a correctness problem, and it's also 
> blocking my *pass-2ndary-filter-to-primary*  patch. Thanks for the help!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1928) Crash during disk component deletion could lead to inability to load lsm index

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045552#comment-16045552
 ] 

ASF subversion and git services commented on ASTERIXDB-1928:


Commit 3c07fe545021d513611a704f6625948408f4852e in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=3c07fe5 ]

ASTERIXDB-1928: Regression test

Change-Id: I3e0c8702e6cdf27c2caed6a77bc182ea3dbbc3fd
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1822
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Crash during disk component deletion could lead to inability to load lsm index
> --
>
> Key: ASTERIXDB-1928
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1928
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Abdullah Alamoudi
>Assignee: Murtadha Hubail
>
> If the crash leaves a btree and not a bloom filter for example, then we would 
> always get.
> Unequal number of valid BTree and bloom filter files found. Aborting cleanup.
> Check out LSMBTreeFileManager.cleanupAndGetValidFiles()



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1939) Query Service API returns content-length for chunked transfer encoding

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045846#comment-16045846
 ] 

ASF subversion and git services commented on ASTERIXDB-1939:


Commit f0759c85b70d18fe9376cdc034ed9e70e1ec0c74 in asterixdb's branch 
refs/heads/master from [~tillw]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=f0759c8 ]

ASTERIXDB-1939: No chunked encoding with content-length.

Change-Id: I06d561eb023f1c84c531e9b2cfe88a626d7e5280
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1632
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Reviewed-by: Michael Blow 
Integration-Tests: Jenkins 


> Query Service API returns content-length for chunked transfer encoding
> --
>
> Key: ASTERIXDB-1939
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1939
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: HTTP API
>Reporter: Till
>Assignee: Till
>
> In some cases the Hyracks HTTP server returns responses that specify that the 
> {{chunked}} transfer encoding is used and they also provide a 
> {{Content-Length}} header. We have seen at least 2 clients that do not handle 
> this combination.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1897) NPE in complex group-by query

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994282#comment-15994282
 ] 

ASF subversion and git services commented on ASTERIXDB-1897:


Commit a8af14319114afee37520ac5310db3a7f316a110 in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=a8af143 ]

ASTERIXDB-1897: fix MoveFreeVariableOperatorOutOfSubplanRule.

Change-Id: If3e9f7cba7ec20e51de9160df598ebcbe88c784e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1718
Reviewed-by: Till Westmann 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 


> NPE in complex group-by query
> -
>
> Key: ASTERIXDB-1897
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1897
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Yingyi Bu
>Assignee: Yingyi Bu
>
> The DDLs and query are as follows:
> {noformat}
> DROP DATAVERSE gby IF EXISTS;
> CREATE DATAVERSE gby;
> USE gby;
> CREATE TYPE PolicyType AS {
> id: UUID
> }
> CREATE DATASET policies(PolicyType) PRIMARY KEY id AUTOGENERATED;
> INSERT INTO policies
> (
>   [ {
> "policyno": "C123",
> "state": "CA",
> "zipcode": "96008",
> "make": "Honda",
> "accidents": [
>   {
> "year": "2015",
> "cost": 5000
>   },
>   {
> "year": "2016",
> "cost": 8000
>   },
>   {
> "year": "2016",
> "cost": 6000
>   }
> ]
>   },
>   {
> "policyno": "C124",
> "state": "CA",
> "zipcode": "96853",
> "make": "Ford",
> "accidents": [
>   {
> "year": "2015",
> "cost": 5000
>   },
>   {
> "year": "2015",
> "cost": 8000
>   },
>   {
> "year": "2016",
> "cost": 6000
>   }
> ]
>   },
>   {
> "policyno": "A123",
> "state": "AZ",
> "zipcode": "86008",
> "make": "Honda",
> "accidents": [
> {
>   "year": "2015",
>   "cost": 5000
> },
> {
>   "year": "2016",
>   "cost": 8000
> },
> {
>   "year": "2016",
>   "cost": 6000
> }
> ]
>   },
>   {
> "policyno": "A124",
> "state": "AZ",
> "zipcode": "86853",
> "make": "Ford",
> "accidents": [
> {
>   "year": "2015",
>   "cost": 5000
> },
> {
>   "year": "2016",
>   "cost": 8000
> },
> {
>   "year": "2016",
>   "cost": 6000
> }
> ]
>   },
>   {
> "policyno": "U123",
> "state": "UT",
> "zipcode": "66008",
> "make": "Honda",
> "accidents": [
> {
>   "year": "2015",
>   "cost": 5000
> },
> {
>   "year": "2016",
>   "cost": 8000
> },
> {
>   "year": "2016",
>   "cost": 6000
> }
> ]
>   },
>   {
> "policyno": "U124",
> "state": "UT",
> "zipcode": "66853",
> "make": "Ford",
> "accidents": [ ]
>   },
>   {
> "policyno": "U125",
> "state": "UT",
> "zipcode": "66853",
> "make": "Ford"
>   } ]
> );
> FROM policies p
> GROUP BY state GROUP AS g
> SELECT state,
>(
>  FROM g
>  SELECT VALUE SUM(
>   (
> FROM g.p.accidents a
> WHERE a.year = "2016"
> SELECT VALUE COUNT(*)
>)[0]
> )
>)[0]  / (COUNT(*) * 1.0 ) AS risk
> ORDER BY risk DESC
> LIMIT 5;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1885) Printer: Malformed result in closed object with MISSING fields

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993272#comment-15993272
 ] 

ASF subversion and git services commented on ASTERIXDB-1885:


Commit 4d2c7cd57295662cd6482ecd0f77a35eb60e8f8f in asterixdb's branch 
refs/heads/master from [~wyk]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=4d2c7cd ]

ASTERIXDB-1885: Fix printing field separators for record printer

- Fix commas to not appear at the beginning of a record.

Change-Id: I19e5c908367490a64104d961146bad2d870d0c58
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1692
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Yingyi Bu 


> Printer: Malformed result in closed object with MISSING fields
> --
>
> Key: ASTERIXDB-1885
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1885
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: AsterixDB
>Reporter: Wail Alkowaileet
>Assignee: Wail Alkowaileet
>
> I got a problem when dealing with closed objects/records that contain MISSING 
> fields.
> Examples:
> {noformat}
> 1: { }
> 2: {,"name":"Alice"}
> {noformat}
> I fixed the commas in (2). However, I'm not sure if (1) is still valid or if 
> it's an expected behavior.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-1918) Parser doesn't support scientific math number

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025015#comment-16025015
 ] 

ASF subversion and git services commented on ASTERIXDB-1918:


Commit 144453b3dd2a603af326936b46032c0b8f8c48d5 in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=144453b ]

ASTERIXDB-1918: support scientific math representation.

Change-Id: Ic0b6661416751a82a552cebc6248596b4eeff500
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1777
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
BAD: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> Parser doesn't support scientific math number
> -
>
> Key: ASTERIXDB-1918
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1918
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: Compiler
>Reporter: Yingyi Bu
>Assignee: Yingyi Bu
>
> For example:
> 1.0e-5, 2e10, 0.3e-2



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ASTERIXDB-2046) Gracefully handle concurrent reads on ClosedByInterruptException

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166500#comment-16166500
 ] 

ASF subversion and git services commented on ASTERIXDB-2046:


Commit 2e3c52015bfed6d6bed6c26124a3cf10b10864c5 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=2e3c520 ]

[ASTERIXDB-2046][STO] Retry Reading a Page on ClosedChannelException

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Currently when concurrent threads are reading from the same file
  and one of them is interrupted, the file channel will be closed and
  other threads will fail with AsynchronousCloseException if they were
  reading or ClosedChannelException if they were about to read the file.
  This change make the threads that were not interrupted retry the read
  operation.

Change-Id: I4f9de9f51596314d17e4c6a4a58333e8fd6c03d1
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2005
Sonar-Qube: Jenkins 
Reviewed-by: abdullah alamoudi 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 


> Gracefully handle concurrent reads on ClosedByInterruptException
> 
>
> Key: ASTERIXDB-2046
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2046
> Project: Apache AsterixDB
>  Issue Type: Improvement
>Reporter: Murtadha Hubail
>
> The fix in ASTERIXDB-2040 solves the issue of ClosedByInterruptException on 
> an index file by opening the file channel again. However, the concurrent 
> reads on the same file channels will fail when the exception is encountered 
> as well. This needs to be handled so that the concurrent reads are either 
> synchronized or retried after the file channel is opened again.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1973) Add Optional Parameters in Query Compilation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172722#comment-16172722
 ] 

ASF subversion and git services commented on ASTERIXDB-1973:


Commit e25df7d7acc0888ee2d946b878ea1a5af21147c0 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=e25df7d ]

[ASTERIXDB-1973][OTR] Add Optional Parameters in Query Compilation

- user model changes: no
- storage format changes: no
- interface changes:
Introduce IRequestParameters to encapsulate request parameters
and use it in IStatementExecutor#compileAndExecute and
IExtensionStatement#handle.

Details:
- Introduce IRequestParameters and its default implementation.
- Add optional parameters supplier in QueryServiceServlet.

Change-Id: Ie918f4d3f8dae41d07536041c591c59946a077f4
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1867
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Add Optional Parameters in Query Compilation
> 
>
> Key: ASTERIXDB-1973
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1973
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: OTH - Other
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> It would be nice to have optional parameters in query compilation so that 
> AsterixDB extension may use them to add any additional parameters.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2091) ConcurrentModificationException during TriggerNCWork

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166786#comment-16166786
 ] 

ASF subversion and git services commented on ASTERIXDB-2091:


Commit e2fa196f8e714acd9d6c7e85159cc4d20f0dfe9e in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=e2fa196 ]

[ASTERIXDB-2091][HYR][CLUS] Fix unsafe concurrent access to config on NC 
registration

Change-Id: I399d26d128794a0a52eff419f59ca28d64b17375
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2007
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> ConcurrentModificationException during TriggerNCWork
> 
>
> Key: ASTERIXDB-2091
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2091
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management, HYR - Hyracks
>Reporter: Michael Blow
>Assignee: Michael Blow
>
> java.util.ConcurrentModificationException
>   at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
>   at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
>   at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
>   at 
> org.apache.hyracks.control.common.config.ConfigManager.toIni(ConfigManager.java:460)
>   at 
> org.apache.hyracks.control.common.controllers.CCConfig.getIni(CCConfig.java:207)
>   at 
> org.apache.hyracks.control.cc.work.TriggerNCWork.lambda$run$0(TriggerNCWork.java:65)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1973) Add Optional Parameters in Query Compilation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172723#comment-16172723
 ] 

ASF subversion and git services commented on ASTERIXDB-1973:


Commit ed4aea2174ed67ab08340674e4036a6c77475d11 in asterixdb-bad's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb-bad.git;h=ed4aea2 ]

[ASTERIXDB-1973][BAD] Coordinated changes for IStatementExecutor

Change-Id: I392d7cda45e2bd39a85c037959ae5483eb48c9ee


> Add Optional Parameters in Query Compilation
> 
>
> Key: ASTERIXDB-1973
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1973
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: OTH - Other
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> It would be nice to have optional parameters in query compilation so that 
> AsterixDB extension may use them to add any additional parameters.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2120) intermittent NPE in JobletCleanupNotificationWork.runWork()

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189223#comment-16189223
 ] 

ASF subversion and git services commented on ASTERIXDB-2120:


Commit 950c6197ba2e374ea2171e7b8642ece901e4ce20 in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=950c619 ]

[ASTERIXDB-2120][HYR][RT] Avoid NPE in case of unknown job

Change-Id: I77267aeaf1a0f6498146fe6f0b5777eb6ba47054
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2049
Reviewed-by: Murtadha Hubail 
Integration-Tests: Michael Blow 
Tested-by: Michael Blow 
Contrib: Jenkins 


> intermittent NPE in JobletCleanupNotificationWork.runWork()
> ---
>
> Key: ASTERIXDB-2120
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2120
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: HYR - Hyracks, RT - Runtime
>Reporter: Michael Blow
>Assignee: Michael Blow
>
> 2017-10-01T17:18:48.890-07:00 INFO CBAS.work.JobletCleanupNotificationWork 
> [Worker:ClusterController] Exception thrown from work
> java.lang.NullPointerException: null
>   at 
> org.apache.hyracks.control.cc.work.JobletCleanupNotificationWork.runWork(JobletCleanupNotificationWork.java:54)
>  ~[hyracks-control-cc-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.hyracks.control.cc.work.AbstractHeartbeatWork.doRun(AbstractHeartbeatWork.java:47)
>  ~[hyracks-control-cc-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.hyracks.control.common.work.SynchronizableWork.run(SynchronizableWork.java:39)
>  [hyracks-control-common-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2117) Eliminate '#' for line comment in SQLPP

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190171#comment-16190171
 ] 

ASF subversion and git services commented on ASTERIXDB-2117:


Commit 9f04f97467b8b59059c4a553be8ff40cccdaa0e4 in asterixdb's branch 
refs/heads/master from [~tillw]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=9f04f97 ]

[ASTERIXDB-2117][SQL] Eliminating '#' for line comment in SQLPP

- user model changes: yes
- storage format changes: no
- interface changes: no

Details:
1. Removed '#' for line comment in SQLPP.
2. Fixed comment stripping for '--' in test executor.

Change-Id: I8fc25ad9cd4923edf0359cf90b466c8fa09a6485
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2037
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> Eliminate '#' for line comment in SQLPP
> ---
>
> Key: ASTERIXDB-2117
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2117
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>Assignee: Xikui Wang
>
> To stick with the standard, '#' should be removed as the line comment 
> indicator, so we can use this in UDF for SQLPP.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2103) Too many disk components for InvertedIndex with CorrelatedMergePolicy

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198853#comment-16198853
 ] 

ASF subversion and git services commented on ASTERIXDB-2103:


Commit 21ed0f72681a20ccb6a654f9aa4d54b8d0ea9c5c in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=21ed0f7 ]

[ASTERIXDB-2103][STO] Too many disk components for CorrelatedPolicy

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
Currently CorrelatedMergePolicy uses component Ids to ensure disk
components of primary and secondary indexes are merged together,
but without synchronization. However, this results in too many disk
components for secondary InvertedIndex. The reason is that secondary
index could miss some round of merges, if the merge policy finds out
the corresponding secondary components are not available (either being
merged or being flushed). Even though flow-control on secondary indexes
can guarantee the secondary index would catch up the next time, it is
still possible that the primary component is finialized, which leaves
the secondary components which miss this round of merge are never merged
again.

This patch fixes this bug by:
- Add the mechanism of depending operations to LSM IO operation. An
operation finishes only after all depending operations have finished.
- For correlated merge policy, the flush/merge of the primary index depends
on all flushes/merges of secondary indexes. This ensures when the
correlated policy schedules merge, all related components of all indexes
are available to merge.

Change-Id: Ib6c06ee23f3bfd16b758802388389c00e29780b1
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2018
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Jianfeng Jia 


> Too many disk components for InvertedIndex with CorrelatedMergePolicy
> -
>
> Key: ASTERIXDB-2103
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2103
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> As reported by [~javierjia], when using CorrelatedPrefixMergePolicy, the 
> secondary InvertedIndex could have too many disk components (the primary 
> index has ~ 200 disk components, while the InvertedIndex has ~500 disk 
> components).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2131) An Aborted Metadata Txn Leads to Invalid Active Ops

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204810#comment-16204810
 ] 

ASF subversion and git services commented on ASTERIXDB-2131:


Commit e2cf9c2715b74d79cb2185ec862bd12d2bfbb7a5 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=e2cf9c2 ]

[ASTERIXDB-2131][TX] Do Not Reset Active Ops For Aborted Metadata Txn

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not reset the primary index operation tracker active operations
  count if the metadata transaction was aborted.
- Add test cases.

Change-Id: Iee47aca1be0675b704ed9f176d9e10daef1cfc7f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2071
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Ian Maxon 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> An Aborted Metadata Txn Leads to Invalid Active Ops
> ---
>
> Key: ASTERIXDB-2131
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2131
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: TX - Transactions
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>Priority: Critical
>
> When a metadata transaction is aborted, the number of active operations on 
> the primary index are decremented twice, one time when the job abort log is 
> written and the other when the transaction completes its abortion. This will 
> result in the following exception when the index is accessed the next time:
> {code:java}
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: The number 
> of active operations cannot be negative!
>   at 
> org.apache.asterix.common.context.PrimaryIndexOperationTracker.completeOperation(PrimaryIndexOperationTracker.java:85)
>  ~[asterix-common-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.metadata.MetadataNode.deleteTupleFromIndex(MetadataNode.java:835)
>  ~[asterix-metadata-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.metadata.MetadataNode.dropNodegroup(MetadataNode.java:734) 
> ~[asterix-metadata-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2051) Variable not found in a complex group-by query

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159432#comment-16159432
 ] 

ASF subversion and git services commented on ASTERIXDB-2051:


Commit 342bd342eb08099ae0fb76df56697924fba8fffd in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=342bd34 ]

[ASTERIXDB-2051][COMP] Fix PushSubplanIntoGroupByRule for complex cases.

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- re-implement PushSubplanIntoGroupByRule and let it handle general cases;
- add an option to LogicalOperatorDeepCopyWithNewVariablesVisitor for
  not re-mapping free variables in the given plan subtree.

Change-Id: I969c40112be0506981357a9c41bf9675ae12ffb9
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1992
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Dmitry Lychagin 


> Variable not found in a complex group-by query
> --
>
> Key: ASTERIXDB-2051
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2051
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: COMP - Compiler
>Reporter: Yingyi Bu
>Assignee: Yingyi Bu
>
> {noformat}
> DROP  DATAVERSE tpch IF EXISTS;
> CREATE  dataverse tpch;
> USE tpch;
> CREATE TYPE LineItemType AS CLOSED {
>   l_orderkey : integer,
>   l_partkey : integer,
>   l_suppkey : integer,
>   l_linenumber : integer,
>   l_quantity : double,
>   l_extendedprice : double,
>   l_discount : double,
>   l_tax : double,
>   l_returnflag : string,
>   l_linestatus : string,
>   l_shipdate : string,
>   l_commitdate : string,
>   l_receiptdate : string,
>   l_shipinstruct : string,
>   l_shipmode : string,
>   l_comment : string
> }
> CREATE DATASET LineItem(LineItemType) PRIMARY KEY l_orderkey,l_linenumber;
> SELECT l_returnflag AS l_returnflag,
>l_linestatus AS l_linestatus,
>coll_count(cheap) AS count_cheaps,
>coll_count(expensive) AS count_expensives
> FROM LineItem AS l
> /* +hash */
> GROUP BY l.l_returnflag AS l_returnflag,l.l_linestatus AS l_linestatus
> GROUP AS g
> LET cheap = (
>   SELECT ELEMENT g.l
>   FROM g
>   WHERE g.l.l_discount > 0.05
> ),
> expensive = (
>   SELECT ELEMENT m
>   FROM (FROM g SELECT VALUE l) AS m
>   WHERE m.l_discount <= 0.05
> )
> ORDER BY l_returnflag,l_linestatus
> ;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1977) array-wrapped formatted json result is empty

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145641#comment-16145641
 ] 

ASF subversion and git services commented on ASTERIXDB-1977:


Commit 080993c3d4118274cb2549b18db91b34e418c21f in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=080993c ]

[ASTERIXDB-1977][UI] Disable wrap into array option for certain cases

1. When formatted JSON is checked, we disable the wrap in array option for
better display. Also, this option is not applicable when output format
is CSV.
2. Fixed the CSV with header format.

Change-Id: I8168e021c1f97e7e3c94fc073447656b481a4380
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1950
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: abdullah alamoudi 


> array-wrapped formatted json result is empty
> 
>
> Key: ASTERIXDB-1977
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1977
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: UI - Web Interface
>Reporter: Jochen Taeschner
>Priority: Minor
>
> Any query run with both output format "JSON (formatted)" and option "Wrap 
> results in outer array" will return an empty result and no success-message
> (additionally, wrap results in outer array doesn't make any sense with csv 
> options and should probably be ignored)
> expected results are probably obvious.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2070) Start feed without connection causes IndexOutOfBoundary exception

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145638#comment-16145638
 ] 

ASF subversion and git services commented on ASTERIXDB-2070:


Commit 80d20195f57b2eab4a7ead81e10e985b0852cdb3 in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=80d2019 ]

[ASTERIXDB-2070][ING] Prevent start feed without connection

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
1. Add connection size check for start feed statement.
2. Remove useless/unassigned variable in RSS feed.

Change-Id: Ic6715b3983ee8a0bb042ef5f34f30381c99466da
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1963
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Start feed without connection causes IndexOutOfBoundary exception
> -
>
> Key: ASTERIXDB-2070
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2070
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>Assignee: Xikui Wang
>
> Currently, the connection checking is missing before we start the feed. This 
> will cause the construct connection job fail.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2050) Enforce a Semicolon After Each SQL++ Statement

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160496#comment-16160496
 ] 

ASF subversion and git services commented on ASTERIXDB-2050:


Commit 8c6e7cdb66f21cd34901f45ac8fb5768d82964b4 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=8c6e7cd ]

[ASTERIXDB-2050][SQL] Update Docs to Reflect New SQL++ Model

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Update docs to reflect the new enforcement of semicolon
  after every statement.
- Add missing semicolons to SET statements in examples.

Change-Id: I31ed7413a6de028fa9d1a0a9d2c6b36ac39ff9c9
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1996
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Enforce a Semicolon After Each SQL++ Statement
> --
>
> Key: ASTERIXDB-2050
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2050
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: SQL - Translator SQL++
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> Currently, only queries must be followed by a semicolon in SQL++ and it's 
> optional to follow DDLs/DMLs with one. While this gives some flexibly to the 
> user, it could lead to many unintentional errors and ambiguity.
> For example, when this is executed:
> {code:java}
> create dataverse x + 1 + 1;
> {code}
> The dataverse 'x' will be created and the following will be returned:
> {code:java}
> 2
> {code}
> The proposed solution to eliminate this ambiguity is to enforce a semicolon 
> after every SQL++ statement.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2087) Failure in HttpServerTest.testChattyServer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16160584#comment-16160584
 ] 

ASF subversion and git services commented on ASTERIXDB-2087:


Commit a281eeaca8ce20ddc1ee00d1a6d07ecf84304b8a in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=a281eea ]

[ASTERIXDB-2087][TEST] Reduce Test Server Budget

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- The test server had budget that is larger than the configured
  max direct memory in the jvm. It was reduced as this causes
  out of memory in one of the test cases.

Change-Id: I451316f2a440d5dc6d4c2a64965a5bdc7f4ad7fc
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2000
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Murtadha Hubail 
Integration-Tests: Jenkins 


> Failure in HttpServerTest.testChattyServer
> --
>
> Key: ASTERIXDB-2087
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2087
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: HYR - Hyracks, TEST - Test Framework
>Reporter: Till
>Assignee: Abdullah Alamoudi
>
> Seen here:
> https://asterix-jenkins.ics.uci.edu/job/hyracks-solo/1670/org.apache.hyracks$hyracks-http/testReport/junit/org.apache.hyracks.http.test/HttpServerTest/testChattyServer/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2069) The MetadataNode failed to bind before the configured timeout in NCServiceExecutionIT

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155841#comment-16155841
 ] 

ASF subversion and git services commented on ASTERIXDB-2069:


Commit a5b37d6785ae8a0bccb2d933a5fa02dd77acc1dc in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=a5b37d6 ]

[ASTERIXDB-2069][TEST] Stop NCServiceExecutionIT on UNUSABLE Cluster

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Stop executing the rest of NCServiceExecutionIT tests if waiting
  for the cluster to become ACTIVE fails.

Change-Id: I8fd470d224aeedbf9c7b0da2bc76f3fa8f0a962e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1990
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Michael Blow 


> The MetadataNode failed to bind before the configured timeout in 
> NCServiceExecutionIT
> -
>
> Key: ASTERIXDB-2069
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2069
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: *DB - AsterixDB
>Reporter: Abdullah Alamoudi
>Assignee: Murtadha Hubail
> Attachments: consoleText.txt
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2086) Add LinearProbeHashTable for InMemoryHashJoin

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162013#comment-16162013
 ] 

ASF subversion and git services commented on ASTERIXDB-2086:


Commit c281d15deae4edae5f1b8fb4e1ad79e796deb399 in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=c281d15 ]

[ASTERIXDB-2086][RT] Remove table size from InMemoryHashJoin interface

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
As this info is available in HashTable already, table size should not
be part of the interface.

Change-Id: I02a677ecfef80ccd4332447a6dcb5d480be6fe80
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1994
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Yingyi Bu 
Integration-Tests: Jenkins 


> Add LinearProbeHashTable for InMemoryHashJoin
> -
>
> Key: ASTERIXDB-2086
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2086
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>
> The current chaining hashtable causes extra overhead in seeking for memory 
> location & cache misses. Change it to LinearProbeHashTable may bring better 
> performances.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2098) Add all AQL udf test cases to SQLPP

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184335#comment-16184335
 ] 

ASF subversion and git services commented on ASTERIXDB-2098:


Commit f418df3c76e482bb6b8cf350dc4f841a3c063a65 in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=f418df3 ]

[ASTERIXDB-2098][TEST] Adding UDF execution tests for SQLPP

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
1. Added test cases for UDFs to SQLPP.
2. Added SqlppExecutionIT to test UDFs in SQLPP.

Change-Id: I3dae4300589f1b16ea5abbf13d81b686a2283add
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2009
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> Add all AQL udf test cases to SQLPP
> ---
>
> Key: ASTERIXDB-2098
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2098
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>
> As part of the AQL deprecation work, it's better to have all tests migrated 
> to SQLPP.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1984) Index-Nested-Loop Join should not care about the contents of the probe branch

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184819#comment-16184819
 ] 

ASF subversion and git services commented on ASTERIXDB-1984:


Commit ebde95d63ca626eef2c957e4c025b8f9aa936687 in asterixdb's branch 
refs/heads/master from [~wangsaeu]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=ebde95d ]

[ASTERIXDB-1984][COMP] probe-subtree init not required

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Let the IntroduceJoinAccessMethod accept arbitrary
  forms of sub-tree for the probe-tree.

Change-Id: Ib353c85bf627d8dd65dba0ea307dee428edb4a26
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2030
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 


> Index-Nested-Loop Join should not care about the contents of the probe branch
> -
>
> Key: ASTERIXDB-1984
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1984
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>
> Currently, when the optimizer tries to transform a query with the "index_nl" 
> hint, it tries to identify datasource, assign, and unnest operators in both 
> outer (probe) and inner branch. It also tries to identify the fieldname of 
> the variables that are being joined. However, the probe branch can be an 
> arbitrary sub-plan and what only really matters for the probe subtree is the 
> type of the field from the probe tree that is being joined. If that field 
> type is correctly identified, then any form of probe-subtree with a simple 
> data-scan on the inner branch can be correctly transformed into an 
> index-utilization plan. The following queries should be transformed into an 
> index-utilization plan. However, they are not transformed now in the current 
> master.
> E.g.,
> {code}
> SELECT * FROM
> [1, 2, 3] AS bar JOIN foo on bar /*+ indexnl */ =  foo.key;
> SELECT  * FROM
> bar JOIN foo on bar.id = foo.key JOIN datac ON foo.key /*+ indexnl */ = 
> datac.val;
> SELECT  * FROM
> (SELECT id, COUNT(*) FROM bar GROUP BY id) AS barr JOIN foo ON barr.id /*+ 
> indexnl */ =  foo.key;
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2104) Avoid always merging old components in correlated policy

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184837#comment-16184837
 ] 

ASF subversion and git services commented on ASTERIXDB-2104:


Commit 56d972fa6dd89769d6ba1220d49e2d54d48657d3 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=56d972f ]

[ASTERIXDB-2104][STO] Optimization for Correlated Policy

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Previously, we introduced an optimization to the prefix merge policy
by ensuring component size grows exponentially after merging. This patch
applies the same optimization to CorrelatedMergePolicy
- A little refactoring of PrefixMergePolicy and CorrelatedMergePolicy
for code reuse

Change-Id: Icd84c77f1e2c34c410508fbc9de70156224ce932
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2019
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Avoid always merging old components in correlated policy
> 
>
> Key: ASTERIXDB-2104
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2104
> Project: Apache AsterixDB
>  Issue Type: Improvement
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> Previously, we introduced an optimization to fix the write amplification 
> problem of the prefix merge policy by ensuring the result disk component size 
> grows exponentially (https://asterix-gerrit.ics.uci.edu/#/c/1818/). A similar 
> optimization needs to be adopted for CorrelatedPrefixPolicy as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2118) Older Memory Component gets flushed to disk first

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186850#comment-16186850
 ] 

ASF subversion and git services commented on ASTERIXDB-2118:


Commit ab8375ea3edec759d5590f5792926211ff22ca1b in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=ab8375e ]

[ASTERIXDB-2118][STO] Ensure flush ordering of memory components

- user model changes: no
- storage format changes: no
- interface changes: no

- Fix the bug of AsynchronousScheduler by waking up the next
flush only when the current operation is a FLUSH operation

Change-Id: I7de4a1625fdd3faaa07f65be2ebc714ec7564b29
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2038
Reviewed-by: Ian Maxon 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 


> Older Memory Component gets flushed to disk first
> -
>
> Key: ASTERIXDB-2118
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2118
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> Currently we have two memory components, say M1 and M2, as double buffering. 
> The AysnchronousScheduler needs to guarantee M1 and M2 are flushed in a 
> proper order. However, the current scheduler is buggy. Consider the following 
> event sequences.
> 1. M1 is full
> 2. schedule flush M1
> 3. activate M2
> 4. M2 is full
> 5. schedule flush M2 (cannot be scheduled since M1 is flushing)
> 6. A merge operation is finished
> At this moment, line 59 of AysnchronousScheduler is executed, and it 
> schedules M2.
> Thus, M2 could be finished earlier before M1, resulting in wrong order of 
> flushed disk components on disk.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2108) Processed Objects Metric to Query Service

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16187195#comment-16187195
 ] 

ASF subversion and git services commented on ASTERIXDB-2108:


Commit 8dd947bab9e8d1efb4cd4e85e017ad56b5e24693 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=8dd947b ]

[ASTERIXDB-2108][API][RT] Add Processed Objects Metric

- user model changes: no
- storage format changes: no
- interface changes: yes
  Introduced IOperatorStats and IStatsCollector APIs to collect
  runtime stats.

Details:
- Introduce OperatorStats API to report operators runtime stats.
- Introduce StatsCollector API to report task runtime stats.
- Implement OperatorStats for IndexSearchOperatorNodePushable
  (tuple counter only).
- Add "processedObjects" metric to QueryService API.
- Add Stats to ExecuteStatementResponseMessage to pass stats
  from CC to NCQueryService.
- Add metrics test cases.

Change-Id: Ie4afe6a676ef0b8a31d36d7dafc13a4023ebf177
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2032
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Processed Objects Metric to Query Service
> -
>
> Key: ASTERIXDB-2108
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2108
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: API - HTTP API, RT - Runtime
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> It would be nice to have a metric that shows the number of processed objects 
> (tuples) to answer a query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2119) Project Rule has non-deterministic ordering of variables

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188514#comment-16188514
 ] 

ASF subversion and git services commented on ASTERIXDB-2119:


Commit ce348d68475a88c9bbf364abc284b64ac7f17e34 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=ce348d6 ]

[ASTERIXDB-2119][COMP] Fix variable ordering of projects

- user model changes: no
- storage format changes: no
- interface changes: no

Details:

The current IntroduceProjectsRule implementation uses HashSet to
calculate projected variables, which makes the ordering of output
variables unpreditable. This patch fixes this undesired behavior by
using LinkedHashSet to ensure the project variables have the same
ordering from the original variables.

Change-Id: Id96a5fe048dd11b7f2e97f4d4a802736ba5ba003
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2043
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Taewoo Kim 
Integration-Tests: Jenkins 


> Project Rule has non-deterministic ordering of variables
> 
>
> Key: ASTERIXDB-2119
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2119
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: COMP - Compiler
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> IntroduceProjectsRule introduces projections whenever needed to eliminate 
> unnecessary variables. However, the current implementation depends on 
> HashSet, which makes the ordering of output variables indeterministic. This 
> is undesirable for tests, and some operators which may depend on the ordering 
> of variables.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2114) New FeedPolicy is incorrectly persisted in Metadata

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186670#comment-16186670
 ] 

ASF subversion and git services commented on ASTERIXDB-2114:


Commit 8b3e67af4a7c761f8087e41baa66c0b33a38378e in asterixdb's branch 
refs/heads/master from [~iabsalyamov]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=8b3e67a ]

[ASTERIXDB-2114][MTD] Fixed incorrectly persisted FeedPolicy

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- New FeedPolicy was incorrectly persisting PolicyName & Description fields

Change-Id: I04beb24c861b116525a9da467fafc1742f4a73d4
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2035
Contrib: Jenkins 
Reviewed-by: Steven Jacobs 
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 


> New FeedPolicy is incorrectly persisted in Metadata
> ---
>
> Key: ASTERIXDB-2114
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2114
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: MTD - Metadata
>Reporter: Ildar Absalyamov
>Assignee: Ildar Absalyamov
>
> A newly added FeedPolicy is incorrectly persisted in Metadata:
> drop dataverse experiments if exists;
> create dataverse experiments;
> use dataverse experiments;
> create ingestion policy testPolicy from path 
> "data/feed-policy/policy.properties"
> definition "someString";
> { "DataverseName": "experiments", "PolicyName": "*someString*", 
> "Description": "*experiments*", "Properties": {{ { "Name": "name", "Value": 
> "testname" }, { "Name": "value", "Value": "testvalue" }, { "Name": "key", 
> "Value": "testkey" } }} }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2096) IntroduceDynamicTypeCastForExternalFunctionRule is not working properly for function call in nested record

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186610#comment-16186610
 ] 

ASF subversion and git services commented on ASTERIXDB-2096:


Commit 0535526938220d0cfab90d5a8921e64694f721cf in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=0535526 ]

[ASTERIXDB-2096][COMP] Fix type casting for ExternalFunction

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
1. The current IntroduceDynamicTypeCastForExternalFunctionRule
cannot handle external function calls in nested record constructor.
This patch fix this issue by visiting all nested parameters for
external functions.
2. The ResultExtractor should be able to handle multiple queries
in single statement file as AQL does.

Change-Id: I65c298def75b18fab01f513012e28fc44fdc2fd4
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2010
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 


> IntroduceDynamicTypeCastForExternalFunctionRule is not working properly for 
> function call in nested record
> --
>
> Key: ASTERIXDB-2096
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2096
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>
> The current type casting rule for ExternalFunction cannot handle when we put 
> function call into a record. The reason is this only checks AssignOp with 
> external function call on its right side. In reality, the function call can 
> be put in an open record constructor which is a builtin function. Also, 
> similar casting rule should be applied to all parameter of the external 
> function in case of recursive calling.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2119) Project Rule has non-deterministic ordering of variables

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190005#comment-16190005
 ] 

ASF subversion and git services commented on ASTERIXDB-2119:


Commit 7926f25955871dd0d78d3a94852a4f3050c18a16 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=7926f25 ]

[ASTERIXDB-2119][COMP] Fix variable ordering of project pushdown

- user model changes: no
- storage format changes: no
- interface changes: no

Details:

Fix variable ordering of PushProjectDownRule by using LinkedHashSet.
Related to previous patch https://asterix-gerrit.ics.uci.edu/#/c/2043/

Change-Id: I21d0a2764e22482a9361b709e566d78acd33cf4d
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2048
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Jianfeng Jia 


> Project Rule has non-deterministic ordering of variables
> 
>
> Key: ASTERIXDB-2119
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2119
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: COMP - Compiler
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> IntroduceProjectsRule introduces projections whenever needed to eliminate 
> unnecessary variables. However, the current implementation depends on 
> HashSet, which makes the ordering of output variables indeterministic. This 
> is undesirable for tests, and some operators which may depend on the ordering 
> of variables.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2105) Support broadcast join hint with multiple predicates

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179562#comment-16179562
 ] 

ASF subversion and git services commented on ASTERIXDB-2105:


Commit 88b154dfd66d1641d0fe25fe4e0ff9e54c24dcad in asterixdb's branch 
refs/heads/master from [~dlychagin-cb]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=88b154d ]

[ASTERIXDB-2105][COMP] Support broadcast join hint with multiple predicates

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Broadcast hint now works if join condition has multiple predicates.
  Only one predicate needs to be annotated.

Change-Id: Id1475c5e574be2632c92ae619f3a36c56c222514
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2024
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Xikui Wang 
Reviewed-by: Till Westmann 


> Support broadcast join hint with multiple predicates
> 
>
> Key: ASTERIXDB-2105
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2105
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: COMP - Compiler
>Reporter: Dmitry Lychagin
>Assignee: Dmitry Lychagin
>Priority: Minor
>
> Currently broadcast join hint does not work if the join condition has 
> multiple predicates. We need to fix that. Annotating only one predicate 
> should be sufficient for the broadcast to be picked by the optimizer.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2107) Dynamic Topology Cluster May Become UNUSABLE on Multiple Nodes Addition

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181952#comment-16181952
 ] 

ASF subversion and git services commented on ASTERIXDB-2107:


Commit 2c13bdcc8eb5f2bc228aeafe8e4c4ce1481c68a0 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=2c13bdc ]

[ASTERIXDB-2107][CLUS] Prevent Invalid UNUSABLE State in Dynamic Topology

- user model changes: no
- storage format changes: no
- interface changes: yes
  Renamed IClusterStateManager add/Remove NCConfig methods
  to notifyNode join/failure.

Details:
- Mark node as participant when it completes its startup and
  not when it joins the cluster.
- Allow partitions to be added with pending activation state.
- Remove the use of static MetadataProperties for reporting number of nodes.
- Add test cases.

Change-Id: I7a0db2d66cf44650dcc673b3f2de537816cb84c7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2029
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> Dynamic Topology Cluster May Become UNUSABLE on Multiple Nodes Addition
> ---
>
> Key: ASTERIXDB-2107
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2107
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> A cluster with a dynamic topology will be unusable for a period when two 
> nodes are added to the cluster. When one of two nodes reports startup 
> completion, the cluster state will be updated and the inactive partitions of 
> the second node will be detected which will result in unusable state.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2057) 500 error from Query API needs to have more detailed information

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138827#comment-16138827
 ] 

ASF subversion and git services commented on ASTERIXDB-2057:


Commit 78b1a694bb6d84a8298a1cb4986fa322f6ccc0ae in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=78b1a69 ]

[ASTERIXDB-2057][API] Add detailed error message for 500 in REST API

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
1. Add error message to 500 response.
2. Specify proper content type for Ansible query execution script, so
LIKE % can be parsed properly.
3. Add semicolons to statements in create.sqlpp.

Change-Id: I17759141116a1baf878abf7d5ec70295a18946e8
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1959
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Yingyi Bu 


> 500 error from Query API needs to have more detailed information
> 
>
> Key: ASTERIXDB-2057
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2057
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>
> The current Ansible execute query script in TPCH benchmark sends query with 
> default content-type which is application/x-www-form-urlencoded. This will 
> cause the parse issue in queries contain LIKE %. This needs to be fixed by 
> change content type to plain text, and add more detailed info for the 500 
> error.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2032) Let stop-sample-cluster script not use the shutdown REST API

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140059#comment-16140059
 ] 

ASF subversion and git services commented on ASTERIXDB-2032:


Commit d828754b334534423e87dc13edf01fd2446b2afc in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=d828754 ]

[ASTERIXDB-2032] Restore use of REST shutdown API

This reverts commit 122a73693e4a6a2484936ed86967e843d4dfa4e1.

Change-Id: Ie73cc5dab3b9ad06120ca634f5aee9dfeef6d07b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1958
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 


> Let stop-sample-cluster script not use the shutdown REST API
> 
>
> Key: ASTERIXDB-2032
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2032
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management
>Reporter: Yingyi Bu
>Assignee: Till
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2058) Job Cancellation returns before the job is cleaned up

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140429#comment-16140429
 ] 

ASF subversion and git services commented on ASTERIXDB-2058:


Commit 87411c22cc30fb425a4839b20b3a259f28d1ef79 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=87411c2 ]

[ASTERIXDB-2058][HYR] Only Complete job cancellation after cleanup

- user model changes: no
- storage format changes: no
- interface changes: yes
  --IJobManager.cancel now takes a callback

details:
- Job cancellation now completes only after the job cleanup work
  has completed and not merely when the abort tasks are executed.
- The NCQueryServiceServlet actively cancels requests that passes
  5 minutes.
- Cancellation of timedout jobs is not done through the Http API
  but through message broker.
- Typically, requests might timeout when the servers are
  overloaded. When that is the case, there is a high chance http
  requests are to be rejected including requests to cancel
  previously submitted queries. This is the reason for using
  Message broker for this task.
- ExecuteStatementRequest used to execute the statement in
  a different executor thread even though it is itself is being
  executed in an executor thread and is not blocking anyone.
  This was fixed as well.

Change-Id: I14b4bbd512cc88e489254d8bf82edba0fd3a3db5
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1961
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> Job Cancellation returns before the job is cleaned up
> -
>
> Key: ASTERIXDB-2058
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2058
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: HYR - Hyracks
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2064) Stop Feed Adapter can hang forever

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140642#comment-16140642
 ] 

ASF subversion and git services commented on ASTERIXDB-2064:


Commit 979012d5e5c2dacde53ac6144c35a155293a8d4e in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=979012d ]

[ASTERIXDB-2064][ING] Timeout Stop Feed

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- The abort feed message stops the reader and wait for the
  dataflow controller to signal end of life.
- If the reader returns true to stop but the dataflow controller
  never signal ends, it can get stuck.
- This change adds a timeout after which, the task thread is
  interrupted.

Change-Id: If609a8343767ee7a80689a58af35a1b3fca2964b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1964
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Michael Blow 
Integration-Tests: Jenkins 


> Stop Feed Adapter can hang forever
> --
>
> Key: ASTERIXDB-2064
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2064
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: ING - Ingestion
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2052) Http server memory leak

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136088#comment-16136088
 ] 

ASF subversion and git services commented on ASTERIXDB-2052:


Commit 7def53f4e75abbb50012823ce6cb44fad911e117 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=7def53f ]

[ASTERIXDB-2052][OTH] Release resources on http request rejection

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- When a request is rejected, we release its resources.
- A test case was added which sends 3500+ rejected requests and
  causes the server to throw out of memory error prior to this
  fix.

Change-Id: Ia0e3f3e6e2f94a31f296b3491a07f624a4fea604
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1955
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Http server memory leak
> ---
>
> Key: ASTERIXDB-2052
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2052
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: OTH - Other
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Sometimes, the http server is stuck with out of memory error



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2055) Double shutdown calls!! java.lang.Exception: Double shutdown calls on graceful shutdown

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136271#comment-16136271
 ] 

ASF subversion and git services commented on ASTERIXDB-2055:


Commit de892d7c075c6341ccfbb7a4d0090487d82dd5b6 in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=de892d7 ]

[ASTERIXDB-2055][HYR][CLUS] Avoid double shutdown

Change-Id: If50651a4c46178f191966e09d365d2015df295bc
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1957
Reviewed-by: Michael Blow 
Tested-by: Michael Blow 


> Double shutdown calls!! java.lang.Exception: Double shutdown calls on 
> graceful shutdown
> ---
>
> Key: ASTERIXDB-2055
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2055
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: CLUS - Cluster management, HYR - Hyracks
>Reporter: Michael Blow
>Assignee: Michael Blow
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1812) OutofMemoryError when group by on a non-existing field with 300k records (tweets)

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136272#comment-16136272
 ] 

ASF subversion and git services commented on ASTERIXDB-1812:


Commit 6ea45c88abc7fab74d9f322b15d95c2d1420a10d in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=6ea45c8 ]

[ASTERIXDB-1812][RT] Budget the memory usage for pre-clustered group-by.

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- let pre-clustered group-by consider memory budget.

Change-Id: I670269b0b8f446d06d8dd73202194574aa524e85
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1940
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Till Westmann 
Integration-Tests: Jenkins 


> OutofMemoryError when group by on a non-existing field with 300k records 
> (tweets)
> -
>
> Key: ASTERIXDB-1812
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1812
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: *DB - AsterixDB, HYR - Hyracks
> Environment: Linux 16.04
> Asterix 0.9.0 with 2 nc nodes and 1 cc node. (all using default 
> configurations from 
> https://asterixdb.apache.org/docs/0.9.0/install.html#Section1SingleMachineAsterixDBInstallation)
>Reporter: Chen Luo
>
> The dataset is a sample tweet dataset provided by Cloudberry, which contains 
> 324000 tweets (about 300M). When issuing the following query, I always get an 
> OutofMemoryError.
> Query:
> {code}
> select * from twitter.ds_tweet t
> group by t.test;
> {code}
> Stacktrace:
> {code}
> org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
> HYR0003: java.lang.OutOfMemoryError: Java heap space
>   at 
> org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
>   at 
> org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: 
> java.lang.OutOfMemoryError: Java heap space
>   at 
> org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
>   at org.apache.hyracks.control.nc.Task.run(Task.java:330)
>   ... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: 
> java.lang.OutOfMemoryError: Java heap space
>   at 
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
>   at 
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
>   at org.apache.hyracks.control.nc.Task.run(Task.java:273)
>   ... 3 more
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.OutOfMemoryError: Java heap space
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>   at 
> org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
>   ... 5 more
> Caused by: java.lang.OutOfMemoryError: Java heap space
>   at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
>   at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
>   at 
> org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
>   at 
> org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
>   at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
>   at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
>   at 
> org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
>   at 
> org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
>   at 
> org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
>   at 
> org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
>   at 
> org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
>   at 
> 

[jira] [Commented] (ASTERIXDB-2072) Checkpoints incorrectly deleted on ClosedByInterruptException

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142886#comment-16142886
 ] 

ASF subversion and git services commented on ASTERIXDB-2072:


Commit 00ce8748cd68ba0ffa3dd74b5c1b976d054a19ca in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=00ce874 ]

[ASTERIXDB-2072][STO] Do Not Delete Checkpoints on ClosedByInterruptException

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- On ClosedByInterruptException, don't delete checkpoints since
  they are not corrupted.

Change-Id: I7581eb15558dd656d9e2de1469845371dcc13e4b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1975
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 
Tested-by: Jenkins 


> Checkpoints incorrectly deleted on ClosedByInterruptException
> -
>
> Key: ASTERIXDB-2072
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2072
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> On ClosedByInterruptException, the checkpoint file shouldn't be deleted since 
> it is not corrupted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2047) Strange/wrong behavior in console with Formatted JSON and multiple statements

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139361#comment-16139361
 ] 

ASF subversion and git services commented on ASTERIXDB-2047:


Commit 52528555e1f18a41a9ea55b8cf19d1f382be795d in asterixdb's branch 
refs/heads/master from [~idleft]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=5252855 ]

[ASTERIXDB-2047][UI] Escape special entities in HTML result delivery

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Escape HTML special entities to make sure we don't have fancy HTML
  style display with user data.

Change-Id: I7aa05fe39b7a1f755574c4f49fd9694239078586
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1949
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Till Westmann 
Integration-Tests: Jenkins 


> Strange/wrong behavior in console with Formatted JSON and multiple statements
> -
>
> Key: ASTERIXDB-2047
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2047
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: *DB - AsterixDB, UI - Web Interface
>Reporter: Michael J. Carey
>Assignee: Xikui Wang
>
> Consider running the following kind of statement sequence two ways - first 
> with unformatted and with formatted JSON.  With larger JSON objects than this 
> unsuccessful repro attempt, sometimes the formatted version doesn't show all 
> of the returned data from the query.  Also, it leaves the browser in a bad 
> way so that it no longer reliably shows all of the results of subsequent 
> queries.  
> DELETE FROM foo;
> SELECT s FROM foo s;
> INSERT INTO foo
> {
>   "key": "xxx123",
>   "class": "CS122a",
>   "content": {
> "DateFrom": "Jan 31, 2012",
> "DateTo": "Apr 09, 2013"
>   }
> };
> SELECT s FROM foo s;
> INSERT INTO foo
>   {
> "rec": {
>   "binary": false,
>   "class": "CS122b",
>   "content": {
> "backOrder": false,
> "frontOrder": true,
> "width": 100
>   },
>   "multiplicationKey": "12"
> }
>   };
> SELECT s FROM foo s;



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2060) Intermittent hangs in ActiveEventsListenerTest

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138835#comment-16138835
 ] 

ASF subversion and git services commented on ASTERIXDB-2060:


Commit 27a75d70883cc753bab39beea745e30e5143dfbb in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=27a75d7 ]

[ASTERIXDB-2060][TEST] Stop activity in tests with active recovery

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Hangs were caused by some test cases leaving a recovery task
  running on the listener. Because each test case has its own
  listener but the test cases share the MetadataLockManager
  a hang occurs.

Change-Id: I8dd31a7b9a82678e682d04d17760aa4b10eb80ce
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1962
Sonar-Qube: Jenkins 
Reviewed-by: Michael Blow 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 


> Intermittent hangs in ActiveEventsListenerTest
> --
>
> Key: ASTERIXDB-2060
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2060
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: ING - Ingestion
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Sometimes, ActiveEventsListenerTest hangs causing asterix-app tests to abort.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2067) Failures during controller flush can be mistaken for failures while reading external data source

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141929#comment-16141929
 ] 

ASF subversion and git services commented on ASTERIXDB-2067:


Commit 4f79620cd71bd750548a749db352c418fbadddae in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=4f79620 ]

[ASTERIXDB-2067][ING] Handle Failures in Controller Flush

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- Failures that happen in feed while reading from external
  sources allows ingestion pipeline to close gracefully
  pushing parsed records in the frame forward before
  failing.
- There was an assumption that when hasNext() or next()
  are being called on a data reader and we fail, then
  the failure didn't affect the integrity of the pipeline.
- This assumption is incorrect as hasNext() and next()
  can themselves flush the pipeline and if the failure
  happened during the flush call, the pipeline must be
  failed.

Change-Id: Ib9be729088bd94338ef235eaea34ba3da99f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1968
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> Failures during controller flush can be mistaken for failures while reading 
> external data source
> 
>
> Key: ASTERIXDB-2067
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2067
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: ING - Ingestion
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2049) Start Feed hangs indefinitely

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135676#comment-16135676
 ] 

ASF subversion and git services commented on ASTERIXDB-2049:


Commit 62914c636ea5f922565b5340c669b4ebb8fc7859 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=62914c6 ]

[ASTERIXDB-2049][ING] Fix hang in Start Feed

- user model changes: no
- storage format changes: no
- interface changes: no

details:
- The hang is caused by one runtime finishing and unregistering
  before another runtime registers. When that happens, the number
  of registered runtimes never reaches the total number of runtimes
  and so the start feed statement doesn't complete.
- To avoid the situation described above, we use different counters
  for registration and deregistration. Since deregistration count
  is now kept in another variable, the registrations will either
  reach the expected count or a failure will happen and both
  cases completes the start feed request.

Change-Id: I0019f5634009bf924fb37acc78eb796842eef492
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1953
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Start Feed hangs indefinitely
> -
>
> Key: ASTERIXDB-2049
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2049
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: ING - Ingestion
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2121) Assertion Errors are Suppressed during Hyracks Job Execution

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198193#comment-16198193
 ] 

ASF subversion and git services commented on ASTERIXDB-2121:


Commit 64965900c4fc4a33fe97bc1db6fa9ba946621b31 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=6496590 ]

[ASTERIXDB-2121][HYR] Handle Throwables during job execution

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
Currently, only exceptions are handled and logged during Hyracks job
execution. However, throwables, such as errors, are not handled and
simply ignored saliently. This patch handles all throwabls during job
execution.

Change-Id: Ibbe09d5231fe2bdfa12d834bb1a6adb46b355a48
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2051
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 


> Assertion Errors are Suppressed during Hyracks Job Execution
> 
>
> Key: ASTERIXDB-2121
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2121
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: HYR - Hyracks
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> Currently during the execution of Hyracks jobs, only Exceptions are logged, 
> while other Throwable, such as Assertion Errors, are missed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2122) Invalid JSON is Returned on Exception While Reading Results

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193750#comment-16193750
 ] 

ASF subversion and git services commented on ASTERIXDB-2122:


Commit 9ac01a25fdeb2a8bb5f672a3218cdb6964198bd6 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=9ac01a2 ]

[ASTERIXDB-2122][API] Ensure Valid JSON on Result Printing

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure results JSON field is closed when an exception is
  encountered.
- Always return errors field if it exists in the API response.
- Add test case.

Change-Id: Ic16e9d234c5e127ea24e382c49f3c7cfa5bce9c7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2057
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 


> Invalid JSON is Returned on Exception While Reading Results
> ---
>
> Key: ASTERIXDB-2122
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2122
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: API - HTTP API
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> QueryService and the Rest AP return invalid JSON if any exception happens 
> while reading the result.
> Example:
> {code:java}
> Caused by: com.fasterxml.jackson.core.JsonParseException: 
> Unexpected character (':' (code 58)): was expecting comma to separate Array 
> entries
>  at [Source: {
>   "requestID": "1bebc7b2-a2fa-4fec-a70f-ebdc0f978a7c",
>   "signature": "*",
>   "type": "application/x-adm",
>   "results": ["errors": [
>   "code": 1,
>   "msg": "IPCException: Cannot send on a closed handle"
>   }],
>   "status": "fatal",
>   "metrics": {
>   "elapsedTime": "170.240113ms",
>   "executionTime": "167.497012ms",
>   "resultCount": 0,
>   "resultSize": 0,
>   "processedObjects": 1412,
>   "errorCount": 1
>   }
> }
> {code}
> Stacktrace:
> {code:java}
> org.apache.hyracks.ipc.exceptions.IPCException: Cannot send on a closed handle
> org.apache.hyracks.api.exceptions.HyracksDataException: 
> org.apache.hyracks.ipc.exceptions.IPCException: Cannot send on a closed handle
>   at 
> org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:47)
>  ~[hyracks-api-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.hyracks.client.dataset.HyracksDatasetReader.nextPartition(HyracksDatasetReader.java:130)
>  ~[hyracks-client-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.hyracks.client.dataset.HyracksDatasetReader.read(HyracksDatasetReader.java:141)
>  ~[hyracks-client-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.asterix.app.result.ResultReader.read(ResultReader.java:49) 
> ~[asterix-app-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.app.result.ResultPrinter.print(ResultPrinter.java:207) 
> ~[asterix-app-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.api.http.server.ResultUtil.printResults(ResultUtil.java:81)
>  ~[asterix-app-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.api.http.server.NCQueryServiceServlet.executeStatement(NCQueryServiceServlet.java:125)
>  [asterix-app-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.api.http.server.QueryServiceServlet.handleRequest(QueryServiceServlet.java:411)
>  [asterix-app-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.asterix.api.http.server.QueryServiceServlet.post(QueryServiceServlet.java:97)
>  [asterix-app-0.9.3-SNAPSHOT.jar:0.9.3-SNAPSHOT]
>   at 
> org.apache.hyracks.http.server.AbstractServlet.handle(AbstractServlet.java:90)
>  [hyracks-http-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> com.couchbase.analytics.servlet.AuthenticatedServlet.handle(AuthenticatedServlet.java:67)
>  [cbas-server-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>   at 
> org.apache.hyracks.http.server.HttpRequestHandler.handle(HttpRequestHandler.java:70)
>  [hyracks-http-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.hyracks.http.server.HttpRequestHandler.call(HttpRequestHandler.java:55)
>  [hyracks-http-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at 
> org.apache.hyracks.http.server.HttpRequestHandler.call(HttpRequestHandler.java:36)
>  [hyracks-http-0.3.3-SNAPSHOT.jar:0.3.3-SNAPSHOT]
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
> [?:1.8.0_131]
>   at 
> 

[jira] [Commented] (ASTERIXDB-2103) Too many disk components for InvertedIndex with CorrelatedMergePolicy

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199708#comment-16199708
 ] 

ASF subversion and git services commented on ASTERIXDB-2103:


Commit 1dc8228b7262e47c06487443da0f735321e63afe in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=1dc8228 ]

Revert "[ASTERIXDB-2103][STO] Too many disk components for CorrelatedPolicy"

This reverts commit 21ed0f72681a20ccb6a654f9aa4d54b8d0ea9c5c.

Change-Id: I670545acd09c678f21be25313353ab306be86202
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2063
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: Ian Maxon 
Integration-Tests: Jenkins 


> Too many disk components for InvertedIndex with CorrelatedMergePolicy
> -
>
> Key: ASTERIXDB-2103
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2103
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> As reported by [~javierjia], when using CorrelatedPrefixMergePolicy, the 
> secondary InvertedIndex could have too many disk components (the primary 
> index has ~ 200 disk components, while the InvertedIndex has ~500 disk 
> components).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1076) False failures cause denying new queries

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199688#comment-16199688
 ] 

ASF subversion and git services commented on ASTERIXDB-1076:


Commit 860fcde90bd799e00eaa61cbf81badfea1c25dda in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=860fcde ]

[ASTERIXDB-1076][HYR] Generate heartbeats in their own thread

- Generate & send NC heartbeats in their own thread to prevent starvation
/ scheduling issues
- Fix retries on IPC connections
- Don't spin on heartbeat send failure

Change-Id: Ieae21b1596013a699f27975fb21894244c536395
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2060
Reviewed-by: Murtadha Hubail 
Integration-Tests: Murtadha Hubail 
Tested-by: Murtadha Hubail 


> False failures cause denying new queries
> 
>
> Key: ASTERIXDB-1076
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1076
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: *DB - AsterixDB
>Reporter: Yingyi Bu
>Assignee: Michael Blow
>  Labels: soon
>
> When CPUs in the cluster are saturated for computations,  the heartbeat from 
> slave nodes to the master node might get delayed.  In this case, the master 
> node thinks a node fails, and can no longer adds the node back.  Hence, the 
> entire cluster is not usable and an instance restart is needed.
> Two things need to be fixed:
> 1.  (at least) expose AsterixDB configuration parameters to allow users to 
> set a large heartbeat threshold;
> 2.  allow a node to leave and re-join a hyracks cluster.
> In the long term, we might need to investigate better liveness check 
> strategies.
> To reproduce that issue,  just let slave nodes' CPUs overloaded and you will 
> see that.
> The exception " Asterix Cluster Global recovery is not yet complete and The 
> system is in ACTIVE state" will be thrown for upcoming queries.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2135) NPE in LSMBTreePointSearchCursor on Failure

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209940#comment-16209940
 ] 

ASF subversion and git services commented on ASTERIXDB-2135:


Commit 89e6a93277205a9dbc76c18e249919a745d224d2 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=89e6a93 ]

[ASTERIXDB-2135][IDX] Prevent NPE in LSMBTreePointSearchCursor

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Prevent NPE on LSMBTreePointSearchCursor.close

Change-Id: I062c1200d9c5a1a574a1ccdb32be0ac011406d92
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2081
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 


> NPE in LSMBTreePointSearchCursor on Failure
> ---
>
> Key: ASTERIXDB-2135
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2135
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: IDX - Indexes
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> If a failure happens before LSMBTreePointSearchCursor.hasNext is called to 
> initialize the search cursor, LSMBTreePointSearchCursor.close is called as 
> part of the pipeline failure and leads to NPE.
> Stacktrace:
> {code:java}
> Caused by: java.lang.NullPointerException at 
> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreePointSearchCursor.close(LSMBTreePointSearchCursor.java:218)
>   at 
> org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeSearchCursor.close(LSMBTreeSearchCursor.java:77)
>   at 
> org.apache.asterix.runtime.operators.LSMPrimaryUpsertOperatorNodePushable.close(LSMPrimaryUpsertOperatorNodePushable.java:360)
>   at 
> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.close(SyncFeedRuntimeInputHandler.java:64)
>   at 
> org.apache.asterix.external.operators.FeedMetaStoreNodePushable.close(FeedMetaStoreNodePushable.java:155)
>   at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:409)
>   at org.apache.hyracks.control.nc.Task.run(Task.java:323)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1911) Allow concurrent executions of one pre-distributed job

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252778#comment-16252778
 ] 

ASF subversion and git services commented on ASTERIXDB-1911:


Commit e6f426b8019957dd15962926788edbe655218cb9 in asterixdb's branch 
refs/heads/master from [~sjaco002]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=e6f426b ]

[ASTERIXDB-1911][HYR,RT,CLUS] Fixes and Improvements for Deployed Jobs

Rename Predistributed Jobs to Deployed Jobs
Enable job executions to have a map of job parameters
Add an Asterix function to retrieve these parameters
which are can be assigned when the job is run, e.g. for Deployed jobs
Allow Deployed jobs to have new TxnIds and JobIds for each execution
Allow simultaneous execution of one Deployed Job

Change-Id: I8f493c1fa977d07dfe8a875f9ebe9515d01d1473
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2045
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Xikui Wang 


> Allow concurrent executions of one pre-distributed job
> --
>
> Key: ASTERIXDB-1911
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1911
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Steven Jacobs
>Assignee: Steven Jacobs
>
> Right now, concurrent attempts to run the same pre-distributed will not work. 
> It would be great if we could find a way to allow them to run together.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2166) Bloomfilter is not correctly bulk loaded for buddy btree

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252885#comment-16252885
 ] 

ASF subversion and git services commented on ASTERIXDB-2166:


Commit 8a7894f582d2441394ab574145df581ff5c0daeb in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=8a7894f ]

[ASTERIXDB-2166] Fix bulk load bloom filters for buddy btree

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Fix bulk loading bloom filters for buddy btrees. For these
bloom filters, the deleted keys are added through the delete method,
which shouldn't be ignored by the bloom filter bulk loader

Change-Id: Icc7ca46c69c9102010f4b407ca0e9d96ba19289b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2152
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Bloomfilter is not correctly bulk loaded for buddy btree
> 
>
> Key: ASTERIXDB-2166
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2166
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: IDX - Indexes, STO - Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> For LSM indexes, conceptually we have two kinds of bloom filters, i.e., for 
> the index itself (if it's a primary index), or for the buddy btree to store 
> deleted keys. For the latter, we use delete() method to bulk load the keys, 
> which is however ignored by the BloomFilterBulkLoader class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2161) java.lang.IllegalStateException when set LSM Memory Component Id

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267560#comment-16267560
 ] 

ASF subversion and git services commented on ASTERIXDB-2161:


Commit c5a0a1974d36d647a22d606d53bdaafd85f641df in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=c5a0a19 ]

[ASTERIXDB-2161][TEST] Add indexes to MultiPartitionTest

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- This change adds secondary indexes to multi partition
  LSM indexes tests.
- This enables testing of specific concurrency scenarios
  and ensuring properties stored in primary and secondary
  indexes are consistent.
- In addition, the call for flushDataset in
  DatasetLifecycleManager now throws an
  IllegalStateException if the number of active
  operations is not 0. Some tests used to call this
  function when there are ongoing operations and that
  is expected to never be the case in the actual system.

Change-Id: I5aea71a87f149b01f6c7310867fc15b5a340b93c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2173
Tested-by: Jenkins 
Reviewed-by: Luo Chen 
Integration-Tests: Jenkins 
Contrib: Jenkins 


> java.lang.IllegalStateException when set LSM Memory Component Id
> 
>
> Key: ASTERIXDB-2161
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2161
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> On one NC with multiple partitions, it may through IllegalStateException when 
> we reset the memory component Id. The memory component should receive a 
> larger Id, but for some reason it still receives the old one.
> {code}
> 16:25:40 SEVERE: SEARCH failed to enter components on {"class" : "LSMBTree", 
> "dir" : 
> "/home/jenkins/jenkins/workspace/asterix-gerrit-verify-asterix-app/asterixdb/asterix-app/target/io/dir/asterix_nc1/iodevice1/storage/partition_1/test/LineID_idx_idx_LineID_suppkey",
>  "memory" : 2, "disk" : 1}
> 16:25:40 java.lang.IllegalStateException: LSM memory component receives 
> illegal id. Old id [1510360027993,1510360027993], new id 
> [1510360027993,1510360027993]
> 16:25:40  at 
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMMemoryComponent.resetId(AbstractLSMMemoryComponent.java:276)
> 16:25:40  at 
> org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.recycled(AbstractLSMIOOperationCallback.java:205)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2149) Improve NormalizedKey on Sorting

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267578#comment-16267578
 ] 

ASF subversion and git services commented on ASTERIXDB-2149:


Commit ed469381235990ce5ecd2f242b679190ef2ca263 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=ed46938 ]

[ASTERIXDB-2149] Enable multiple normalized keys in sort

- user model changes: no
- storage format changes: no
- interface changes: yes. The interface of sort is changed.

Currently, during the (in-memory) sort, we use an int normalized keys to
speed up comparisions by avoiding random memory accesses. However, this
technique is inefficient if the first 4 bytes of the sorting keys are
not distinctive. From performance point of view, it's better to use
longer normalized keys when it's possible (2-3x improvements).

This is enabled by this patch by:
- Allowing multiple normalized keys during sort, and the length of each
normalized key can be longer (multiple integers).
- Enable memory budgeting of pointer directories as well during sort
(but for performance, we still use int[], instead of byte[] from frame).

The next patch will enable the AsterixDB layer to use this feature to
speed up sort performance.

Change-Id: I4354242ff731b4b006b8446b58f65873047dde78
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2127
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Improve NormalizedKey on Sorting
> 
>
> Key: ASTERIXDB-2149
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2149
> Project: Apache AsterixDB
>  Issue Type: Improvement
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> Normalized keys are used to improve sorting performance by avoiding random 
> memory accesses. However, currently it has several limitations:
> # Normalized key computer is missing for some types, e.g., UUID
> # It only helps with unequal keys. When two keys are equal, it still requires 
> random accesses to do comparison.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2174) Improve partitioning property propagation in AbstractPreclusteredGroupByPOperator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269494#comment-16269494
 ] 

ASF subversion and git services commented on ASTERIXDB-2174:


Commit dc7e68a0212fd74147c8a77d1cf83a69a186ff57 in asterixdb's branch 
refs/heads/master from [~dlychagin-cb]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=dc7e68a ]

[ASTERIXDB-2174][COMP] Improve partitioning propagation in GroupBy

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Improve partitioning property computation in
  AbstractPreclusteredGroupByPOperator
  If "gby v1 as v2" and input partitioned on (v1)
  then output is partitioned on (v2)

Change-Id: Ib679278d6ee6c53b1d9f581438e8bd04c56c2b08
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2176
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Improve partitioning property propagation in 
> AbstractPreclusteredGroupByPOperator
> -
>
> Key: ASTERIXDB-2174
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2174
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: COMP - Compiler
>Reporter: Dmitry Lychagin
>Assignee: Dmitry Lychagin
>Priority: Minor
>
> Currently AbstractPreclusteredGroupByPOperator propagates its input 
> partitioning property as is. For example if "gby var1 as var2" and input is 
> partitioned on "var1" the output partitioning would also be reported on 
> "var1". The problem is that "var1" is not visible after the group by. The 
> variable in the output partitioning property should be changed from "var1" to 
> "var2" because both these variables are bound to the same value and "var2" is 
> visible after the group by.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1984) Index-Nested-Loop Join should not care about the contents of the probe branch

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263444#comment-16263444
 ] 

ASF subversion and git services commented on ASTERIXDB-1984:


Commit 4dad5dfa87bd7e353f60a7889478a2966daa43ef in asterixdb's branch 
refs/heads/master from [~wangsaeu]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=4dad5df ]

[ASTERIXDB-1984][COMP] probe-subtree init not required - fix #2

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Let the IntroduceJoinAccessMethod accept arbitrary
  forms of sub-tree for the probe-tree.

Change-Id: Ibdf0baa060efa35a173d00e15f0a348937f2e7f6
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2175
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Contrib: Jenkins 
Reviewed-by: Taewoo Kim 


> Index-Nested-Loop Join should not care about the contents of the probe branch
> -
>
> Key: ASTERIXDB-1984
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1984
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>
> Currently, when the optimizer tries to transform a query with the "index_nl" 
> hint, it tries to identify datasource, assign, and unnest operators in both 
> outer (probe) and inner branch. It also tries to identify the fieldname of 
> the variables that are being joined. However, the probe branch can be an 
> arbitrary sub-plan and what only really matters for the probe subtree is the 
> type of the field from the probe tree that is being joined. If that field 
> type is correctly identified, then any form of probe-subtree with a simple 
> data-scan on the inner branch can be correctly transformed into an 
> index-utilization plan. The following queries should be transformed into an 
> index-utilization plan. However, they are not transformed now in the current 
> master.
> E.g.,
> {code}
> SELECT * FROM
> [1, 2, 3] AS bar JOIN foo on bar /*+ indexnl */ =  foo.key;
> SELECT  * FROM
> bar JOIN foo on bar.id = foo.key JOIN datac ON foo.key /*+ indexnl */ = 
> datac.val;
> SELECT  * FROM
> (SELECT id, COUNT(*) FROM bar GROUP BY id) AS barr JOIN foo ON barr.id /*+ 
> indexnl */ =  foo.key;
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2161) java.lang.IllegalStateException when set LSM Memory Component Id

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264009#comment-16264009
 ] 

ASF subversion and git services commented on ASTERIXDB-2161:


Commit 98b9d603e8531139a4668af48092e9c961ee41fb in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=98b9d60 ]

[ASTERIXDB-2161] Fix component id manage lifecycle

- user model changes: no
- storage format changes: no
- interface changes: yes. The interface of LMSIOOperationCallback
is changed

Details:
- The current way of management component ids is not correct,
in presence of that multiple partitions sharing the same primary op
tracker. It's possible when a partition is empty/being flushed,
the next flush is scheduled by another partition, which
will disturb the partition. This patch fixes this by
using the same logic of maintaining flushed LSNs to maintain
component id.
- Extend recycle memory component interface to indicate whether it
switches the new component or not.
- Also fixes [ASTERIXDB-2168] to ensure we do not miss latest flushed
LSNs by advancing io callback before finishing flush

Change-Id: Ifc35184c4d431db9af71cab302439e165ee55f54
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2153
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: abdullah alamoudi 


> java.lang.IllegalStateException when set LSM Memory Component Id
> 
>
> Key: ASTERIXDB-2161
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2161
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> On one NC with multiple partitions, it may through IllegalStateException when 
> we reset the memory component Id. The memory component should receive a 
> larger Id, but for some reason it still receives the old one.
> {code}
> 16:25:40 SEVERE: SEARCH failed to enter components on {"class" : "LSMBTree", 
> "dir" : 
> "/home/jenkins/jenkins/workspace/asterix-gerrit-verify-asterix-app/asterixdb/asterix-app/target/io/dir/asterix_nc1/iodevice1/storage/partition_1/test/LineID_idx_idx_LineID_suppkey",
>  "memory" : 2, "disk" : 1}
> 16:25:40 java.lang.IllegalStateException: LSM memory component receives 
> illegal id. Old id [1510360027993,1510360027993], new id 
> [1510360027993,1510360027993]
> 16:25:40  at 
> org.apache.hyracks.storage.am.lsm.common.impls.AbstractLSMMemoryComponent.resetId(AbstractLSMMemoryComponent.java:276)
> 16:25:40  at 
> org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.recycled(AbstractLSMIOOperationCallback.java:205)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2168) Flushed LSM components receive non-increasing LSNs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264011#comment-16264011
 ] 

ASF subversion and git services commented on ASTERIXDB-2168:


Commit 98b9d603e8531139a4668af48092e9c961ee41fb in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=98b9d60 ]

[ASTERIXDB-2161] Fix component id manage lifecycle

- user model changes: no
- storage format changes: no
- interface changes: yes. The interface of LMSIOOperationCallback
is changed

Details:
- The current way of management component ids is not correct,
in presence of that multiple partitions sharing the same primary op
tracker. It's possible when a partition is empty/being flushed,
the next flush is scheduled by another partition, which
will disturb the partition. This patch fixes this by
using the same logic of maintaining flushed LSNs to maintain
component id.
- Extend recycle memory component interface to indicate whether it
switches the new component or not.
- Also fixes [ASTERIXDB-2168] to ensure we do not miss latest flushed
LSNs by advancing io callback before finishing flush

Change-Id: Ifc35184c4d431db9af71cab302439e165ee55f54
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2153
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Reviewed-by: abdullah alamoudi 


> Flushed LSM components receive non-increasing LSNs
> --
>
> Key: ASTERIXDB-2168
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2168
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage
>Reporter: Chen Luo
>Assignee: Chen Luo
>
> Every time when we flush a memory component for a durable dataset, we always 
> force a log record, and use the LSN of the log record as the FLUSH_LSN of the 
> flushed disk component. Since LSNs are strictly increasing, then FLUSH_LSNs 
> of a index on a partition should be strictly increasing as well. However, 
> this properly may not hold in some case.
> One cause (and there may be other causes as well) is that is the 
> ioOperationCallback.afterFinalize is not synchronized on opTracker. When a 
> flush is finished, we set flushRequested to false and advance the readIndex. 
> And in updateLSN, if a component has been requested to flush, we cannot reset 
> the LSN because otherwise we may lose the previous LSN.
> Here is the issue. A memory component has been flushed. Then the next flush 
> is to be scheduled (coming from other partitions), and we try to updateLSN. 
> But this time afterFinalize is not called yet, and we'll skip the newest LSN 
> (which we shouldn't). Then the flush is scheduled, and we call afterFinalize 
> on the previous flushed component (which is already late at this moment).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2167) Transaction Id should be factor out of the job specification

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271892#comment-16271892
 ] 

ASF subversion and git services commented on ASTERIXDB-2167:


Commit 7650420d90a60974d05894491e9e58ede26f8996 in asterixdb's branch 
refs/heads/master from [~sjaco002]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=7650420 ]

[ASTERIXDB-2167][TX][RT] Remove TxnId from JobSpecification

- user model changes: no
- storage format changes: no
- interface changes: IJobEventListenerFactory

details:
- Remove the TxnId from the compiled job specification
- This enables one job spec to be used by multiple jobs/transactions
- Runtime operators who need the TxnId will pull it from the EventListener

Change-Id: I9526d50b31aebc3bf971d95ba3edf29c0c1066a7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2154
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 


> Transaction Id should be factor out of the job specification
> 
>
> Key: ASTERIXDB-2167
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2167
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Xikui Wang
>
> In building the deployed job feature, the transaction id caused a lot of 
> issues for us since it's embedded in the serialized job spec in some places. 
> It would be good to factor that out of the job spec.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2176) Deletion doesn't work on the RTree index.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273559#comment-16273559
 ] 

ASF subversion and git services commented on ASTERIXDB-2176:


Commit 50de6669f24d376398d6977a363ac4223897c579 in asterixdb's branch 
refs/heads/master from [~luochen01]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=50de666 ]

[ASTERIXDB-2176] Fix deletion on LSMRTree

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- The deletion on LSMRTree is not working on master, because initially
we insert anti-matter tuples into the memory BTree, and during flush
these tuples are copied to the RTree again. However, we forgot to
set these tuples from the in-memory BTree as anti-matter tuples. This
patch fixes this.
- Also modifies the test case to cover this case.

Change-Id: I3d9417e56f06044f585e019089004efd2b2af3b7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2188
Sonar-Qube: Jenkins 
Reviewed-by: Taewoo Kim 
Tested-by: Jenkins 
Integration-Tests: Jenkins 


> Deletion doesn't work on the RTree index.
> -
>
> Key: ASTERIXDB-2176
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2176
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Chen Luo
>
> This is a simplified version of "upsert/primary-secondary-tree" AQL test case.
> spatialData.json file
> { "id": 12, "point": point("6.0,3.0") }
> moreSpatialData.json file
> {"id": 12, "point": point("4.1,7.0")}
> DDL: 
> {code}
> drop dataverse test if exists;
> create dataverse test;
> use dataverse test;
> create type MyRecord as closed {
>   id: int64,
>   point: point
> }
> create dataset UpsertTo(MyRecord)
>  primary key id;
> create dataset UpsertFrom(MyRecord)
>  primary key id;
> create index rtree_index_point on UpsertTo(point) type rtree;
> {code}
> DML
> {code}
> load dataset UpsertTo
> using localfs
> (("path"="asterix_nc1://data/spatial/spatialData.json"),("format"="adm"));
> load dataset UpsertFrom
> using localfs
> (("path"="asterix_nc1://data/spatial/moreSpatialData.json"),("format"="adm"));
> upsert into dataset UpsertTo(
> for $x in dataset UpsertFrom
> return $x
> );
> for $o in dataset('UpsertTo')
> where spatial-intersect($o.point, 
> create-polygon([4.0,1.0,4.0,4.0,12.0,4.0,12.0,1.0]))
> order by $o.id
> return $o;
> {code}
> This DML returns the new record correctly. But, the issue is that the indexed 
> value in RTree has not been updated.
> When searching the rtree_index_point index, the searcher sees the previous 
> value - point("6.0,3.0"), not the new value - point("4.1,7.0"). So, this 
> record will be fetched from the primary index. However, the primary index 
> search returns the updated value and the select() verifies the value. It 
> returns true by coincidence because the new value satisfies the 
> spatial-intersect() condition.
> The secondary index search should see the updated value, not the previous 
> value. And this is an issue for the index-only plan case since it only uses 
> the value from a secondary index.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2177) Overhead of Maintaining Configurable Storage Dir Name

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272634#comment-16272634
 ] 

ASF subversion and git services commented on ASTERIXDB-2177:


Commit d71214ae5197df2b3e39742731b78f8109c05f61 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=d71214a ]

[ASTERIXDB-2177][STO] Use Fixed Storage Root Dir Name

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Eliminate the need to read the storage root dir name
  from cluster properties and use a fixed name (storage).
- Eliminate the need to maintain root_metadata file.

Change-Id: I4e9772e9da10cff33f11353610788ba541a35571
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2182
Sonar-Qube: Jenkins 
Reviewed-by: Michael Blow 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 


> Overhead of Maintaining Configurable Storage Dir Name
> -
>
> Key: ASTERIXDB-2177
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2177
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: STO - Storage
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>Priority: Minor
>
> Currently, users are allowed to specify the storage root directory name 
> before instance creation. Due to this, each node has to maintain another file 
> with a fixed name (root_metadata) to store the name of the storage directory 
> that was used in instance creation. Maintaining this (root_metadata) file 
> adds an overhead without much added value to the user. Therefore, it would be 
> better to have a fixed storage root directory name (storage) and eliminate 
> the need to maintain and look up the (root_metadata) file.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2182) Use JSON-like representation for parameters

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275396#comment-16275396
 ] 

ASF subversion and git services commented on ASTERIXDB-2182:


Commit f3aa19fb150177e21e3cd35803672d38abdb0f6f in asterixdb's branch 
refs/heads/master from Till Westmann
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=f3aa19f ]

[ASTERIXDB-2182][SQL] Change merge policy syntax

- user model changes: yes
  - change the way a merge policy is specified
- storage format changes: no
- interface changes: no

details:
- previously, merge policies are specified as follows:

  prefix_merge (("number"="123"),("size"="456"));
- After this change, the policies are specified as:
   {
"merge-policy": {
  "name": "prefix",
  "parameters": { "number": 123, "size": 456 }
}
  };

- compaction and policy are not key words anymore

Change-Id: I040f4c74cfa0170b128ad5f975e196658776
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2183
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> Use JSON-like representation for parameters
> ---
>
> Key: ASTERIXDB-2182
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2182
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: AQL - Translator AQL, SQL - Translator SQL++
>Reporter: Till
>
> Currently parameters to statements in SQL++ and AQL are provided using a 
> custom syntax, e.g.
> {noformat}
> create dataset LineItem(LineItemType) primary key l_orderkey, l_linenumber 
> using compaction policy correlated-prefix
> (("max-mergable-component-size"="1048576"),
>  ("max-tolerance-component-count"="3"));
> {noformat}
> Instead of using this custom syntax, we should use a JSON-like syntax that 
> aligns better with our data model.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2194) Introduce datasource functions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294130#comment-16294130
 ] 

ASF subversion and git services commented on ASTERIXDB-2194:


Commit b4d166b3ca042ce34d737f5d2a4fb758fa45d3e5 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=b4d166b ]

[ASTERIXDB-2194][COMP] Introduce datasource functions

- user model changes: yes
  Some functions can be datasources
- storage format changes: no
- interface changes: yes
  - Add IDatasourceFunction: A function that is also a datasource
  - Add IFunctionToDataSourceTransformer: transform an unnest
function into a datascan during compilation

Details:
- Currently, functions are location agnostic and are run on
  parameters that are either passed through them during compile
  time or runtime.
- An exception to this is the dataset function which has
  an associated location constraints running on the nodes
  which host the dataset.
- In this change, we introduce a general framework that allows
  creation of new functions similar to the dataset function.
- Such functions are called datasource Functions.
- A datasource function takes constant parameters and run on
  a set of partitions similar to the dataset function.
- The first example of such functions is the DatasetResources
  function.
- The DatasetResources function takes two parameters, a dataverse
  and a dataset. It is then run on all nodes and returns a set
  of dataset resources.
- Test cases are added for this function.

Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2216
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: abdullah alamoudi 
Tested-by: abdullah alamoudi 


> Introduce datasource functions
> --
>
> Key: ASTERIXDB-2194
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2194
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: COMP - Compiler
>Reporter: Abdullah Alamoudi
>Assignee: Abdullah Alamoudi
>
> Sometimes, we would like to be able to query system status. For example:
>1. Disk space.
>2. Number of components of a dataset.
>3. Memory usage.
> And many others. Being able to query such information and utilize the power 
> of the query language and the runtime makes a great investigative/diagnostic 
> tool.
> Currently, there is no easy way to do that. Such functionality can be 
> achieved through:
> 1. External datasets but that takes a lot of work in terms of development and 
> usage.
> 2. Use specific diagnostic end points but then that is also a lot of 
> development work and you end up losing the ability to use the query language.
> Current proposal is to introduce datasource functions. A datasource function 
> is different from normal functions as:
> 1. Takes constants ( as opposed to variables).
> 2. Has location constraints "For a start, it can be on all nodes".
> An example would be the function dataset_resources(String dataverse, String 
> dataset);
> This function takes a dataverse and a dataset and produce a set of json 
> representing the disk resources of the dataset.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2170) SQLPP - Resolution order of implicit field access

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293551#comment-16293551
 ] 

ASF subversion and git services commented on ASTERIXDB-2170:


Commit 0fb6b2302ec95ab14085ac17a8b17d0662e774ad in asterixdb-bad's branch 
refs/heads/master from [~sjaco002]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb-bad.git;h=0fb6b23 ]

Coordinated Change for ASTERIXDB-2170

Change-Id: I96d46b159f1705c644d595f77e5a7c9c9e925375


> SQLPP - Resolution order of implicit field access
> -
>
> Key: ASTERIXDB-2170
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2170
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: SQL - Translator SQL++
>Reporter: Dmitry Lychagin
>Assignee: Dmitry Lychagin
>
> The resolution order should be closest to furthest.
>  
> select * from orders   
> where pid in (select value pid from parts); 
> means
>  
> select * from orders o 
> where o.pid in (select value p.pid from parts p); 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2196) MultiPartitionLSMIndexTest sporadically fails

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288304#comment-16288304
 ] 

ASF subversion and git services commented on ASTERIXDB-2196:


Commit 40ed45f89ef19891fef6750e005aa48010d61da7 in asterixdb's branch 
refs/heads/master from [~alamoudi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=40ed45f ]

[ASTERIXDB-2196][TEST] Fix MultiPartitionLSMIndexTest

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- There is a small window where a flush can be scheduled
  during the check of the component ids. This is avoided
  now by waiting for all io operations.

Change-Id: I7da1ba5859aea4e75eca605113a1f72faa1914c2
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2219
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 


> MultiPartitionLSMIndexTest sporadically fails
> -
>
> Key: ASTERIXDB-2196
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2196
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: STO - Storage, TEST - Test Framework
>Reporter: Ian Maxon
>Assignee: Abdullah Alamoudi
>
> Sometimes the MultiPartitionLSMIndexTest will fail sporadically, with an 
> error like " expected:<[1512792358263,1512792358263]> but 
> was:<[1512792358261,1512792358261]> " 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-2124) Feed fails to start when filter and index both exist on one field

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291745#comment-16291745
 ] 

ASF subversion and git services commented on ASTERIXDB-2124:


Commit 1e864a69e0f31a23b711a221965166066671b05a in asterixdb's branch 
refs/heads/master from [~tillw]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=1e864a6 ]

[ASTERIXDB-2124] Throw NPE early

Change-Id: I84a458d2293450bce973788ac84b6fbc4c993a4a
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2055
Reviewed-by: Ian Maxon 
Tested-by: Ian Maxon 
Reviewed-by: Murtadha Hubail 
Integration-Tests: Murtadha Hubail 
Tested-by: Murtadha Hubail 
Sonar-Qube: Jenkins 
Contrib: Jenkins 
Integration-Tests: Jenkins 


> Feed fails to start when filter and index both exist on one field
> -
>
> Key: ASTERIXDB-2124
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2124
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Steven Jacobs
>Assignee: Xikui Wang
>
> The following gives an internal error, but works successfully when either the 
> index or the filter are removed from the DDLs:
> drop dataverse channels if exists;
> create dataverse channels;
> use dataverse channels;
> create type UserLocation as closed {
>   recordId: integer,
>   location: circle,
>   userName: string,
>   timeStamp: datetime
> };
> create dataset UserLocations(UserLocation)
> primary key recordId with filter on timeStamp;
> create index time2 on UserLocations(timeStamp);
> create feed LocationFeed using socket_adapter
> (
> ("sockets"="127.0.0.1:10009"),
> ("address-type"="IP"),
> ("type-name"="UserLocation"),
> ("upsert-feed"="true"),
> ("format"="adm")
> );
> connect feed LocationFeed to dataset UserLocations;
> start feed LocationFeed;
> StackTrace:
> SEVERE: java.lang.NullPointerException
> org.apache.hyracks.api.exceptions.HyracksDataException: 
> java.lang.NullPointerException
>   at 
> org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:47)
>   at 
> org.apache.asterix.app.active.FeedEventsListener.doStart(FeedEventsListener.java:110)
>   at 
> org.apache.asterix.app.active.ActiveEntityEventsListener.start(ActiveEntityEventsListener.java:374)
>   at 
> org.apache.asterix.app.translator.QueryTranslator.handleStartFeedStatement(QueryTranslator.java:2120)
>   at 
> org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:367)
>   at 
> org.apache.asterix.api.http.server.RestApiServlet.doHandle(RestApiServlet.java:209)
>   at 
> org.apache.asterix.api.http.server.RestApiServlet.getOrPost(RestApiServlet.java:177)
>   at 
> org.apache.asterix.api.http.server.RestApiServlet.get(RestApiServlet.java:161)
>   at 
> org.apache.hyracks.http.server.AbstractServlet.handle(AbstractServlet.java:86)
>   at 
> org.apache.hyracks.http.server.HttpRequestHandler.handle(HttpRequestHandler.java:70)
>   at 
> org.apache.hyracks.http.server.HttpRequestHandler.call(HttpRequestHandler.java:55)
>   at 
> org.apache.hyracks.http.server.HttpRequestHandler.call(HttpRequestHandler.java:36)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
>   at 
> org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression.hashCode(VariableReferenceExpression.java:86)
>   at java.util.HashMap.hash(HashMap.java:338)
>   at java.util.HashMap.get(HashMap.java:556)
>   at 
> org.apache.hyracks.algebricks.rewriter.rules.ExtractCommonExpressionsRule$CommonExpressionSubstitutionVisitor.transform(ExtractCommonExpressionsRule.java:241)
>   at 
> org.apache.hyracks.algebricks.core.algebra.operators.logical.IndexInsertDeleteUpsertOperator.acceptExpressionTransform(IndexInsertDeleteUpsertOperator.java:106)
>   at 
> org.apache.hyracks.algebricks.rewriter.rules.ExtractCommonExpressionsRule.removeCommonExpressions(ExtractCommonExpressionsRule.java:176)
>   at 
> org.apache.hyracks.algebricks.rewriter.rules.ExtractCommonExpressionsRule.removeCommonExpressions(ExtractCommonExpressionsRule.java:144)
>   at 
> org.apache.hyracks.algebricks.rewriter.rules.ExtractCommonExpressionsRule.removeCommonExpressions(ExtractCommonExpressionsRule.java:144)
>   at 
> 

[jira] [Commented] (ASTERIXDB-2201) Add Http Entity + Delete Method to Test Framework

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16295728#comment-16295728
 ] 

ASF subversion and git services commented on ASTERIXDB-2201:


Commit f99da85d77eeabc7d01f8f178b112d63d63b3e42 in asterixdb's branch 
refs/heads/master from [~mhubail]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=f99da85 ]

[ASTERIXDB-2201][TEST] Allow Http Body + Delete Method in Tests

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Allow http body to be specified.
- Allow delete http method.

Change-Id: I43e467951dee69bf5ce5ee112b315beaa354f603
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2234
Sonar-Qube: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Michael Blow 


> Add Http Entity + Delete Method to Test Framework
> -
>
> Key: ASTERIXDB-2201
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2201
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: TEST - Test Framework
>Reporter: Murtadha Hubail
>Assignee: Murtadha Hubail
>
> Add the ability to specify http request entity (body) in test files. In 
> addition, allow http delete method requests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ASTERIXDB-1723) Move *IT tests to use NCService in lieu of asterix-installer/Managix

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292092#comment-16292092
 ] 

ASF subversion and git services commented on ASTERIXDB-1723:


Commit 62b4a027e8eec26f248e6af6254da53e9adb08b9 in asterixdb's branch 
refs/heads/master from [~imaxon]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=62b4a02 ]

[ASTERIXDB-1723] Remove asterix-installer temp pom

This is just to remove the dummy pom that was used before
the main change was merged, to allow all of the tests to
run. Now the jobs have been changed to not require it.

Change-Id: Iaca791e965c095c23ccc5af0f95cba5adcbd9d1e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2228
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Michael Blow 


> Move *IT tests to use NCService in lieu of asterix-installer/Managix
> 
>
> Key: ASTERIXDB-1723
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1723
> Project: Apache AsterixDB
>  Issue Type: Improvement
>  Components: *DB - AsterixDB, ADM - Admin, TEST - Test Framework
>Reporter: Ian Maxon
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)