Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?

[Yingyi Bu]

+1 for HTTP API.
The end point can be easily removed from a real deployment by using a
config parameter.

Best,
Yingyi

On Wed, Apr 12, 2017 at 6:12 PM, Till Westmann  wrote:

> Coming back to the question in the subject: If we need such functionality
> e.g. for testing of performance measurements, how should we expose it?
> Should we
> - extend the language with new keywords or
> - introduce on keyword for this that is followed by some parameters or
> - expose new HTTP APIs or
> - …
>
> Thoughts?
>
> Cheers,
> Till
>
>
> On 12 Apr 2017, at 15:11, Ian Maxon wrote:
>
> Yes, more or less. I'm not sure that I would force the buffer cache,
>> but at least I want all of the "object" representations outside of
>> that to have vanished such that their state must be regenerated from
>> the at-rest format.
>>
>> On Wed, Apr 12, 2017 at 2:41 PM, Mike Carey  wrote:
>>
>>> So you want to basically quiesce a dataset "all the way" - so that it's
>>> not
>>> open at all, and perhaps so that the buffer pool also has none of its
>>> pages
>>> resident?
>>>
>>>
>>>
>>> On 4/12/17 2:26 PM, Ian Maxon wrote:
>>>

 That's a little different than from what I'd like to do. Basically I
 want to evict the dataset entirely. My alterior motive here is to test
 the correctness of persisted storage artifacts as right now in many
 cases this is not covered. It has been a recurring theme in many bugs
 that there is some stored artifact on disk that is written incorrectly
 but never read during tests, because often the tests simply never
 encounter the path in which this would occur (restart,eviction, etc.)

 On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
  wrote:

>
> Ian,
>
> There is an existing API, which does exactly that.
> https://github.com/apache/asterixdb/blob/master/asterixdb/
> asterix-app/src/main/java/org/apache/asterix/api/http/
> server/ConnectorApiServlet.java#L123
>  asterix-app/src/main/java/org/apache/asterix/api/http/
> server/ConnectorApiServlet.java#L123>
> I have been successfully using it in my experiments.
>
> On Apr 12, 2017, at 13:41, Ian Maxon  wrote:
>>
>> Hey all,
>> I was working on a patch that would add a 'flush dataset' DDL command
>> (mainly for testing), and of course this would require adding 'flush'
>> as a new reserved word. What is the consensus on when this would be
>> permitted at this point? There are other ways to do this, of course, I
>> could expose this functionality through a separate API, and I am not
>> really partial to one solution or the other.
>>
>> Thanks,
>> - Ian
>>
>
> Best regards,
> Ildar
>
>
>>>

Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?

[Till Westmann]

Coming back to the question in the subject: If we need such 
functionality e.g. for testing of performance measurements, how should 
we expose it? Should we

- extend the language with new keywords or
- introduce on keyword for this that is followed by some parameters or
- expose new HTTP APIs or
- …

Thoughts?

Cheers,
Till

On 12 Apr 2017, at 15:11, Ian Maxon wrote:


Yes, more or less. I'm not sure that I would force the buffer cache,
but at least I want all of the "object" representations outside of
that to have vanished such that their state must be regenerated from
the at-rest format.

On Wed, Apr 12, 2017 at 2:41 PM, Mike Carey  wrote:
So you want to basically quiesce a dataset "all the way" - so that 
it's not
open at all, and perhaps so that the buffer pool also has none of its 
pages

resident?



On 4/12/17 2:26 PM, Ian Maxon wrote:


That's a little different than from what I'd like to do. Basically I
want to evict the dataset entirely. My alterior motive here is to 
test

the correctness of persisted storage artifacts as right now in many
cases this is not covered. It has been a recurring theme in many 
bugs
that there is some stored artifact on disk that is written 
incorrectly

but never read during tests, because often the tests simply never
encounter the path in which this would occur (restart,eviction, 
etc.)


On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
 wrote:


Ian,

There is an existing API, which does exactly that.
https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123

I have been successfully using it in my experiments.


On Apr 12, 2017, at 13:41, Ian Maxon  wrote:

Hey all,
I was working on a patch that would add a 'flush dataset' DDL 
command
(mainly for testing), and of course this would require adding 
'flush'
as a new reserved word. What is the consensus on when this would 
be
permitted at this point? There are other ways to do this, of 
course, I
could expose this functionality through a separate API, and I am 
not

really partial to one solution or the other.

Thanks,
- Ian


Best regards,
Ildar

Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?

[Ian Maxon]

Yes, more or less. I'm not sure that I would force the buffer cache,
but at least I want all of the "object" representations outside of
that to have vanished such that their state must be regenerated from
the at-rest format.

On Wed, Apr 12, 2017 at 2:41 PM, Mike Carey  wrote:
> So you want to basically quiesce a dataset "all the way" - so that it's not
> open at all, and perhaps so that the buffer pool also has none of its pages
> resident?
>
>
>
> On 4/12/17 2:26 PM, Ian Maxon wrote:
>>
>> That's a little different than from what I'd like to do. Basically I
>> want to evict the dataset entirely. My alterior motive here is to test
>> the correctness of persisted storage artifacts as right now in many
>> cases this is not covered. It has been a recurring theme in many bugs
>> that there is some stored artifact on disk that is written incorrectly
>> but never read during tests, because often the tests simply never
>> encounter the path in which this would occur (restart,eviction, etc.)
>>
>> On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
>>  wrote:
>>>
>>> Ian,
>>>
>>> There is an existing API, which does exactly that.
>>> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123
>>> 
>>> I have been successfully using it in my experiments.
>>>
 On Apr 12, 2017, at 13:41, Ian Maxon  wrote:

 Hey all,
 I was working on a patch that would add a 'flush dataset' DDL command
 (mainly for testing), and of course this would require adding 'flush'
 as a new reserved word. What is the consensus on when this would be
 permitted at this point? There are other ways to do this, of course, I
 could expose this functionality through a separate API, and I am not
 really partial to one solution or the other.

 Thanks,
 - Ian
>>>
>>> Best regards,
>>> Ildar
>>>
>

Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?

[Mike Carey]

So you want to basically quiesce a dataset "all the way" - so that it's 
not open at all, and perhaps so that the buffer pool also has none of 
its pages resident?



On 4/12/17 2:26 PM, Ian Maxon wrote:

That's a little different than from what I'd like to do. Basically I
want to evict the dataset entirely. My alterior motive here is to test
the correctness of persisted storage artifacts as right now in many
cases this is not covered. It has been a recurring theme in many bugs
that there is some stored artifact on disk that is written incorrectly
but never read during tests, because often the tests simply never
encounter the path in which this would occur (restart,eviction, etc.)

On Wed, Apr 12, 2017 at 1:46 PM, Ildar Absalyamov
 wrote:

Ian,

There is an existing API, which does exactly that. 
https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123
 

I have been successfully using it in my experiments.


On Apr 12, 2017, at 13:41, Ian Maxon  wrote:

Hey all,
I was working on a patch that would add a 'flush dataset' DDL command
(mainly for testing), and of course this would require adding 'flush'
as a new reserved word. What is the consensus on when this would be
permitted at this point? There are other ways to do this, of course, I
could expose this functionality through a separate API, and I am not
really partial to one solution or the other.

Thanks,
- Ian

Best regards,
Ildar

Re: When is it appropirate to add reserved words to the AQL/SQL++ Grammar?

[Ildar Absalyamov]

Ian,

There is an existing API, which does exactly that. 
https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ConnectorApiServlet.java#L123
 

I have been successfully using it in my experiments.

> On Apr 12, 2017, at 13:41, Ian Maxon  wrote:
> 
> Hey all,
> I was working on a patch that would add a 'flush dataset' DDL command
> (mainly for testing), and of course this would require adding 'flush'
> as a new reserved word. What is the consensus on when this would be
> permitted at this point? There are other ways to do this, of course, I
> could expose this functionality through a separate API, and I am not
> really partial to one solution or the other.
> 
> Thanks,
> - Ian

Best regards,
Ildar

When is it appropirate to add reserved words to the AQL/SQL++ Grammar?

[Ian Maxon]

Hey all,
I was working on a patch that would add a 'flush dataset' DDL command
(mainly for testing), and of course this would require adding 'flush'
as a new reserved word. What is the consensus on when this would be
permitted at this point? There are other ways to do this, of course, I
could expose this functionality through a separate API, and I am not
really partial to one solution or the other.

Thanks,
- Ian