Re: Debugging source code

[Vignesh Raghunathan]

Hi,

I am trying to step through some of the function descriptor code. For instance, 
I launched an AsterixDB instance via AsterixHyracksIntegrationUtil, placed a 
breakpoint in 

org.apache.asterix.runtime.evaluators.constructors.ATimeConstructorDescriptor.createEvaluatorFactory(...).new
 IScalarEvaluatorFactory() {...}.createScalarEvaluator(...).new 
IScalarEvaluator() {...}.evaluate(IFrameTupleReference, IPointable)

and ran the query 

let $v := time(“12:23:23”)
return $v

However, execution doesn’t stop at the breakpoint. My questions are as follows:

1. http://sigmod.github.io/papers/socc15-final153.pdf 
 mentions runtime artifacts 
called evaluators for function expressions. Is the evaluate function mentioned 
above part of an evaluator for the time constructor?

2. Is a separate JVM launched for each virtual node controller? 

3. How can I debug a task that is scheduled to run in a particular node 
controller?

As an aside, http://sigmod.github.io/papers/socc15-final153.pdf 
 mentions that translation 
between logical functions and evaluators happen during the job generation 
phase. Does that mean the rule based physical optimization phase is skipped for 
functions? 

I am also not able to figure out where the function evaluators are actually 
called in code. Where can I find them?

Thanks,
Vignesh


> On Jul 7, 2016, at 2:23 AM, Mike Carey  wrote:
> 
> Vignesh,
> 
> Count would actually be a particularly bad example to start with.  :-)  
> Aggregate functions (count, min, max, sum, avg) are not like other functions 
> - they are special and more complicated / very different.  Instead, you 
> should look at simple(r) functions like almost anything else described at 
> https://asterixdb.apache.org/docs/0.8.8-incubating/aql/functions.html.
> 
> 
> Cheers,
> 
> Mike
> 
> On 7/6/16 11:29 AM, Xikui Wang wrote:
>> Hi Vignesh,
>> 
>> There is an entrance class called: AsterixHyracksIntegrationUtil. You can
>> find it under: org.apache.asterix.api.common. The main method in this class
>> enables you to launch a local instance for debugging. From my own
>> experience, QueryTranslator will be a good start point to trace the process.
>> 
>> Best,
>> Xikui
>> 
>> On Wed, Jul 6, 2016 at 11:16 AM, Vignesh Raghunathan <
>> vignesh.raghunatha...@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I've been trying to figure out how functions are implemented under the hood
>>> in AsterixDB. For example, I would like to know what happens when the aql
>>> statement count([1,2,3]) is executed.
>>> 
>>> I tried to set up remote debugging in eclipse to step through the code.
>>> However, I'm not sure what to change to launch an asterixdb instance in
>>> debug mode. Is there a way to do it?
>>> 
>>> Thanks,
>>> Vignesh
>>> 
> 

Re: Errors, exceptions, and error codes

[Taewoo Kim]

Got it. Thanks!

Best,
Taewoo

On Fri, Jul 15, 2016 at 1:30 PM, Till Westmann  wrote:

> There’s not example so far. I think/hope that we’ll see one in one of
> Abdullah’s bug fixes :)
>
>
> On 15 Jul 2016, at 13:16, Taewoo Kim wrote:
>
> It's a great idea. Can you show us a before/after example that you have
>> replaced already in your branch if there is any?
>>
>> Best,
>> Taewoo
>>
>> On Fri, Jul 15, 2016 at 11:50 AM, Till Westmann  wrote:
>>
>> Hi,
>>>
>>> triggered by a recent bug fix that required an non-trivial error
>>> reporting/handling workflow across the Hyracks/AsterixDB boundary, I’d
>>> like to
>>> propose a change/extension to the way we manage exceptions.
>>>
>>> The change is that we’d limit the number of exception classes to a very
>>> few
>>> (HyracksDataException for runtime errors, AlgebricksExeptions for
>>> compiler
>>> errors, maybe 1 or 2 more) and introduce error codes to identify errors
>>> further.
>>>
>>> The use of error codes has a number of advantages:
>>> 1) Error codes usually use simple types (strings or integers) and thus
>>> make it
>>>easy to be transported across system boundaries (Hyracks, AsterixDB,
>>> VXQuery,
>>>clients, …) as opposed to subclasses of exceptions that need to be
>>> available
>>>in the respective VMs.
>>> 2) Error codes (once established) make it easy for customers to search
>>> for
>>>existing solutions (try to google for e.g. "ORA-12154" [1] :) ).
>>> 3) Error codes can be used a a key for error messages in different
>>> languages
>>>(if we want to expand our reach).
>>>
>>> I’ve created en example of what this could initially look like for
>>> HyracksDataException [2]. You’ll see there that I propose to add
>>> a) a component (e.g. "Hyracks", "Hyracks Storage", "AsterixDB", …),
>>> b) an error code (that is unique within a component), and
>>> c) an array of Serializable objects
>>> to the HyracksDataException. While the component and the error code are
>>> used to
>>> uniquely identify an error, the array is used to provide additional
>>> information
>>> for an error. This information that could either be used when handling
>>> the
>>> error or when parameterizing a (potential language-specific) error
>>> message.
>>> E.g. the error message could be "%1$s does not contain %2$s" and %1 and
>>> %2
>>> would be substituted with actual values when reporting the error.
>>>
>>> Please take a look at the proposal and reply with questions, comments,
>>> suggestions, or alternatives.
>>>
>>> Thanks,
>>> Till
>>>
>>> [1] https://www.google.com/search?q=ORA-12154
>>> [2] https://asterix-gerrit.ics.uci.edu/#/c/1000/
>>>
>>>

Re: Errors, exceptions, and error codes

[Till Westmann]

There’s not example so far. I think/hope that we’ll see one in one 
of

Abdullah’s bug fixes :)

On 15 Jul 2016, at 13:16, Taewoo Kim wrote:

It's a great idea. Can you show us a before/after example that you 
have

replaced already in your branch if there is any?

Best,
Taewoo

On Fri, Jul 15, 2016 at 11:50 AM, Till Westmann  
wrote:



Hi,

triggered by a recent bug fix that required an non-trivial error
reporting/handling workflow across the Hyracks/AsterixDB boundary, 
I’d

like to
propose a change/extension to the way we manage exceptions.

The change is that we’d limit the number of exception classes to a 
very few
(HyracksDataException for runtime errors, AlgebricksExeptions for 
compiler
errors, maybe 1 or 2 more) and introduce error codes to identify 
errors

further.

The use of error codes has a number of advantages:
1) Error codes usually use simple types (strings or integers) and 
thus

make it
   easy to be transported across system boundaries (Hyracks, 
AsterixDB,

VXQuery,
   clients, …) as opposed to subclasses of exceptions that need to 
be

available
   in the respective VMs.
2) Error codes (once established) make it easy for customers to 
search for

   existing solutions (try to google for e.g. "ORA-12154" [1] :) ).
3) Error codes can be used a a key for error messages in different
languages
   (if we want to expand our reach).

I’ve created en example of what this could initially look like for
HyracksDataException [2]. You’ll see there that I propose to add
a) a component (e.g. "Hyracks", "Hyracks Storage", "AsterixDB", …),
b) an error code (that is unique within a component), and
c) an array of Serializable objects
to the HyracksDataException. While the component and the error code 
are

used to
uniquely identify an error, the array is used to provide additional
information
for an error. This information that could either be used when 
handling the
error or when parameterizing a (potential language-specific) error 
message.
E.g. the error message could be "%1$s does not contain %2$s" and %1 
and %2

would be substituted with actual values when reporting the error.

Please take a look at the proposal and reply with questions, 
comments,

suggestions, or alternatives.

Thanks,
Till

[1] https://www.google.com/search?q=ORA-12154
[2] https://asterix-gerrit.ics.uci.edu/#/c/1000/

Re: Errors, exceptions, and error codes

[Taewoo Kim]

It's a great idea. Can you show us a before/after example that you have
replaced already in your branch if there is any?

Best,
Taewoo

On Fri, Jul 15, 2016 at 11:50 AM, Till Westmann  wrote:

> Hi,
>
> triggered by a recent bug fix that required an non-trivial error
> reporting/handling workflow across the Hyracks/AsterixDB boundary, I’d
> like to
> propose a change/extension to the way we manage exceptions.
>
> The change is that we’d limit the number of exception classes to a very few
> (HyracksDataException for runtime errors, AlgebricksExeptions for compiler
> errors, maybe 1 or 2 more) and introduce error codes to identify errors
> further.
>
> The use of error codes has a number of advantages:
> 1) Error codes usually use simple types (strings or integers) and thus
> make it
>easy to be transported across system boundaries (Hyracks, AsterixDB,
> VXQuery,
>clients, …) as opposed to subclasses of exceptions that need to be
> available
>in the respective VMs.
> 2) Error codes (once established) make it easy for customers to search for
>existing solutions (try to google for e.g. "ORA-12154" [1] :) ).
> 3) Error codes can be used a a key for error messages in different
> languages
>(if we want to expand our reach).
>
> I’ve created en example of what this could initially look like for
> HyracksDataException [2]. You’ll see there that I propose to add
> a) a component (e.g. "Hyracks", "Hyracks Storage", "AsterixDB", …),
> b) an error code (that is unique within a component), and
> c) an array of Serializable objects
> to the HyracksDataException. While the component and the error code are
> used to
> uniquely identify an error, the array is used to provide additional
> information
> for an error. This information that could either be used when handling the
> error or when parameterizing a (potential language-specific) error message.
> E.g. the error message could be "%1$s does not contain %2$s" and %1 and %2
> would be substituted with actual values when reporting the error.
>
> Please take a look at the proposal and reply with questions, comments,
> suggestions, or alternatives.
>
> Thanks,
> Till
>
> [1] https://www.google.com/search?q=ORA-12154
> [2] https://asterix-gerrit.ics.uci.edu/#/c/1000/
>

Errors, exceptions, and error codes

[Till Westmann]

Hi,

triggered by a recent bug fix that required an non-trivial error
reporting/handling workflow across the Hyracks/AsterixDB boundary, I’d 
like to

propose a change/extension to the way we manage exceptions.

The change is that we’d limit the number of exception classes to a 
very few
(HyracksDataException for runtime errors, AlgebricksExeptions for 
compiler

errors, maybe 1 or 2 more) and introduce error codes to identify errors
further.

The use of error codes has a number of advantages:
1) Error codes usually use simple types (strings or integers) and thus 
make it
   easy to be transported across system boundaries (Hyracks, AsterixDB, 
VXQuery,
   clients, …) as opposed to subclasses of exceptions that need to be 
available

   in the respective VMs.
2) Error codes (once established) make it easy for customers to search 
for

   existing solutions (try to google for e.g. "ORA-12154" [1] :) ).
3) Error codes can be used a a key for error messages in different 
languages

   (if we want to expand our reach).

I’ve created en example of what this could initially look like for
HyracksDataException [2]. You’ll see there that I propose to add
a) a component (e.g. "Hyracks", "Hyracks Storage", "AsterixDB", …),
b) an error code (that is unique within a component), and
c) an array of Serializable objects
to the HyracksDataException. While the component and the error code are 
used to
uniquely identify an error, the array is used to provide additional 
information
for an error. This information that could either be used when handling 
the
error or when parameterizing a (potential language-specific) error 
message.
E.g. the error message could be "%1$s does not contain %2$s" and %1 and 
%2

would be substituted with actual values when reporting the error.

Please take a look at the proposal and reply with questions, comments,
suggestions, or alternatives.

Thanks,
Till

[1] https://www.google.com/search?q=ORA-12154
[2] https://asterix-gerrit.ics.uci.edu/#/c/1000/

Re: Status

[Murtadha Hubail]

It's been a busy week for me. I just worked a little bit more on 
AqlMetadataProvider refactoring.

-Murtadha

> On Jul 15, 2016, at 2:10 PM, Chris Hillery  wrote:
> 
> - Discussions inside Couchbase regarding perf testing (meeting with perf
> team member later today)
> - Worked with Mike B a bit on improvements to current functional testing
> 
> Ceej
> aka Chris Hillery

Status

[Chris Hillery]

- Discussions inside Couchbase regarding perf testing (meeting with perf
team member later today)
- Worked with Mike B a bit on improvements to current functional testing

Ceej
aka Chris Hillery