Re: new AsterixDB web interface demo up and running

[Kaveen Rodrigo]

Hey Mike,

unordered list , Since I'm parsing the data from the Asterix HTTP API which
returns results using JSON, therefore the JSON encoder at the sending end
has already converted the unordered list to JavaScript Array.

the thing with the existing UI is that UI's result is generated by the
servlet it's hosting the UI (see APIServlet.java), When I initially
discussed about the UI with Ian we wanted the new UI to work with the
Asterix HTTP API which holds the issue of not responding with a proper JSON.

On 17 June 2016 at 06:15, Mike Carey  wrote:

> Sounds like a bug in the underlying http UI?!  It would be nice to
> preserve the multiple-result-area approach that the existing web UI uses in
> that case, somehow...  It's interesting that this hasn't come up before -
> we should have test cases for the basic UI for that, I would think?
>
>
>
> On 6/16/16 5:24 PM, Ian Maxon wrote:
>
>> Kaveen and I talked about this earlier today actually, the result (from
>> *DB) for the two above queries is actually fine and parseable JSON. It's
>> just a labeling issue as the result is shown as if it were 3 records
>> rather
>> than 3 lists.
>> A more vexing question however that came up is what to do about multiple
>> queries in one submission. Right now those come back as multiple JSON
>> objects appended to each other apparently.
>>
>> On Thu, Jun 16, 2016 at 5:14 PM, Mike Carey  wrote:
>>
>> @Ian & @Chris:  Can you provide some helpful hints in the direction of
>>> parsing returned ADM?  (Since you are kind of addressing that as we speak
>>> for other reasons?)
>>>
>>> @Kaveen:  Off to a cool start!  In terms of the sorts of things that can
>>> come back, *conceptually*, the return clause of a query can yield a
>>> scalar
>>> value, an ordered list, an unordered list, or a record.  (The various
>>> possible scalar values are all of the data types listed in the ADM data
>>> model spec.)  A for-clause actually always returns a list of whatever the
>>> return clause says to return - and a let-clause (I believe) or a
>>> standalone
>>> expression can return a singleton object (of any of the aforementioned
>>> forms) if I'm not mistaken.  For testing the Web UI, it would probably be
>>> worth coming up with a set of test queries that returns each of those
>>> things.  (Mixed of them are also possible - life in semistructured data
>>> land can be messy.)
>>>
>>> Cheers,
>>>
>>> Mike
>>>
>>>
>>> On 6/16/16 11:11 AM, Kaveen Rodrigo wrote:
>>>
>>> oh I see, Thank you Yingyi,

 I did update the VPS with the fixes for Q1, the only way to fix Q2 and
 the
 new query is to write a little parser since that output isn't valid
 json.

 cheers,
 Kaveen

 On 16 June 2016 at 22:59, Yingyi Bu  wrote:

 Any valid ADM (asterix data model) instance can be a result row.

> ADM: https://ci.apache.org/projects/asterixdb/aql/datamodel.html
>
> A single curly bracket means a record constructor.  A record consists
> of
> fields, where each field is an name-value pair.
> Therefore,
> {
>   [1,2,3],
>   [2,3,4],
>   [5,6,7]
> }
> cannot be a valid result.
>
> But you are able to get
>   [1,2,3],
>   [2,3,4],
>   [5,6,7]
>
> by running the following query:
>
> for $x in [
>   [1,2,3],
>   [2,3,4],
>   [5,6,7]
> ]
> return $x;
>
> Let me know if you have more questions.
>
> Best,
> Yingyi
>
>
>
>
>
>
> On Thu, Jun 16, 2016 at 10:23 AM, Kaveen Rodrigo <
> u.k.k.rodr...@gmail.com>
> wrote:
>
> Hey Yingyi,
>
>> I fixed that issue, didn't update the VPS yet.  One question Can there
>> be
>> results which returns arrays?
>>
>> for example
>> {
>>   [1,2,3],
>>   [2,3,4],
>>   [5,6,7]
>> }
>>
>> if that's so, if you have some time can you give me an AQl query which
>> will produce something like that.
>>
>> thanks in advance,
>> Kaveen
>>
>> On 16 June 2016 at 22:28, Yingyi Bu  wrote:
>>
>> Awesome!  Thanks, Kaveen!
>>
>>> Best,
>>> Yingyi
>>>
>>> On Thu, Jun 16, 2016 at 9:56 AM, Kaveen Rodrigo <
>>>
>>> u.k.k.rodr...@gmail.com
>> wrote:
>>
>>> Yikes, Thanks Yingyi,
>>>
 I never expected the results array to contain values, I'll get on it

 On 16 June 2016 at 21:48, Yingyi Bu  wrote:

 Very cool!!

> It seems that there is an assumption that returned results are
>
> records?

>>> For example,  you can try the following query:
>>>
 Q1:
> for $m  in dataset Metadata.Dataset
> return $m.DatasetName;
>
> Q2:
> 1+1;
>
> 

Re: new AsterixDB web interface demo up and running

[Mike Carey]

Sounds like a bug in the underlying http UI?!  It would be nice to 
preserve the multiple-result-area approach that the existing web UI uses 
in that case, somehow...  It's interesting that this hasn't come up 
before - we should have test cases for the basic UI for that, I would think?



On 6/16/16 5:24 PM, Ian Maxon wrote:

Kaveen and I talked about this earlier today actually, the result (from
*DB) for the two above queries is actually fine and parseable JSON. It's
just a labeling issue as the result is shown as if it were 3 records rather
than 3 lists.
A more vexing question however that came up is what to do about multiple
queries in one submission. Right now those come back as multiple JSON
objects appended to each other apparently.

On Thu, Jun 16, 2016 at 5:14 PM, Mike Carey  wrote:


@Ian & @Chris:  Can you provide some helpful hints in the direction of
parsing returned ADM?  (Since you are kind of addressing that as we speak
for other reasons?)

@Kaveen:  Off to a cool start!  In terms of the sorts of things that can
come back, *conceptually*, the return clause of a query can yield a scalar
value, an ordered list, an unordered list, or a record.  (The various
possible scalar values are all of the data types listed in the ADM data
model spec.)  A for-clause actually always returns a list of whatever the
return clause says to return - and a let-clause (I believe) or a standalone
expression can return a singleton object (of any of the aforementioned
forms) if I'm not mistaken.  For testing the Web UI, it would probably be
worth coming up with a set of test queries that returns each of those
things.  (Mixed of them are also possible - life in semistructured data
land can be messy.)

Cheers,

Mike


On 6/16/16 11:11 AM, Kaveen Rodrigo wrote:


oh I see, Thank you Yingyi,

I did update the VPS with the fixes for Q1, the only way to fix Q2 and the
new query is to write a little parser since that output isn't valid json.

cheers,
Kaveen

On 16 June 2016 at 22:59, Yingyi Bu  wrote:

Any valid ADM (asterix data model) instance can be a result row.

ADM: https://ci.apache.org/projects/asterixdb/aql/datamodel.html

A single curly bracket means a record constructor.  A record consists of
fields, where each field is an name-value pair.
Therefore,
{
  [1,2,3],
  [2,3,4],
  [5,6,7]
}
cannot be a valid result.

But you are able to get
  [1,2,3],
  [2,3,4],
  [5,6,7]

by running the following query:

for $x in [
  [1,2,3],
  [2,3,4],
  [5,6,7]
]
return $x;

Let me know if you have more questions.

Best,
Yingyi






On Thu, Jun 16, 2016 at 10:23 AM, Kaveen Rodrigo <
u.k.k.rodr...@gmail.com>
wrote:

Hey Yingyi,

I fixed that issue, didn't update the VPS yet.  One question Can there
be
results which returns arrays?

for example
{
  [1,2,3],
  [2,3,4],
  [5,6,7]
}

if that's so, if you have some time can you give me an AQl query which
will produce something like that.

thanks in advance,
Kaveen

On 16 June 2016 at 22:28, Yingyi Bu  wrote:

Awesome!  Thanks, Kaveen!

Best,
Yingyi

On Thu, Jun 16, 2016 at 9:56 AM, Kaveen Rodrigo <


u.k.k.rodr...@gmail.com
wrote:

Yikes, Thanks Yingyi,

I never expected the results array to contain values, I'll get on it

On 16 June 2016 at 21:48, Yingyi Bu  wrote:

Very cool!!

It seems that there is an assumption that returned results are


records?

For example,  you can try the following query:

Q1:
for $m  in dataset Metadata.Dataset
return $m.DatasetName;

Q2:
1+1;

Best,
Yingyi


On Thu, Jun 16, 2016 at 12:20 AM, Kaveen Rodrigo <


u.k.k.rodr...@gmail.com>


wrote:

Hello all,

I have been working on the WebUI for AsterixDB during the last


month,

and I

would like to get some input from you guys. There are a couple of


visual
tweaks to be applied but the base features are pretty much done.

http://173.82.2.197:19006/

please excuse the slow performance of my docker instance, If you


want

to

check this out locally, It's available here





https://github.com/KaveenR/incubator-asterixdb/commits/asterixdb-1375-static-servlet


cheers,

Kaveen Rodrigo

--
Kaveen Rodrigo
Personal Website - (lifeofenigma.com)


Software Engineering Undergraduate,
Informatics Institute of Technology Sri Lanka (Affiliated with


University
of Westminster, UK)


*Contact Information*

*Mobile* +94779684740

*E-Mail*  kaveen.2013...@iit.ac.lk | u.k.k.rodr...@gmail.com



--
Kaveen Rodrigo
Personal Website - (lifeofenigma.com)


Software Engineering Undergraduate,
Informatics Institute of Technology Sri Lanka (Affiliated with


University
of Westminster, UK)


*Contact Information*

*Mobile* +94779684740

*E-Mail*  kaveen.2013...@iit.ac.lk | u.k.k.rodr...@gmail.com



--
Kaveen Rodrigo
Personal Website - (lifeofenigma.com)


Software Engineering Undergraduate,
Informatics Institute of Technology Sri Lanka (Affiliated with
University
of Westminster, UK)


*Contact 

Re: Unsigned integers data types

[Ian Maxon]

My 0.2c is that int64 is big enough if size was the consideration. Usually,
the times that I have wished I had unsigned integers in Java, were not
related to size constraints, but rather when I had to implement something
that required a lot of bitwise operations, since signedness makes that more
complicated. Usually int64 is big enough, and if it isn't, uint64 isn't
much better because you are probably representing something so huge that
arbitrary precision arithmetic is more appropriate.

On Thu, Jun 16, 2016 at 3:47 PM, Ildar Absalyamov <
ildar.absalya...@gmail.com> wrote:

> Things like Spark and Flink don’t do that as well, but because they need
> integration with proper Java types.
>
> > On Jun 16, 2016, at 15:45, Yingyi Bu  wrote:
> >
> >>> Is there any database or SQL implementation supporting that?
> > Ok, it turns out MySQL supports that,  while Postgres, MS SQL and Hive do
> > not have that.
> >
> > Best,
> > Yingyi
> >
> > On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu  wrote:
> >
>  I guess part of the reason why we do that is because Java used to lack
> >> native support of unsigned integers.
> >> Is there any database or SQL implementation supporting that?
> >>
> >> FYI:
> >>
> >>
> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
> >>
> >> Best,
> >> Yingyi
> >>
> >>
> >> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
> >> ildar.absalya...@gmail.com> wrote:
> >>
> >>> Hi devs,
> >>>
> >>> As I was generating various data distributions for statistics
> experiments
> >>> one thing kept bothering me.
> >>> All Asterix integer types (int8, int16, int32, int64) are signed.
> However
> >>> majority of real use cases does not require negative integer values.
> Seems
> >>> like we are waisting half of the data range on something which does
> not get
> >>> used that often. I guess part of the reason why we do that is because
> Java
> >>> used to lack native support of unsigned integers. But since Java 8
> there
> >>> are methods which do unsigned comparison and division (summation,
> >>> subtraction, multiplication are the same in both signed and unsigned
> >>> cases). So it seems like conversion to support unsigned integers would
> not
> >>> be that difficult.
> >>>
> >>> Any thoughts on whether we need unsigned integers in the type system?
> >>>
> >>> Best regards,
> >>> Ildar
> >>>
> >>>
> >>
>
> Best regards,
> Ildar
>
>

Re: Unsigned integers data types

[Ildar Absalyamov]

Things like Spark and Flink don’t do that as well, but because they need 
integration with proper Java types.

> On Jun 16, 2016, at 15:45, Yingyi Bu  wrote:
> 
>>> Is there any database or SQL implementation supporting that?
> Ok, it turns out MySQL supports that,  while Postgres, MS SQL and Hive do
> not have that.
> 
> Best,
> Yingyi
> 
> On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu  wrote:
> 
 I guess part of the reason why we do that is because Java used to lack
>> native support of unsigned integers.
>> Is there any database or SQL implementation supporting that?
>> 
>> FYI:
>> 
>> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
>> 
>> Best,
>> Yingyi
>> 
>> 
>> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
>> ildar.absalya...@gmail.com> wrote:
>> 
>>> Hi devs,
>>> 
>>> As I was generating various data distributions for statistics experiments
>>> one thing kept bothering me.
>>> All Asterix integer types (int8, int16, int32, int64) are signed. However
>>> majority of real use cases does not require negative integer values. Seems
>>> like we are waisting half of the data range on something which does not get
>>> used that often. I guess part of the reason why we do that is because Java
>>> used to lack native support of unsigned integers. But since Java 8 there
>>> are methods which do unsigned comparison and division (summation,
>>> subtraction, multiplication are the same in both signed and unsigned
>>> cases). So it seems like conversion to support unsigned integers would not
>>> be that difficult.
>>> 
>>> Any thoughts on whether we need unsigned integers in the type system?
>>> 
>>> Best regards,
>>> Ildar
>>> 
>>> 
>> 

Best regards,
Ildar

Re: Unsigned integers data types

[Yingyi Bu]

>> Is there any database or SQL implementation supporting that?
Ok, it turns out MySQL supports that,  while Postgres, MS SQL and Hive do
not have that.

Best,
Yingyi

On Thu, Jun 16, 2016 at 3:40 PM, Yingyi Bu  wrote:

> >> I guess part of the reason why we do that is because Java used to lack
> native support of unsigned integers.
> Is there any database or SQL implementation supporting that?
>
> FYI:
>
> http://dba.stackexchange.com/questions/53050/why-arent-unsigned-integer-types-available-in-the-top-database-platforms
>
> Best,
> Yingyi
>
>
> On Thu, Jun 16, 2016 at 3:27 PM, Ildar Absalyamov <
> ildar.absalya...@gmail.com> wrote:
>
>> Hi devs,
>>
>> As I was generating various data distributions for statistics experiments
>> one thing kept bothering me.
>> All Asterix integer types (int8, int16, int32, int64) are signed. However
>> majority of real use cases does not require negative integer values. Seems
>> like we are waisting half of the data range on something which does not get
>> used that often. I guess part of the reason why we do that is because Java
>> used to lack native support of unsigned integers. But since Java 8 there
>> are methods which do unsigned comparison and division (summation,
>> subtraction, multiplication are the same in both signed and unsigned
>> cases). So it seems like conversion to support unsigned integers would not
>> be that difficult.
>>
>> Any thoughts on whether we need unsigned integers in the type system?
>>
>> Best regards,
>> Ildar
>>
>>
>

Unsigned integers data types

[Ildar Absalyamov]

Hi devs,

As I was generating various data distributions for statistics experiments one 
thing kept bothering me.
All Asterix integer types (int8, int16, int32, int64) are signed. However 
majority of real use cases does not require negative integer values. Seems like 
we are waisting half of the data range on something which does not get used 
that often. I guess part of the reason why we do that is because Java used to 
lack native support of unsigned integers. But since Java 8 there are methods 
which do unsigned comparison and division (summation, subtraction, 
multiplication are the same in both signed and unsigned cases). So it seems 
like conversion to support unsigned integers would not be that difficult.

Any thoughts on whether we need unsigned integers in the type system?

Best regards,
Ildar

Re: new AsterixDB web interface demo up and running

[Kaveen Rodrigo]

Hey Yingyi,

I fixed that issue, didn't update the VPS yet.  One question Can there be
results which returns arrays?

for example
{
[1,2,3],
[2,3,4],
[5,6,7]
}

if that's so, if you have some time can you give me an AQl query which
will produce something like that.

thanks in advance,
Kaveen

On 16 June 2016 at 22:28, Yingyi Bu  wrote:

> Awesome!  Thanks, Kaveen!
>
> Best,
> Yingyi
>
> On Thu, Jun 16, 2016 at 9:56 AM, Kaveen Rodrigo 
> wrote:
>
> > Yikes, Thanks Yingyi,
> >
> > I never expected the results array to contain values, I'll get on it
> >
> > On 16 June 2016 at 21:48, Yingyi Bu  wrote:
> >
> > > Very cool!!
> > > It seems that there is an assumption that returned results are records?
> > >
> > > For example,  you can try the following query:
> > > Q1:
> > > for $m  in dataset Metadata.Dataset
> > > return $m.DatasetName;
> > >
> > > Q2:
> > > 1+1;
> > >
> > > Best,
> > > Yingyi
> > >
> > >
> > > On Thu, Jun 16, 2016 at 12:20 AM, Kaveen Rodrigo <
> > u.k.k.rodr...@gmail.com>
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > > I have been working on the WebUI for AsterixDB during the last month,
> > > and I
> > > > would like to get some input from you guys. There are a couple of
> > visual
> > > > tweaks to be applied but the base features are pretty much done.
> > > >
> > > > http://173.82.2.197:19006/
> > > >
> > > > please excuse the slow performance of my docker instance, If you want
> > to
> > > > check this out locally, It's available here
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/KaveenR/incubator-asterixdb/commits/asterixdb-1375-static-servlet
> > > >
> > > > cheers,
> > > > Kaveen Rodrigo
> > > >
> > > > --
> > > > Kaveen Rodrigo
> > > > Personal Website - (lifeofenigma.com)
> > > >
> > > >
> > > > Software Engineering Undergraduate,
> > > > Informatics Institute of Technology Sri Lanka (Affiliated with
> > University
> > > > of Westminster, UK)
> > > >
> > > >
> > > > *Contact Information*
> > > >
> > > > *Mobile* +94779684740
> > > >
> > > > *E-Mail*  kaveen.2013...@iit.ac.lk | u.k.k.rodr...@gmail.com
> > > >
> > >
> >
> >
> >
> > --
> > Kaveen Rodrigo
> > Personal Website - (lifeofenigma.com)
> >
> >
> > Software Engineering Undergraduate,
> > Informatics Institute of Technology Sri Lanka (Affiliated with University
> > of Westminster, UK)
> >
> >
> > *Contact Information*
> >
> > *Mobile* +94779684740
> >
> > *E-Mail*  kaveen.2013...@iit.ac.lk | u.k.k.rodr...@gmail.com
> >
>



-- 
Kaveen Rodrigo
Personal Website - (lifeofenigma.com)


Software Engineering Undergraduate,
Informatics Institute of Technology Sri Lanka (Affiliated with University
of Westminster, UK)


*Contact Information*

*Mobile* +94779684740

*E-Mail*  kaveen.2013...@iit.ac.lk | u.k.k.rodr...@gmail.com