[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Ian Maxon (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494780#comment-15494780
 ] 

Ian Maxon commented on ASTERIXDB-1616:
--

It's fixed for me. 

> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Wail Alkowaileet
>  Labels: soon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488901#comment-15488901
 ] 

ASF subversion and git services commented on ASTERIXDB-1616:


Commit 93a67deebf5bcbcb525623cd183b09ac9f0f5e02 in asterixdb's branch 
refs/heads/master from [~buyingyi]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=93a67de ]

ASTERIXDB-1334 and ASTERIXDB-1616: Add a regression test.

Change-Id: I7e88adbfa2721fa589e1d0564bb9e0889b722e2e
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1165
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Ian Maxon 
Integration-Tests: Jenkins 


> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482466#comment-15482466
 ] 

ASF subversion and git services commented on ASTERIXDB-1616:


Commit dd3660520bcf3bbd1fb209af62a454e5259e69f2 in asterixdb's branch 
refs/heads/master from [~wyk]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=dd36605 ]

ASTERIXDB-1334 and ASTERIXDB-1616: Fix null bitmap size.

Change-Id: I1eaa215b8fc2df02fecb7f86cc5169f07765b09d
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1163
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Yingyi Bu 


> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Wail Alkowaileet (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482171#comment-15482171
 ] 

Wail Alkowaileet commented on ASTERIXDB-1616:
-

Hi Ian,

I apologies for hijacking this ... but after 7 hours of debugging I found the 
bug!  

> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Ian Maxon (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476208#comment-15476208
 ] 

Ian Maxon commented on ASTERIXDB-1616:
--

I have where it should be happening in ADMDataParser isolated now, but I am 
still digging. Nothing is seeming obviously wrong with how the record is 
written just yet. There is more than one tweet ID that will cause this, but I 
am looking at 662781883844030464. 

> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Ian Maxon (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15473748#comment-15473748
 ] 

Ian Maxon commented on ASTERIXDB-1616:
--

This doesnt seem to be deterministic exactly. The fact it happens inside the 
stdUrl field is deterministic, but which record will cause this changes per 
load. I have it pared down to about ~1000 records and it reproduces, any less 
and it works fine. 

> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Wail Alkowaileet (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457826#comment-15457826
 ] 

Wail Alkowaileet commented on ASTERIXDB-1616:
-

Hi Ian,

Unfortunately, I tried to see what's going on but really couldn't figure it out.

> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Ian Maxon (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451380#comment-15451380
 ] 

Ian Maxon commented on ASTERIXDB-1616:
--

I did also notice that the size looked more reasonable if I shifted up one byte 
on the size. I wasn't really sure if that was a coincidence or what though. 
1334 and this do bear a striking resemblance now that I look at it closely, 
even down to the coincidence that the tweet that seems to have an error uses 
Arabic script as well. I remember last week swearing I saw this issue before, 
but I couldn't find the JIRA...
Are you also working on this actively? If so we should probably decide who 
should be assigned, because it seems odd for both of us to work on the same 
thing simultaneously.  I was looking at this bug because it's blocking some of 
the SDSC guys' queries (hence the urls in the description). 

> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ASTERIXDB-1616) NPE when printing record inside open type with unicode fields

[Wail Alkowaileet (JIRA)]

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451212#comment-15451212
 ] 

Wail Alkowaileet commented on ASTERIXDB-1616:
-

The problem occurs with variable length fields. When you project (return) only 
primitive types at level zero (the dataset type), the issue disappears.
I remember I tried to debug it and it seems that the length offset of a string 
(for example) is off by one byte (which leads to read some random 4-bytes that 
are way larger than the frame size. Hence, it throws IndexOutOfBoundExcpetion 
sometimes). However, if you make the schema all open (key-only) this should 
solve the problem. Which makes me wonder if that has anything to do with the 
parser? 

I suspect the problem resides in the ARecordBuilder and friends. I will try to 
print the record right before flush (and after the parsing) and see if I get 
the same issue.



> NPE when printing record inside open type with unicode fields
> -
>
> Key: ASTERIXDB-1616
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1616
> Project: Apache AsterixDB
>  Issue Type: Bug
>Reporter: Ian Maxon
>Assignee: Ian Maxon
>
> DDL: 
> https://github.com/kevincoakley/asterixdb_tests/blob/master/notebooks/asterixdb-spark/Count%20one_percent%20Tweets%20Spark%20Single.ipynb
> Data: 
> https://object.cloud.sdsc.edu/v1/AUTH_kcoakley/asterixdblogs/2015_11_07_00_onepercent.txt
> Basically just a scan+limit on the one_percent dataset will give 
> IndexOutOfBounds. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)