[jira] [Commented] (DRILL-4054) convert_from(,'JSON') gives JsonParseException
[
https://issues.apache.org/jira/browse/DRILL-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997917#comment-14997917
]
Khurram Faraaz commented on DRILL-4054:
---
(1) It is not an easy to understand error message.
(2) When we pass a valid JSON as input source we see a SchemaChangeException
{code}
0: jdbc:drill:schema=dfs.tmp> select t.c1, convert_from(t.c1,'JSON') jsn from
`test.json` t;
Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize
incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[convert_fromjson(MAP-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: dd9c7d11-fb07-436d-adac-c5efd9372cb4 on centos-04.qa.lab:31010]
(state=,code=0)
{code}
Data from test.son file used in above test
{code}
[root@centos-01 ~]# cat test.json
{
"c1":{"key":"value1"}
}
[root@centos-01 ~]#
{code}
> convert_from(,'JSON') gives JsonParseException
> ---
>
> Key: DRILL-4054
> URL: https://issues.apache.org/jira/browse/DRILL-4054
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 1.3.0
>Reporter: Khurram Faraaz
>
> convert_from(,'JSON') gives JsonParseException
> sys.version => 3a73f098
> Drill 1.3
> 4 node cluster CentOS
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select columns[3], convert_from(CAST(columns[3]
> AS VARCHAR(64)),'JSON') json FROM `allData.csv`;
> Error: SYSTEM ERROR: JsonParseException: Unrecognized token
> 'AXCB': was expecting
> ('true', 'false' or 'null')
> at [Source:
> org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line:
> 1, column: 105]
> Fragment 0:0
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> (state=,code=0)
> Stack trace from drillbit.log
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> JsonParseException: Unrecognized token
> 'AXCB': was expecting
> ('true', 'false' or 'null')
> at [Source:
> org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line:
> 1, column: 105]
> Fragment 0:0
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.3.0.jar:1.3.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_85]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_85]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
> Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error
> while converting from JSON.
> at
> org.apache.drill.exec.test.generated.ProjectorGen6.doEval(ProjectorTemplate.java:126)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.ProjectorGen6.projectRecords(ProjectorTemplate.java:62)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:131)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:156)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
>
[jira] [Commented] (DRILL-4054) convert_from(,'JSON') gives JsonParseException
[
https://issues.apache.org/jira/browse/DRILL-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997926#comment-14997926
]
Steven Phillips commented on DRILL-4054:
(1) I agree the message is bad. If that's the issue, please be explicit. It
wasn't clear from the description whether there was an actual bug or just a
request for better error message.
(2) This doesn't seem to have anything to do with the original bug. In fact,
this isn't even a bug. The convert_from function requires a varbinary or
varchar input. It is not possible to perform this function against a MAP type.
> convert_from(,'JSON') gives JsonParseException
> ---
>
> Key: DRILL-4054
> URL: https://issues.apache.org/jira/browse/DRILL-4054
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 1.3.0
>Reporter: Khurram Faraaz
>
> convert_from(,'JSON') gives JsonParseException
> sys.version => 3a73f098
> Drill 1.3
> 4 node cluster CentOS
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select columns[3], convert_from(CAST(columns[3]
> AS VARCHAR(64)),'JSON') json FROM `allData.csv`;
> Error: SYSTEM ERROR: JsonParseException: Unrecognized token
> 'AXCB': was expecting
> ('true', 'false' or 'null')
> at [Source:
> org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line:
> 1, column: 105]
> Fragment 0:0
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> (state=,code=0)
> Stack trace from drillbit.log
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> JsonParseException: Unrecognized token
> 'AXCB': was expecting
> ('true', 'false' or 'null')
> at [Source:
> org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line:
> 1, column: 105]
> Fragment 0:0
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.3.0.jar:1.3.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_85]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_85]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
> Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error
> while converting from JSON.
> at
> org.apache.drill.exec.test.generated.ProjectorGen6.doEval(ProjectorTemplate.java:126)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.ProjectorGen6.projectRecords(ProjectorTemplate.java:62)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:131)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:156)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.7.0_85]
> at javax.security.auth.Subject.doAs(Subject.java:415)
[jira] [Commented] (DRILL-4054) convert_from(,'JSON') gives JsonParseException
[
https://issues.apache.org/jira/browse/DRILL-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997896#comment-14997896
]
Steven Phillips commented on DRILL-4054:
Could you explain what the bug is here?
> convert_from(,'JSON') gives JsonParseException
> ---
>
> Key: DRILL-4054
> URL: https://issues.apache.org/jira/browse/DRILL-4054
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 1.3.0
>Reporter: Khurram Faraaz
>
> convert_from(,'JSON') gives JsonParseException
> sys.version => 3a73f098
> Drill 1.3
> 4 node cluster CentOS
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select columns[3], convert_from(CAST(columns[3]
> AS VARCHAR(64)),'JSON') json FROM `allData.csv`;
> Error: SYSTEM ERROR: JsonParseException: Unrecognized token
> 'AXCB': was expecting
> ('true', 'false' or 'null')
> at [Source:
> org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line:
> 1, column: 105]
> Fragment 0:0
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> (state=,code=0)
> Stack trace from drillbit.log
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> JsonParseException: Unrecognized token
> 'AXCB': was expecting
> ('true', 'false' or 'null')
> at [Source:
> org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line:
> 1, column: 105]
> Fragment 0:0
> [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.3.0.jar:1.3.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_85]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_85]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85]
> Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error
> while converting from JSON.
> at
> org.apache.drill.exec.test.generated.ProjectorGen6.doEval(ProjectorTemplate.java:126)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.ProjectorGen6.projectRecords(ProjectorTemplate.java:62)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:131)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:156)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
> ~[drill-java-exec-1.3.0.jar:1.3.0]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.7.0_85]
> at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_85]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
> ~[hadoop-common-2.7.0-mapr-1506.jar:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250)
> [drill-java-exec-1.3.0.jar:1.3.0]
> ... 4 common frames omitted
> Caused by:
