Good afternoon,
I am receiving the following error when using the flatten function to query a
MongoDB collection:
Error: SYSTEM ERROR: IllegalArgumentException: You tried to write a VarChar
type when you are using a ValueWriter of type NullableFloat8WriterImpl.
The collection contains a number documents, where each document consists of
some key-value pairs and a single array which itself contains exactly 2
key-value pairs per element.
Here is an example document in the collection:
{
"_id" : ObjectId("1234567890abcdef12345678"),
"TradeId" : NumberInt(12345),
"DateValueCollection" : [
{
"ScenarioDate" : ISODate("2011-05-20T00:00:00.000+0000"),
"Pnl" : 22.0
},
{
"ScenarioDate" : ISODate("2011-05-23T00:00:00.000+0000"),
"Pnl" : -30.0
},
{
"ScenarioDate" : ISODate("2011-05-24T00:00:00.000+0000"),
"Pnl" : 15.0
},
{
"ScenarioDate" : ISODate("2011-05-25T00:00:00.000+0000"),
"Pnl" : 9.0
}
]
}
Within the array, I have checked that every ScenarioDate value is of MongoDB
type Date and that every PnL value is of MongoDB type double. There are no
null values.
Each document contains this same array structure with approx. 1k such elements.
When I copy a small set of these documents into a new collection, the flatten
functions works correctly. But in the full collection, the flatten function
fails with the error above.
I have tried toggling the store.mongo.read_numbers_as_double and
store.mongo.all_text_mode flags with no luck.
Command used (via shell or web client) and verbose error output for reference:
0: jdbc:drill:zk=local> SELECT flatten(`x`.`DateValueCollection`) FROM
`mongo`.`db_name`.`some.random.collection.name` AS `x` limit 10;
Error: SYSTEM ERROR: IllegalArgumentException: You tried to write a VarChar
type when you are using a ValueWriter of type NullableFloat8WriterImpl.
Fragment 0:0
[Error Id: a43e57f6-f8ce-4850-8040-65858828056f on SYM156.options-it.com:31010]
(state=,code=0)
java.sql.SQLException: SYSTEM ERROR: IllegalArgumentException: You tried to
write a VarChar type when you are using a ValueWriter of type
NullableFloat8WriterImpl.
Fragment 0:0
[Error Id: a43e57f6-f8ce-4850-8040-65858828056f on somehostname:31010]
at
org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
at
org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(DrillCursor.java:290)
at
org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:1923)
at
org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:73)
at
net.hydromatic.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:404)
at
net.hydromatic.avatica.AvaticaStatement.executeQueryInternal(AvaticaStatement.java:355)
at
net.hydromatic.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:338)
at
net.hydromatic.avatica.AvaticaStatement.execute(AvaticaStatement.java:69)
at
org.apache.drill.jdbc.impl.DrillStatementImpl.execute(DrillStatementImpl.java:101)
at sqlline.Commands.execute(Commands.java:841)
at sqlline.Commands.sql(Commands.java:751)
at sqlline.SqlLine.dispatch(SqlLine.java:746)
at sqlline.SqlLine.begin(SqlLine.java:621)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM
ERROR: IllegalArgumentException: You tried to write a VarChar type when you are
using a ValueWriter of type NullableFloat8WriterImpl.
Fragment 0:0
[Error Id: a43e57f6-f8ce-4850-8040-65858828056f on somehostname:31010]
at
org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:119)
at
org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:113)
at
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:46)
at
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:31)
at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:67)
at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:374)
at
org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:89)
at
org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:252)
at
org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:123)
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:285)
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:257)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Unknown Source)
0: jdbc:drill:zk=local>
Thanks for your help.
Kind regards,
Arman
This message (and any attached documents) contains confidential information and
is intended only for the named recipient and may contain confidential,
proprietary or legally privileged information. If you are not the named
addressee you should not disseminate, distribute or copy this e-mail or its
attachments. Please notify the sender immediately by e-mail if you have
received this e-mail by mistake and delete this e-mail and any attachments from
your system. Any views or opinions presented are solely those of the author and
do not necessarily represent those of Symmetry Investments UK LLP or any of its
affiliates.
This message has been issued by Symmetry Investments, or on behalf of one of
its affiliates.
E-mail transmission cannot be guaranteed to be secure or error-free as
information could be intercepted, corrupted, lost, destroyed, arrive late or
incomplete, or contain viruses. The sender therefore does not accept liability
for any errors or omissions in the contents of this message which arise as a
result of e-mail transmission. If verification is required please request a
hard-copy version. This message is provided for informational purposes and
should not be construed as a solicitation or offer to buy or sell any
securities or related financial instruments.
We do not warrant that this e-mail is virus free. You must therefore take full
responsibility for virus checking.
Symmetry Investments UK LLP is incorporated in England & Wales, registration
number OC386690 and whose registered office is at 2nd Floor, 15 Sackville
Street, London, W1S 3DJ, United Kingdom.
Symmetry Investments UK LLP is authorised and regulated in the United Kingdom
by the Financial Conduct Authority.
