Hi Charles, In the code that you had pasted
this.buffer.setBytes(0, bytes, 0, stringLength); what guarantees that this.buffer has enough space for stringLength chars? Karthik On Fri, Jan 25, 2019 at 2:32 PM Karthikeyan Manivannan <kmanivan...@mapr.com> wrote: > Charles, > > Does this work on 1.15 ? > > Drill 1.16 is able to correctly read CSV files with > 256 char strings, so > I guess the problem might be in the Syslog plugin code. > Can you share your format plugin code ? > > Thanks. > > Karthik > > > On Thu, Jan 24, 2019 at 4:24 PM Charles Givre <cgi...@gmail.com> wrote: > >> Here you go… Thanks for your help! >> >> >> SELECT >> event_date,severity_code,facility_code,severity,facility,ip,app_name,process_id,message_id,structured_data_text,structured_data_UserAgent,structured_data_UserHostAddress,structured_data_BrowserSession,structured_data_Realm,structured_data_Appliance,structured_data_Company,structured_data_UserID,structured_data_PEN,structured_data_HostName,structured_data_Category,structured_data_Priority,message >> FROM cp.`syslog/test.syslog1` >> 18:16:50.061 [main] ERROR org.apache.drill.TestReporter - Test Failed (d: >> 0 B(1 B), h: 21.6 MiB(130.9 MiB), nh: 2.0 MiB(82.2 MiB)): >> testExplicitFlattenedStructuredDataQuery(org.apache.drill.exec.store.syslog.TestSyslogFormat) >> java.lang.NullPointerException: null >> at >> org.apache.drill.test.rowSet.RowSetComparison.verifyAndClearAll(RowSetComparison.java:276) >> ~[drill-java-exec-1.16.0-SNAPSHOT-tests.jar:1.16.0-SNAPSHOT] >> at >> org.apache.drill.exec.store.syslog.TestSyslogFormat.testExplicitFlattenedStructuredDataQuery(TestSyslogFormat.java:310) >> ~[test-classes/:na] >> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_65] >> 18:16:50.571 [main] ERROR org.apache.drill.TestReporter - Test Failed (d: >> 0 B(1 B), h: 20.5 MiB(209.9 MiB), nh: 411.1 KiB(85.4 MiB)): >> testStarFlattenedStructuredDataQuery(org.apache.drill.exec.store.syslog.TestSyslogFormat) >> java.lang.NullPointerException: null >> at >> org.apache.drill.test.rowSet.RowSetComparison.verifyAndClearAll(RowSetComparison.java:276) >> ~[drill-java-exec-1.16.0-SNAPSHOT-tests.jar:1.16.0-SNAPSHOT] >> at >> org.apache.drill.exec.store.syslog.TestSyslogFormat.testStarFlattenedStructuredDataQuery(TestSyslogFormat.java:248) >> ~[test-classes/:na] >> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_65] >> [ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: >> 3.173 s <<< FAILURE! - in >> org.apache.drill.exec.store.syslog.TestSyslogFormat >> [ERROR] >> testExplicitFlattenedStructuredDataQuery(org.apache.drill.exec.store.syslog.TestSyslogFormat) >> Time elapsed: 0.234 s <<< ERROR! >> java.lang.NullPointerException >> at >> org.apache.drill.exec.store.syslog.TestSyslogFormat.testExplicitFlattenedStructuredDataQuery(TestSyslogFormat.java:310) >> >> [ERROR] >> testStarFlattenedStructuredDataQuery(org.apache.drill.exec.store.syslog.TestSyslogFormat) >> Time elapsed: 0.125 s <<< ERROR! >> java.lang.NullPointerException >> at >> org.apache.drill.exec.store.syslog.TestSyslogFormat.testStarFlattenedStructuredDataQuery(TestSyslogFormat.java:248) >> >> [INFO] >> [INFO] Results: >> [INFO] >> [ERROR] Errors: >> [ERROR] TestSyslogFormat.testExplicitFlattenedStructuredDataQuery:310 » >> NullPointer >> [ERROR] TestSyslogFormat.testStarFlattenedStructuredDataQuery:248 » >> NullPointer >> [INFO] >> [ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 0 >> [INFO] >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] BUILD FAILURE >> [INFO] >> ------------------------------------------------------------------------ >> [INFO] Total time: 33.919 s >> [INFO] Finished at: 2019-01-24T18:16:51-05:00 >> [INFO] >> ------------------------------------------------------------------------ >> [ERROR] Failed to execute goal >> org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M2:test (default-test) >> on project drill-format-syslog: There are test failures. >> [ERROR] >> [ERROR] Please refer to >> /Users/cgivre/github/drill-dev/drill/contrib/format-syslog/target/surefire-reports >> for the individual test results. >> [ERROR] Please refer to dump files (if any exist) [date].dump, >> [date]-jvmRun[N].dump and [date].dumpstream. >> [ERROR] -> [Help 1] >> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute >> goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M2:test >> (default-test) on project drill-format-syslog: There are test failures. >> >> Please refer to >> /Users/cgivre/github/drill-dev/drill/contrib/format-syslog/target/surefire-reports >> for the individual test results. >> Please refer to dump files (if any exist) [date].dump, >> [date]-jvmRun[N].dump and [date].dumpstream. >> at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> (MojoExecutor.java:215) >> at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> (MojoExecutor.java:156) >> at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> (MojoExecutor.java:148) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject >> (LifecycleModuleBuilder.java:117) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject >> (LifecycleModuleBuilder.java:81) >> at >> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build >> (SingleThreadedBuilder.java:56) >> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute >> (LifecycleStarter.java:128) >> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) >> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) >> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) >> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) >> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) >> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) >> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke >> (NativeMethodAccessorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke >> (DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke (Method.java:497) >> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced >> (Launcher.java:289) >> at org.codehaus.plexus.classworlds.launcher.Launcher.launch >> (Launcher.java:229) >> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode >> (Launcher.java:415) >> at org.codehaus.plexus.classworlds.launcher.Launcher.main >> (Launcher.java:356) >> Caused by: org.apache.maven.plugin.MojoFailureException: There are test >> failures. >> >> Please refer to >> /Users/cgivre/github/drill-dev/drill/contrib/format-syslog/target/surefire-reports >> for the individual test results. >> Please refer to dump files (if any exist) [date].dump, >> [date]-jvmRun[N].dump and [date].dumpstream. >> at org.apache.maven.plugin.surefire.SurefireHelper.throwException >> (SurefireHelper.java:271) >> at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution >> (SurefireHelper.java:159) >> at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary >> (SurefirePlugin.java:373) >> at >> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked >> (AbstractSurefireMojo.java:1016) >> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute >> (AbstractSurefireMojo.java:846) >> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo >> (DefaultBuildPluginManager.java:137) >> at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> (MojoExecutor.java:210) >> at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> (MojoExecutor.java:156) >> at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> (MojoExecutor.java:148) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject >> (LifecycleModuleBuilder.java:117) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject >> (LifecycleModuleBuilder.java:81) >> at >> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build >> (SingleThreadedBuilder.java:56) >> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute >> (LifecycleStarter.java:128) >> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) >> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) >> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) >> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) >> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) >> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) >> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke >> (NativeMethodAccessorImpl.java:62) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke >> (DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke (Method.java:497) >> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced >> (Launcher.java:289) >> at org.codehaus.plexus.classworlds.launcher.Launcher.launch >> (Launcher.java:229) >> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode >> (Launcher.java:415) >> at org.codehaus.plexus.classworlds.launcher.Launcher.main >> (Launcher.java:356) >> [ERROR] >> [ERROR] >> [ERROR] For more information about the errors and possible solutions, >> please read the following articles: >> [ERROR] [Help 1] >> https://urldefense.proofpoint.com/v2/url?u=http-3A__cwiki.apache.org_confluence_display_MAVEN_MojoFailureException&d=DwIFaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=HlugibuI4IVjs-VMnFvNTcaBtEaDDqE4Ya96cugWqJ8&m=FYH3iU2DaKxg6aW__bi0KbNTl6lqB4g5vyB4-lVHbPk&s=dp6wxwSeKJ-a87PHGpnBjn7aOhg4DNJucCXguyddfMQ&e= >> >> >> If you run the query in sqline, here is the result: >> >> >> jdbc:drill:zk=local> select * from dfs.test.`test.syslog1`; >> Error: DATA_READ ERROR: Error parsing file >> >> DATA_READ ERROR: Maximum Error Threshold Exceeded: >> >> Line: 1 >> DATA_WRITE ERROR: null >> >> Could not write string: >> index: 0, length: 342 (expected: range(0, 256)) >> >> [Error Id: 39ac3038-63b7-4190-bcc7-e2bdda924021 ] >> >> [Error Id: 84807daf-ae51-4a2b-ba2a-1729facdb891 ] >> Fragment 0:0 >> >> [Error Id: 6054aa6b-91df-43b5-8157-a45b4e316e7f on 192.168.1.21:31010] >> (state=,code=0) >> java.sql.SQLException: DATA_READ ERROR: Error parsing file >> >> DATA_READ ERROR: Maximum Error Threshold Exceeded: >> >> Line: 1 >> DATA_WRITE ERROR: null >> >> Could not write string: >> index: 0, length: 342 (expected: range(0, 256)) >> >> [Error Id: 39ac3038-63b7-4190-bcc7-e2bdda924021 ] >> >> [Error Id: 84807daf-ae51-4a2b-ba2a-1729facdb891 ] >> Fragment 0:0 >> >> [Error Id: 6054aa6b-91df-43b5-8157-a45b4e316e7f on 192.168.1.21:31010] >> at >> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) >> at >> org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(DrillCursor.java:608) >> at >> org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:1288) >> at >> org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:61) >> at >> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:667) >> at >> org.apache.drill.jdbc.impl.DrillMetaImpl.prepareAndExecute(DrillMetaImpl.java:1107) >> at >> org.apache.drill.jdbc.impl.DrillMetaImpl.prepareAndExecute(DrillMetaImpl.java:1118) >> at >> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) >> at >> org.apache.drill.jdbc.impl.DrillConnectionImpl.prepareAndExecuteInternal(DrillConnectionImpl.java:200) >> at >> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) >> at >> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:217) >> at sqlline.Commands.execute(Commands.java:938) >> at sqlline.Commands.sql(Commands.java:882) >> at sqlline.SqlLine.dispatch(SqlLine.java:725) >> at sqlline.SqlLine.begin(SqlLine.java:540) >> at sqlline.SqlLine.start(SqlLine.java:264) >> at sqlline.SqlLine.main(SqlLine.java:195) >> Caused by: org.apache.drill.common.exceptions.UserRemoteException: >> DATA_READ ERROR: Error parsing file >> >> DATA_READ ERROR: Maximum Error Threshold Exceeded: >> >> Line: 1 >> DATA_WRITE ERROR: null >> >> Could not write string: >> index: 0, length: 342 (expected: range(0, 256)) >> >> [Error Id: 39ac3038-63b7-4190-bcc7-e2bdda924021 ] >> >> [Error Id: 84807daf-ae51-4a2b-ba2a-1729facdb891 ] >> Fragment 0:0 >> >> [Error Id: 6054aa6b-91df-43b5-8157-a45b4e316e7f on 192.168.1.21:31010] >> at >> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) >> at >> org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) >> at >> org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) >> at >> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) >> at >> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) >> at >> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) >> at >> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) >> at >> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) >> at >> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) >> at >> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) >> at >> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) >> at >> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) >> at >> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) >> at >> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) >> at >> io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) >> at >> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) >> at >> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) >> at >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) >> at >> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911) >> at >> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) >> at >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) >> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) >> at >> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.lang.Exception: DATA_READ ERROR: Error parsing file >> >> DATA_READ ERROR: Maximum Error Threshold Exceeded: >> >> Line: 1 >> DATA_WRITE ERROR: null >> >> Could not write string: >> index: 0, length: 342 (expected: range(0, 256)) >> >> [Error Id: 39ac3038-63b7-4190-bcc7-e2bdda924021 ] >> >> [Error Id: 84807daf-ae51-4a2b-ba2a-1729facdb891 ] >> Fragment 0:0 >> >> [Error Id: 6054aa6b-91df-43b5-8157-a45b4e316e7f on 192.168.1.21:31010] >> at >> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) >> at >> org.apache.drill.exec.store.syslog.SyslogRecordReader.next(SyslogRecordReader.java:169) >> at >> org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:223) >> at >> org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:271) >> at >> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) >> at >> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) >> at >> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) >> at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:143) >> at >> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) >> at >> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) >> at >> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83) >> at >> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) >> at org.apache.drill.exec.work >> .fragment.FragmentExecutor$1.run(FragmentExecutor.java:297) >> at org.apache.drill.exec.work >> .fragment.FragmentExecutor$1.run(FragmentExecutor.java:284) >> at .......(:0) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) >> at org.apache.drill.exec.work >> .fragment.FragmentExecutor.run(FragmentExecutor.java:284) >> at >> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) >> at .......(:0) >> >> >> If you run a query w/o long fields it works fine: >> dbc:drill:zk=local> select * from dfs.test.`logs.syslog1`; >> >> +--------------------------+----------------+----------------+-----------+-----------+------------------------+-----------+-------------+------------------------------------------------+-------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------+ >> | event_date | severity_code | facility_code | severity >> | facility | ip | app_name | message_id | >> message | process_id | >> structured_data_text | >> structured_data | >> >> +--------------------------+----------------+----------------+-----------+-----------+------------------------+-----------+-------------+------------------------------------------------+-------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------+ >> | 2003-10-11 22:14:15.003 | 2 | 4 | CRIT >> | AUTH | mymachine.example.com | su | ID47 | BOM'su >> root' failed for lonvick on /dev/pts/8 | null | null >> | {} >> | >> | 1985-04-12 23:20:50.52 | 2 | 4 | CRIT >> | AUTH | mymachine.example.com | su | ID47 | BOM'su >> root' failed for lonvick on /dev/pts/8 | null | null >> | {} >> | >> | 1985-04-12 23:20:50.52 | 2 | 4 | CRIT >> | AUTH | mymachine.example.com | su | ID47 | BOM'su >> root' failed for lonvick on /dev/pts/8 | null | null >> | {} >> | >> | 2003-10-11 22:14:15.003 | 2 | 4 | CRIT >> | AUTH | mymachine.example.com | su | ID47 | BOM'su >> root' failed for lonvick on /dev/pts/8 | null | null >> | {} >> | >> | 2003-08-24 12:14:15.0 | 2 | 4 | CRIT >> | AUTH | mymachine.example.com | su | ID47 | BOM'su >> root' failed for lonvick on /dev/pts/8 | null | null >> | {} >> | >> | 2003-08-24 12:14:15.0 | 5 | 20 | NOTICE >> | LOCAL4 | 192.0.2.1 | myproc | null | %% It's >> time to make the do-nuts. | 8710 | null >> | {} >> | >> | 2003-10-11 22:14:15.003 | 5 | 20 | NOTICE >> | LOCAL4 | mymachine.example.com | evntslog | ID47 | null >> | null | >> {examplePriority@32473=[class=high], exampleSDID@32473=[iut=3, >> eventSource=Application, eventID=1011]} | >> {"class":"high","iut":"3","eventSource":"Application","eventID":"1011"} | >> | 2003-10-11 22:14:15.003 | 5 | 20 | NOTICE >> | LOCAL4 | mymachine.example.com | evntslog | ID47 | - and >> thats a wrap! | null | >> {examplePriority@32473=[class=high], exampleSDID@32473=[iut=3, >> eventSource=Application, eventID=1011]} | >> {"class":"high","iut":"3","eventSource":"Application","eventID":"1011"} | >> >> +--------------------------+----------------+----------------+-----------+-----------+------------------------+-----------+-------------+------------------------------------------------+-------------+----------------------------------------------------------------------------------+--------------------------------------------------------------------------+ >> 8 rows selected (1.702 seconds) >> >> >> > On Jan 24, 2019, at 17:02, Karthikeyan Manivannan <kmanivan...@mapr.com> >> wrote: >> > >> > Hi Charles, >> > >> > Can you please provide the stack trace. >> > >> > Thanks >> > >> > Karthik >> > >> > On Tue, Jan 22, 2019 at 9:18 PM Charles Givre <cgi...@gmail.com> wrote: >> > >> >> Hello all, >> >> I’m working on a format plugin to read syslog data, and have >> encountered >> >> what seems to be a bit of a regression (maybe). The code below is a >> >> helper function which writes strings from the data. As of Drill 1.16, >> the >> >> varchar holder seems to throw an error if the string you are trying to >> >> write is > 256 characters. Is there a workaround? >> >> >> >> Thanks! >> >> >> >> >> >> //Helper function to map strings >> >> private void mapStringField(String name, String value, >> >> BaseWriter.MapWriter map) { >> >> if (value == null) { >> >> return; >> >> } >> >> try { >> >> byte[] bytes = value.getBytes("UTF-8"); >> >> int stringLength = bytes.length; >> >> this.buffer.setBytes(0, bytes, 0, stringLength); >> >> map.varChar(name).writeVarChar(0, stringLength, buffer); >> >> } catch (Exception e) { >> >> throw UserException >> >> .dataWriteError() >> >> .addContext("Could not write string: ") >> >> .addContext(e.getMessage()) >> >> .build(logger); >> >> } >> >> } >> >> >> >> >> >> >> >> >> >> >> >>
