[
https://issues.apache.org/jira/browse/AVRO-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900551#comment-17900551
]
Thiruvalluvan M. G. commented on AVRO-4092:
-------------------------------------------
This is neither the problem with Avro nor with Parquet. It so happens that a a
valid field name for Avro *ISSUE(AAA_BBB_CCC)* is not a valid name for Parquet.
On way to resolve the problem is to use the schema-resolving version of Avro
GenericReader. [There are two versions of GenericReader in Avro. The first
takes a single Avro schema and reads the contents conforming to that schema.
The second takes two schemas - a reader schema and a writer schema, it reads
the contents conforming to writer schema and makes it appear as if it is for
the reader schema. This process is called schema resolution]. The solution is
to have another schema identical to the original except that the offending
field names are renamed to those acceptable to Parquet. Keep the original field
names as aliases for the renamed ones. Since {{AvroParquetWriter}} no longer
sees the offending field names, the problem should go away.
> Failed to write Parquet files when column names contains special characters
> like ()*[]{}.
> -----------------------------------------------------------------------------------------
>
> Key: AVRO-4092
> URL: https://issues.apache.org/jira/browse/AVRO-4092
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Reporter: Harini K.K.
> Priority: Blocker
>
> java.lang.Exception: *Illegal character in: ISSUE(AAA_BBB_CCC)*
> at
> com.hexaware.jumbo.util.DrillUtil.processQueryAndSaveResult(DrillUtil.java:803)
> at
> com.hexaware.jumbo.service.DataSourceImpl.getDataCompareResult(DataSourceImpl.java:864)
> at
> com.hexaware.jumbo.controller.DrillApiController.executeTest(DrillApiController.java:221)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
> at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
> at
> org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
> at
> org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
> at
> org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
> at
> org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)
> at
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> at
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:895)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1732)
> at
> org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
> at
> org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
> at
> org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.base/java.lang.Thread.run(Thread.java:833)
> 14-11-2024 11:21:54.618 [http-nio-9091-exec-8] INFO
> com.hexaware.jumbo.util.DrillUtil.saveExportDetails - saveExportDetails()
> starts
> 14-11-2024 11:21:54.622 [Thread-30] INFO
> com.hexaware.jumbo.util.DrillUtil.lambda$9 - saveExportDetails() inside thread
> 14-11-2024 11:21:54.621 [http-nio-9091-exec-8] INFO
> com.hexaware.jumbo.util.DrillUtil.saveExportDetails - saveExportDetails()
> stops
> 14-11-2024 11:21:54.625 [http-nio-9091-exec-8] INFO
> c.h.j.controller.DrillApiController.executeTest - Reponse in Drill
> executeTest : {"errorCode":500,"errorMsg":"Illegal character in: ISSUE
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
