Hi Marco,
Thank you very much for your time. I abandoned the classpath approach and went
back to "copy-dependency" approach. Given below is the output of sqlline.bat.
Can you quickly confirm if the output is as expected? I will send across the
patch for Windows setup separately for code review.
D:\workspace\incubator-drill>sqlline.bat -u jdbc:drill:schema=parquet-local -n
admin -p admin
D:\workspace\incubator-drill>if not exist contrib\sqlline\target\dependencies
(call mvn -B -f contrib\sqlline\pom.xml package )
D:\workspace\incubator-drill>java -Xmx1G
-Dlogback.configurationFile=.\sqlparser\src\test\resources\logback.xml -cp
".\contrib\sqlline\target\dependencies\*" sqlline.SqlLine --verbose=true -u
jdbc:drill:schema=parquet-local -n admin -p adminLoaded singnal handler:
SunSignalHandlerissuing: !connect jdbc:drill:schema=parquet-local admin
adminscan complete in 9ms22:51:57,241 |-INFO in
ch.qos.logback.classic.LoggerContext[default] - Found resource
[.\sqlparser\src\test\resources\logback.xml] at
[file:/D:/workspace/incubator-drill/./sqlparser/src/test/resources/logback.xml]22:51:57,361
|-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug
attribute not set22:51:57,374 |-INFO in
ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender
of type
[de.huxhorn.lilith.logback.appender.ClassicMultiplexSocketAppender]22:51:57,385
|-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as
[SOCKET]22:51:57,434 |-INFO in
de.huxhorn.lilith.logback.appender.ClassicMultiplexSocketAppender[SOCKET] -
Waiting 1s to establish connections.22:51:58,434 |-INFO in
de.huxhorn.lilith.logback.appender.ClassicMultiplexSocketAppender[SOCKET] -
Started
de.huxhorn.lilith.logback.appender.ClassicMultiplexSocketAppender[SOCKET]22:51:58,434
|-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to
instantiate appender of type [ch.qos.logback.core.ConsoleAppender]22:51:58,439
|-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as
[STDOUT]22:51:58,447 |-INFO in
ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default
type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder]
property22:51:58,506 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter@33:27
- no applicable action for [level], current ElementPath is
[[configuration][appender][level]]22:51:58,506 |-INFO in
ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger
[org.apache.drill] to false22:51:58,506 |-INFO in
ch.qos.logback.classic.joran.action.LevelAction - org.apache.drill level set to
DEBUG22:51:58,507 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction
- Attaching appender named [SOCKET] to Logger[org.apache.drill]22:51:58,507
|-INFO in ch.qos.logback.classic.joran.action.LevelAction - ROOT level set to
DEBUG22:51:58,507 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction
- Attaching appender named [SOCKET] to Logger[ROOT]22:51:58,507 |-INFO in
ch.qos.logback.classic.joran.action.ConfigurationAction - End of
configuration.22:51:58,508 |-INFO in
ch.qos.logback.classic.joran.JoranConfigurator@8c0e89 - Registering current
configuration as safe fallback point
scan complete in 2207msConnecting to jdbc:drill:schema=parquet-localConnected
to: Drill (version 1.0)Driver: Apache Drill JDBC Driver (version 1.0)Autocommit
status: trueTransaction isolation: TRANSACTION_REPEATABLE_READsqlline version
??? by Marc Prud'hommeaux0: jdbc:drill:schema=parquet-local>
Regards,Ravi
> Date: Sun, 29 Dec 2013 12:29:29 -0800
> Subject: Re: [OS: Windows] Could not find or load main class sqlline.SqlLine
> From: [email protected]
> To: [email protected]
>
> Ravi,
>
> sqlline.properties is included in sqlline-1.1.0.jar. You can get rid of
> this message by extracting the file and copying it into your user folder
> C:\Users\ravisankars\sqlline\. Anyway, this is not critical, you can work
> without that file.
>
> The real problem is the other error [No known driver to handle
> "jdbc:drill:schema=parquet-local"]. It looks like the scan of the classes
> does not work with jars included from the manifest file of an executable
> jar (-jar option). If you specify the full class name for the driver by
> appending "-d org.apache.drill.jdbc.Driver", then it finds the Driver class
> but it fails loading it because of a deserialization error in Jackson:
> com.fasterxml.jackson.databind.JsonMappingException: Could not resolve type
> id 'parquet' into a subtype of [simple type, class
> org.apache.drill.common.logical.StorageEngineConfig]
> Basically, Jackson cannot find annotated classes in the external jars.
>
> At this point I think the only other way to get this working is by copying
> all the dependent jars in a "dependencies" folder. You can do this with
> maven:
> ...
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-dependency-plugin</artifactId>
> <version>2.8</version>
> <executions>
> <execution>
> <id>copy-dependencies</id>
> <phase>package</phase>
> <goals>
> <goal>copy-dependencies</goal>
> </goals>
> <configuration>
>
> <outputDirectory>${project.build.directory}/dependencies/</outputDirectory>
> <overWriteReleases>false</overWriteReleases>
> <overWriteSnapshots>false</overWriteSnapshots>
> <overWriteIfNewer>true</overWriteIfNewer>
> </configuration>
> </execution>
> </executions>
> </plugin>
> ...
>
> Then execute sqlline:
> java -Xmx1G
> -Dlogback.configurationFile=./sqlparser/src/test/resources/logback.xml -cp
> "./contrib/sqlline/target/dependencies/*" sqlline.SqlLine --verbose=true -u
> jdbc:drill:schema=parquet-local -n admin -p admin
>
> Hope this will help.
>
> Marco
>
>
>
> On Sat, Dec 28, 2013 at 9:43 PM, Ravisankar Sivasubramaniam <
> [email protected]> wrote:
>
> > Marco,
> > Thank you very much. That worked for me as well. Now I am getting the
> > following error. What is the deal on sqlline.properties?
> > D:\workspace\incubator-drill>java -Xmx1G
> > -Dlogback.configurationFile=.\sqlparser\src\test\resources\logback.xml -jar
> > contrib\sqlline\target\drill-sqlline-1.0.0-
> > m2-incubating-SNAPSHOT.jar --verbose=true -u
> > jdbc:drill:schema=parquet-local -n admin -p admin
> > Loaded singnal handler: SunSignalHandler
> > C:\Users\ravisankars\sqlline\sqlline.properties (The system cannot find the
> > file specified)
> > issuing: !connect jdbc:drill:schema=parquet-local admin admin scan
> > complete in 5ms scan complete in 7ms No known driver to handle
> > "jdbc:drill:schema=parquet-local" sqlline version ??? by Marc Prud'hommeaux
> > sqlline>
> > Best Regards,Ravi> Date: Sat, 28 Dec 2013 12:04:42 -0800
> > > Subject: Re: [OS: Windows] Could not find or load main class
> > sqlline.SqlLine
> > > From: [email protected]
> > > To: [email protected]
> > >
> > > Ravi,
> > >
> > > Can you try changing the classpathPrefix in your maven pom file from
> > > <classpathPrefix>c:\Users\ravisankars\.m2\repository</classpathPrefix>
> > > to
> > > <classpathPrefix>/c:/Users/ravisankars/.m2/repository/</classpathPrefix>
> > > Or better:
> > > <classpathPrefix>/${settings.localRepository}/</classpathPrefix>
> > >
> > > Notice the '/' at the beginning and at the end of the path. This works
> > for
> > > me.
> > >
> > > Best Regards,
> > > Marco
> > >
> > > On Sat, Dec 28, 2013 at 10:00 AM, Ravisankar Sivasubramaniam <
> > > [email protected]> wrote:
> > >
> > > > Progress so far
> > > > -------------------
> > > > Looks like apache-rat-plugin didn't like me logging the content to
> > log.txt
> > > > under incubator-drill. Once I instructed mvn to log the output outside
> > > > the folder, the build got completed without any error.
> > > >
> > > > Open issue
> > > > ---------------
> > > > The sqlline.bat looks as follows:
> > > >
> > > >
> > > >
> > > >
> > > > *if not exist contrib\sqlline\target ( cd contrib\sqlline mvn
> > > > package cd ..\..)*
> > > >
> > > > *java -Xmx1G
> > > > -Dlogback.configurationFile=.\sqlparser\src\test\resources\logback.xml
> > -jar
> > > > contrib\sqlline\target\drill-sqlline-1.0.0-m2-incubating-SNAPSHOT.jar
> > > > --verbose=true*
> > > > The updated pom.xml has been attached. In the pom.xml I have included
> > > > maven-jar-plugin to add Manifest entries for Classpath and Main-Class.
> > I
> > > > have attached the MANIFEST.MF as well for reference. As per jar spec<
> > http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Notes_on_Manifest_and_Signature_Files
> > >,
> > > > maven breaks the classpath at 72 and the continuation starts with a
> > single
> > > > space. I have checked that all the files specified in the classpath is
> > > > available in my local repository.
> > > >
> > > > The problem I am facing is that java is not able to find
> > sqlline.SqlLine
> > > > class from the classpath.
> > > >
> > > >
> > > > *D:\workspace\incubator-drill>java -Xmx1G
> > > > -Dlogback.configurationFile=.\sqlparser\src\test\resources\logback.xml
> > -jar
> > > > contrib\sqlline\target\drill-sqlline-1.0.0-m2-incubating-SNAPSHOT.jar
> > > > --verbose=trueError: Could not find or load main class sqlline.SqlLine*
> > > >
> > > > For now I have ignored passing the command line parameter. But that is
> > > > not related to the problem at hand.
> > > >
> > > > This
> > > > <http://stackoverflow.com/questions/6780678/run-class-in-jar-file>SO
> > answer suggests the command be included in the back quotes, if the
> > > > command is not executed from the directory containing the jar, but
> > Windows
> > > > doesn't support command substitution.
> > > >
> > > > And this<
> > http://stackoverflow.com/questions/16626883/maven-error-trying-to-adding-arbitrary-classpath-in-manifest-mf>SO
> > answer suggests to downgrade the maven version to 2.3.2 and change
> > > > classpath configuration so that each jar path starts in a new line.
> > Since
> > > > I am not manually adding the classpath, I am not sure how to implement
> > this
> > > > suggestion.
> > > >
> > > > Would you please let me know your thoughts on this? I appreciate any
> > help
> > > > on this.
> > > >
> > > > Regards,
> > > > Ravi
> > > >
> > > > ------------------------------
> > > > From: [email protected]
> > > > To: [email protected]
> > > > Subject: RE: [OS: Windows] Could not find or load main class
> > > > sqlline.SqlLine
> > > > Date: Sat, 28 Dec 2013 15:33:44 +0530
> > > >
> > > >
> > > > Some more information to troubleshoot the problem. If I execute "mvn
> > > > clean install -DskipTests", I am getting the output given at the
> > bottom of
> > > > this email, which indicates success. On the other hand if I try to
> > log the
> > > > output into file, I am getting an error. I am attaching the log file
> > for
> > > > reference.
> > > >
> > > > [INFO]
> > > > [INFO] --- maven-install-plugin:2.3.1:install (default-install) @
> > > > distribution ---
> > > > [INFO] Installing D:\workspace\incubator-drill\distribution\pom.xml to
> > > >
> > C:\Users\ravisankars\.m2\repository\org\apache\drill\distribution\1.0.0-m2-incubating-SNA
> > > > PSHOT\distribution-1.0.0-m2-incubating-SNAPSHOT.pom
> > > > [INFO] Installing
> > > >
> > D:\workspace\incubator-drill\distribution\target\apache-drill-1.0.0-m2-incubating-SNAPSHOT-binary-release.tar.gz
> > > > to C:\Users\ravisankars\.m2\r
> > > >
> > > >
> > epository\org\apache\drill\distribution\1.0.0-m2-incubating-SNAPSHOT\distribution-1.0.0-m2-incubating-SNAPSHOT-binary-release.tar.gz
> > > > [INFO]
> > > >
> > ------------------------------------------------------------------------
> > > > [INFO] Reactor Summary:
> > > > [INFO]
> > > > [INFO] Apache Drill Root POM ............................. SUCCESS
> > [2.047s]
> > > > [INFO] Drill Protocol .................................... SUCCESS
> > [3.090s]
> > > > [INFO] Common (Logical Plan, Base expressions) ........... SUCCESS
> > [2.753s]
> > > > [INFO] contrib/Parent Pom ................................ SUCCESS
> > [0.165s]
> > > > [INFO] contrib/hbase-storage-engine ...................... SUCCESS
> > [0.247s]
> > > > [INFO] exec/Parent Pom ................................... SUCCESS
> > [0.179s]
> > > > [INFO] exec/Reference Interpreter ........................ SUCCESS
> > [1.677s]
> > > > [INFO] exec/Netty Little Endian Buffers .................. SUCCESS
> > [0.383s]
> > > > [INFO] exec/Java Execution Engine ........................ SUCCESS
> > > > [12.188s]
> > > > [INFO] SQL Parser ........................................ SUCCESS
> > [1.703s]
> > > > [INFO] contrib/sqlline ................................... SUCCESS
> > [1.897s]
> > > > [INFO] Packaging and Distribution Assembly ............... SUCCESS
> > > > [56.521s]
> > > > [INFO]
> > > >
> > ------------------------------------------------------------------------
> > > > [INFO] BUILD SUCCESS
> > > > [INFO]
> > > >
> > ------------------------------------------------------------------------
> > > > [INFO] Total time: 1:23.110s
> > > > [INFO] Finished at: Sat Dec 28 15:29:55 IST 2013
> > > > [INFO] Final Memory: 52M/168M
> > > > [INFO]
> > > >
> > ------------------------------------------------------------------------
> > > > D:\workspace\incubator-drill>
> > > >
> > > > Regards,
> > > > Ravi
> > > >
> > > > ------------------------------
> > > > From: [email protected]
> > > > To: [email protected]
> > > > Subject: RE: [OS: Windows] Could not find or load main class
> > > > sqlline.SqlLine
> > > > Date: Sat, 28 Dec 2013 14:38:13 +0530
> > > >
> > > > Hello Team,
> > > >
> > > > I resolved the below error by reducing the heap memory. I am not
> > getting
> > > > the following error?
> > > >
> > > > D:\workspace\incubator-drill>java -Xmx1G
> > > > -Dlogback.configurationFile=.\sqlparser\src\test\resources\logback.xml
> > -cp
> > > > contrib\sqlline\target\drill-sqlline-1.0.0-m2-incubating-SNAPSHOT.jar
> > > > sqlline.SqlLine --verbose=true "$@"
> > > > Error: Could not find or load main class sqlline.SqlLine
> > > >
> > > >
> > > > I am not able to identify what I am doing wrong. As per the unix
> > script,
> > > > an sqlline jar is supposed to be available in incubator-drill folder
> > but
> > > > mine doesn't have it. Did I not checkout the folder correctly or did I
> > > > miss a compilation step?
> > > >
> > > > Regards,
> > > > Ravi
> > > >
> > > > > From: [email protected]
> > > > > To: [email protected]
> > > > > Subject: [OS: Windows] Error initializing VM due to lack of space for
> > > > object heap
> > > > > Date: Thu, 26 Dec 2013 18:54:40 +0530
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hello team,
> > > > >
> > > > > I am getting the following error when trying to execute sqlline.
> > This is
> > > > my first attempt. Would you please let me know how much memory is
> > required
> > > > by the VM for Object Heap. Would you please let me know if you have
> > found
> > > > the optimium JVM settings for Apache Drill?
> > > > >
> > > > > D:\workspace\incubator-drill>java -Xmx2G
> > > > -Dlogback.configurationFile=.\sqlparser\src\test\resources\logback.xml
> > -cp
> > > > contrib\sqlline\target\dependency sqlline.Sq
> > > > > lLine --verbose=true "$@"
> > > > > Error occurred during initialization of VM
> > > > > Could not reserve enough space for object heap
> > > > > Error: Could not create the Java Virtual Machine.
> > > > > Error: A fatal exception has occurred. Program will exit.
> > > > >
> > > > > Regards,
> > > > > Ravi
> > > > >
> > > > > From: [email protected]
> > > > > To: [email protected]
> > > > > Subject: Challenge in creating of sqlline.bat due to windows
> > limitation
> > > > > Date: Sun, 22 Dec 2013 22:17:41 +0530
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hello all,
> > > > > Please find the WIP windows batch program. This works only for 86
> > file
> > > > names in .classpath (or 8138 characters long). But the working version
> > of
> > > > .classpath has 131 filenames spanning 12,958 characters. The article
> > > > http://support.microsoft.com/kb/830473 seem to support the variable
> > > > length limitation - official limt is 8191.
> > > > > One possible option is to collate all the required jar files into a
> > > > common folder so that we can work around the variable value limitation
> > in
> > > > Windows platform. Are there any other suggestion from the team?
> > Regards,Ravi
> > > > >
> > > > > > Date: Sun, 22 Dec 2013 11:07:50 +0000
> > > > > > From: [email protected]
> > > > > > To: [email protected]
> > > > > > Subject: [jira] [Created] (DRILL-338) sqlline script not available
> > for
> > > > windows OS
> > > > > >
> > > > > > Ravisankar Sivasubramaniam created DRILL-338:
> > > > > > ------------------------------------------------
> > > > > >
> > > > > > Summary: sqlline script not available for windows OS
> > > > > > Key: DRILL-338
> > > > > > URL: https://issues.apache.org/jira/browse/DRILL-338
> > > > > > Project: Apache Drill
> > > > > > Issue Type: Bug
> > > > > > Affects Versions: 1.0.0-milestone-1, 1.0.0-milestone-2,
> > > > 1.0.0-milestone-3, 1.0.0-milestone-4, 1.0.0
> > > > > > Environment: Windows OS
> > > > > > Reporter: Ravisankar Sivasubramaniam
> > > > > >
> > > > > >
> > > > > > Currently sqlline may be executed only on *nix platforms. Windows
> > > > version of sqlline need to be created for developers using windows pcs.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > This message was sent by Atlassian JIRA
> > > > > > (v6.1.5#6160)
> > > > >
> > > > >
> > > >
> >
> >
