Hi Todd,
I agree with you. It seems that the line in function appendCellValueDebugString
in PartialRow.java (line 595 in current source code) checks the column status
and throws the exception:
Preconditions.checkState(columnsBitSet.get(idx), "Column %s is not
set", col.getName());
I think as the purpose of function appendCellValueDebugString is to append
debug information, this line may be inappropriate. But I’m not actually
familiar with kudu source code and not sure about it.
An issue has been created on Jira for it, I can try to set up a development
environment and work on it, but if anyone is interested, please go ahead and
fix it:
https://issues.apache.org/jira/browse/KUDU-1757 .
Best Regards,
Yanlong
From: Todd Lipcon [mailto:[email protected]]
Sent: 2016年11月24日 11:53
To: [email protected]
Subject: Re: Issue with newInsert() in scala
Hi Yanlong,
Interesting. It looks like the issue is not Scala, but the fact that you are
using a REPL (the Spark shell). The REPL tries to print the result of each
line, and in this case it's trying to stringify the insert before you've set
any columns on it.
We should probably fix the toString() method here so that it doesn't throw an
exception when all of the fields are unset. Any interest in working on a patch?
(or any other contributor who might be interested?)
-Todd
On Wed, Nov 23, 2016 at 7:32 PM, Zheng Yanlong 郑延龙(IS,SGM)
<[email protected]<mailto:[email protected]>> wrote:
Hi all,
I’m trying java kudu api in scala. I can create table and read records. But met
error when tried to insert data into kudu.
In spark-shell:
scala> import org.apache.kudu.ColumnSchema;
import org.apache.kudu.ColumnSchema
scala> import org.apache.kudu.Schema;
import org.apache.kudu.Schema
scala> import org.apache.kudu.Type;
import org.apache.kudu.Type
scala> import org.apache.kudu.client._
import org.apache.kudu.client._
scala> import java.util.ArrayList;
import java.util.ArrayList
scala> import java.util.List;
import java.util.List
scala> val kc = new KuduClient.KuduClientBuilder("XXX.XXX.XXX.XXX:7051").build()
kc: org.apache.kudu.client.KuduClient =
org.apache.kudu.client.KuduClient@2b852e9<mailto:org.apache.kudu.client.KuduClient@2b852e9>
scala> val ktbl = kc.openTable("kudu_scala_test")
ktbl: org.apache.kudu.client.KuduTable =
org.apache.kudu.client.KuduTable@7e26fb98<mailto:org.apache.kudu.client.KuduTable@7e26fb98>
scala> val ksession = kc.newSession()
ksession: org.apache.kudu.client.KuduSession =
org.apache.kudu.client.KuduSession@cc1f71e<mailto:org.apache.kudu.client.KuduSession@cc1f71e>
scala> val kins = ktbl.newInsert()
java.lang.IllegalStateException: Column id is not set <== id is the first
column in the table.
at
org.apache.kudu.client.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:197)
at org.apache.kudu.client.PartialRow.appendDebugString(PartialRow.java:559)
at org.apache.kudu.client.PartialRow.stringifyRowKey(PartialRow.java:532)
at org.apache.kudu.client.Operation.toString(Operation.java:202)
at
scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)
at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)
at .<init>(<console>:10)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1045)
at
org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1326)
at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:821)
at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:852)
at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:800)
at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)
at
org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)
at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:657)
at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:665)
at
org.apache.spark.repl.SparkILoop.org<http://org.apache.spark.repl.SparkILoop.org>$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:670)
at
org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:997)
at
org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
at
org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)
at
scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
at
org.apache.spark.repl.SparkILoop.org<http://org.apache.spark.repl.SparkILoop.org>$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)
at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1064)
at org.apache.spark.repl.Main$.main(Main.scala:31)
at org.apache.spark.repl.Main.main(Main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
I have tried the java sample, it worked well in java. Could you help with this
issue? Or any other ways to insert data in scala?
Please let me know if you need other information.
Best Regards,
Yanlong
*********************************************************************
Email Disclaimer
----------------
The information in this email is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this
email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying,
distribution or any action taken or omitted to be taken in reliance
on it, is prohibited and may be unlawful.
Any opinions or advice contained in this email are subject to the
terms and conditions expressed in the governing SGM client
engagement letter and should not be relied upon unless they are
confirmed in writing on SGM's letterhead.
*********************************************************************
--
Todd Lipcon
Software Engineer, Cloudera
*********************************************************************
Email Disclaimer
----------------
The information in this email is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this
email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying,
distribution or any action taken or omitted to be taken in reliance
on it, is prohibited and may be unlawful.
Any opinions or advice contained in this email are subject to the
terms and conditions expressed in the governing SGM client
engagement letter and should not be relied upon unless they are
confirmed in writing on SGM's letterhead.
*********************************************************************
