[
https://issues.apache.org/jira/browse/SPARK-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jai Kumar Singh updated SPARK-1452:
-----------------------------------
Description:
dynamic partition creation via shark QL command not working with cached table.
Though it works fine with non-cached tables.
Also static partition is working fine with cached table.
shark> desc sample;
OK
cid string None
host string None
url string None
bytes int None
pckts int None
app string None
cat string None
Time taken: 0.149 seconds
shark>
shark> desc sample_cached;
OK
cat string from deserializer
host string from deserializer
cid string None
# Partition Information
# col_name data_type comment
cid string None
Time taken: 0.15 seconds
shark>
shark> insert into table sample_cached partition(cid) select cat,host,cid from
sample;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
shark>
shark> insert into table sample_cached partition(cid="my-cid") select cat,host
from sample limit 20;
java.lang.InstantiationException: scala.Some
Continuing ...
java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
Continuing ...
Loading data to table default.sample_cached partition (cid=my-cid)
OK
Time taken: 64.268 seconds
I am logging this issue over here because
https://spark-project.atlassian.net/browse/SHARK/?selectedTab=com.atlassian.jira.jira-projects-plugin:issues-panel
not allowing me to log the issue there.
was:
dynamic partition creation via shark QL command not working with cached table.
Though it works fine with non-cached tables.
shark> desc sample;
OK
cid string None
host string None
url string None
bytes int None
pckts int None
app string None
cat string None
Time taken: 0.149 seconds
shark>
shark> desc sample_cached;
OK
cat string from deserializer
host string from deserializer
cid string None
# Partition Information
# col_name data_type comment
cid string None
Time taken: 0.15 seconds
shark>
shark> insert into table sample_cached partition(cid) select cat,host,cid from
sample;
FAILED: Hive Internal Error: java.lang.NullPointerException(null)
shark>
shark> insert into table sample_cached partition(cid="my-cid") select cat,host
from sample limit 20;
java.lang.InstantiationException: scala.Some
Continuing ...
java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
Continuing ...
Loading data to table default.sample_cached partition (cid=my-cid)
OK
Time taken: 64.268 seconds
> dynamic partition creation not working on cached table
> ------------------------------------------------------
>
> Key: SPARK-1452
> URL: https://issues.apache.org/jira/browse/SPARK-1452
> Project: Spark
> Issue Type: Bug
> Affects Versions: 0.9.0
> Environment: Shark git
> commit dfc0e81366c0e1d0293ecf9b490eeabcc2a9c904
> Merge: 517ebca 7652f0d
> Reporter: Jai Kumar Singh
> Labels: Shark
>
> dynamic partition creation via shark QL command not working with cached
> table. Though it works fine with non-cached tables.
> Also static partition is working fine with cached table.
> shark> desc sample;
> OK
> cid string None
> host string None
> url string None
> bytes int None
> pckts int None
> app string None
> cat string None
> Time taken: 0.149 seconds
> shark>
> shark> desc sample_cached;
> OK
> cat string from deserializer
> host string from deserializer
> cid string None
>
> # Partition Information
> # col_name data_type comment
>
> cid string None
> Time taken: 0.15 seconds
> shark>
> shark> insert into table sample_cached partition(cid) select cat,host,cid
> from sample;
> FAILED: Hive Internal Error: java.lang.NullPointerException(null)
> shark>
> shark> insert into table sample_cached partition(cid="my-cid") select
> cat,host from sample limit 20;
> java.lang.InstantiationException: scala.Some
> Continuing ...
> java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
> Continuing ...
> Loading data to table default.sample_cached partition (cid=my-cid)
> OK
> Time taken: 64.268 seconds
> I am logging this issue over here because
> https://spark-project.atlassian.net/browse/SHARK/?selectedTab=com.atlassian.jira.jira-projects-plugin:issues-panel
> not allowing me to log the issue there.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
