Kingsley Chen created CARBONDATA-1168:
-----------------------------------------
Summary: Driver Delete data operation is failed due to failure in
creating delete delta file for segment
Key: CARBONDATA-1168
URL: https://issues.apache.org/jira/browse/CARBONDATA-1168
Project: CarbonData
Issue Type: Bug
Components: sql
Affects Versions: 1.1.0
Environment: spark1.6+carbon1.1.0
we have a 20 nodes cluster and 32GB ram each
Reporter: Kingsley Chen
Fix For: NONE
We use spark code to delete data from table as below
------------------spark code----------------------
val deleteSql = s"DELETE FROM $tableName WHERE $rowkeyName IN
(${rowKeyVals.mkString(",")})"
cc.sql(deleteSql).show()
------------------spark code----------------------
when the array size of rowKeyVals is larger than 200, the delete operation will
be failed, and print the log:
Delete data request has been received for default.item
Delete data operation is failed for default.item
Driver Delete data operation is failed due to failure in creating delete delta
file for segment : null block : null
++
||
++
++
That is to say, it only delete success maximun at 200 a batch,and took about
1min which is
too slow. So my question is how to tuning the performance to make the batch
larger
and delete faster
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
