Hi, I was reading https://www.cloudera.com/documentation/enterprise/5-11-x/topics/impala_partitioning.html#partition_stats And noticed this warning "If this metadata for all tables combined exceeds 2 GB, you might experience service downtime.”
But I thought that this limit applies for single table. Having large number of partitions per table can be an issue because each metadata operation for this table would require the whole metadata be updated (as far as I understand Impala doesn’t update partially the metadata for only the changed partitions) Also because of Java serialization limitation where you cannot serialize it to more than 1G or 2G I am guessing this is related to https://issues.apache.org/jira/browse/IMPALA-5058 - but that’s only if you do frequent DDL operations I suppose. Am I understanding things so far correctly. So why can there be service downtime in this case ? Thanks, Antoni
