hycdong commented on issue #1140: URL: https://github.com/apache/incubator-pegasus/issues/1140#issuecomment-1236776146
Thanks for your bug report~ > `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` ‘s order are not guaranteed. Function `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` are execute in thread pool `THREAD_POOL_META_STATE`, which is single thread in current pegasus design. As a result, the order can be guaranteed, `update_partition_metadata_on_remote_storage` will always execute before `update_partition_info_on_remote_storage`. However, in current bulk load design, bulk load will be failed if there are some partitions don't have data. For example, if you have a table will 16 partitions, and partition[0] doesn't have data to be ingested, bulk load will be failed. I suggest: 1. check your configuration whether threadpool THREAD_POOL_META_STATE is single thread 2. check if all partitions have data to be ingested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pegasus.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pegasus.apache.org For additional commands, e-mail: dev-h...@pegasus.apache.org
