richt richt created HIVE-24128:
----------------------------------
Summary: transactions cannot recognize bucket file
Key: HIVE-24128
URL: https://issues.apache.org/jira/browse/HIVE-24128
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 3.1.1
Reporter: richt richt
* Error while compiling statement: FAILED: SemanticException [Error 10141]:
Bucketed table metadata is not correct. Fix the metadata or don't use bucketed
mapjoin, by setting hive.enforce.bucketmapjoin to false. The number of buckets
for table dcp partition ods_load_date=20200701 is 2, whereas the number of
files is 1
* the trnasaaction table manage file like below
{code:java}
-rw-r--r-- 3 hadoop supergroup 1 2020-09-08 10:20
/usr/hive/warehouse/test_etl_dwd.db/dcp/ods_load_date=20200701/_orc_acid_version
drwxr-xr-x - hadoop supergroup 0 2020-09-07 19:28
/usr/hive/warehouse/test_etl_dwd.db/dcp/ods_load_date=20200701/base_0000021
-rw-r--r-- 3 hadoop supergroup 15401449 2020-09-08 10:20
/usr/hive/warehouse/test_etl_dwd.db/dcp/ods_load_date=20200701/bucket_00000
-rw-r--r-- 3 hadoop supergroup 15408471 2020-09-08 10:20
/usr/hive/warehouse/test_etl_dwd.db/dcp/ods_load_date=20200701/bucket_00001
{code}
* it put the bucket file into. dir base**,
* but when I use merge the table , hive will check the bucket file number ,
*
{code:java}
public static List<String> getBucketFilePathsOfPartition( Path location,
ParseContext pGraphContext) throws SemanticException { List<String>
fileNames = new ArrayList<String>(); try { FileSystem fs =
location.getFileSystem(pGraphContext.getConf()); FileStatus[] files =
fs.listStatus(new Path(location.toString()),
FileUtils.HIDDEN_FILES_PATH_FILTER); if (files != null) { for
(FileStatus file : files) { fileNames.add(file.getPath().toString());
} } } catch (IOException e) { throw new
SemanticException(e); } return fileNames; }
{code}
it only. check the file under the partition , not check the base** directory
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
