Re: [I] [SUPPORT] Insert overwrite with replacement instant cannot execute archive [hudi]

[via GitHub]

xuzifu666 commented on issue #10873:
URL: https://github.com/apache/hudi/issues/10873#issuecomment-2002776170

   > @xuzifu666 When I tried the below code, it was properly archiving. Can you 
check below or provide your table/writer configurations.
   > 
   > ```
   > fake = Faker()
   > data = [{"transactionId": fake.uuid4(), "EventTime": "2014-01-01 
23:00:01","storeNbr" : "1",
   >          "FullName": fake.name(), "Address": fake.address(),
   >          "CompanyName": fake.company(), "JobTitle": fake.job(),
   >          "EmailAddress": fake.email(), "PhoneNumber": fake.phone_number(),
   >          "RandomText": fake.sentence(), "City": "US",
   >          "State": "NYC", "Country": "US"} for _ in range(5)]
   > 
   > hudi_options = {
   >     "hoodie.datasource.write.keygenerator.class": 
"org.apache.hudi.keygen.ComplexKeyGenerator",
   >     "hoodie.datasource.write.recordkey.field": 
"transactionId,storeNbr,EventTime",
   >     "hoodie.datasource.write.precombine.field": "Country",
   >     "hoodie.table.name": "huditransaction",
   >     "hoodie.datasource.write.operation" : "insert_overwrite",
   >     "hoodie.datasource.write.partitionpath.field" : "city"
   > }
   > 
   > pandas_df = pd.DataFrame(data)
   > df = spark.createDataFrame(pandas_df).withColumn("EventTime", 
expr("cast(EventTime as timestamp)"))
   > 
   > for i in range(1,100):
   >     
(df.write.format("hudi").options(**hudi_options).mode("append").save(PATH))
   > ```
   > 
   > <img alt="image" width="1694" 
src="https://private-user-images.githubusercontent.com/63430370/313497000-052419d1-2d9e-4d75-b588-fd1c887fee4f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTA3MzAwMjUsIm5iZiI6MTcxMDcyOTcyNSwicGF0aCI6Ii82MzQzMDM3MC8zMTM0OTcwMDAtMDUyNDE5ZDEtMmQ5ZS00ZDc1LWI1ODgtZmQxYzg4N2ZlZTRmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAzMTglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMzE4VDAyNDIwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWMwZWVkZDViYzdlNWI4ZGFkZmViYmM0ZGE3ODExOWQ5YjIzY2IxM2JkZDllOTFiMjc0MTJmZWY4MDM4ZWY2NGUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.eweItACh8rn1PMFa5pZKpbZ9_YIeUOuosA7QbClldOM";>
   
   @ad1happy2go Hi，you can set partition key 'city' different in 
range(1,100)，it would not archive，please try it again，thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org