Hi team,
Thank you for your response. Could you please provide a sample
regex(source.path.regex-pattern) for the following scenarios:
Matching filenames that start with "flink" Eg : flink_2023_11_08.csv
Matching filenames that end with "flink.csv" Eg:
customer_2023_11_08_flink.csv
Thanks and rega
Hi Arjun,
As stated in the document, 'This regex pattern should be matched with the
absolute file path.'
Therefore, you should adjust your regular expression to match absolute paths.
Please let me know if there are any other problems.
Best,
Yu Chen
> 2023年11月7日 18:11，arjun s 写道：
>
> Hi Chen,
Hi Chen,
I attempted to configure the 'source.path.regex-pattern' property in the
table settings as '^customer.*' to ensure that the Flink job only processes
file names starting with "customer" in the specified directory. However, it
appears that this configuration is not producing the expected res
> unpredictable file schema(Table API) in the source directory
You'll probably have to write some logic that helps predict the schema :)
Are there actual schemas for the CSV files somewhere? JSONSchema or
something of the like?At Wikimedia we use JSONSchema (not with CSV
data, but it could
Thanks for your response.
How should we address the issue of dealing with the unpredictable file
schema(Table API) in the source directory, as I previously mentioned in my
email?
Thanks and regards,
Arjun
On Mon, 6 Nov 2023 at 20:56, Chen Yu wrote:
> Hi Arjun,
>
> If you can filter files by a
