Claus Ibsen created CAMEL-26923:
-----------------------------------

             Summary: camel-duckdb - copy loads one local file unless options 
allow URLs or globs
                 Key: CAMEL-26923
                 URL: https://issues.apache.org/jira/browse/CAMEL-26923
             Project: Camel
          Issue Type: Improvement
            Reporter: Claus Ibsen
            Assignee: Claus Ibsen
             Fix For: 4.24.0, 4.22.3


The {{copy}} operation of the camel-duckdb producer takes the source file from 
the message body and loads it into a table with DuckDB's {{read_csv}}, 
{{read_parquet}} or {{read_json}}.

h3. Behaviour today

{{DuckDbProducer.buildCopySql}} passes the resolved body value to the read 
function as it is. DuckDB interprets that value itself: a value with {{*}}, 
{{?}}, {{[}} or {{\{}} is expanded as a glob pattern over many files, and a 
value with a {{scheme://}} prefix (http, https, s3, ...) is read as a remote 
source. The route has no way to say that {{copy}} should load one local file, 
or only files from a given directory.

h3. Behaviour after the change

By default {{copy}} loads a single local file. Remote sources and glob patterns 
become explicit opt-ins, and a base directory can restrict where the source may 
be:

* {{allowUrlCopySource}} (default {{false}}): when {{false}}, a source that 
starts with a URL scheme ({{^[A-Za-z][A-Za-z0-9+.-]*://}}) is refused.
* {{allowGlobCopySource}} (default {{false}}): when {{false}}, a source that 
contains glob characters ({{*}}, {{?}}, {{[}}, {{\{}}) is refused. This applies 
to String bodies as well as {{File}} and {{Path}} bodies.
* {{copyBaseDirectory}} (unset by default): when set, a local source must 
resolve inside this directory, with symbolic links resolved 
({{Path.toRealPath}}). For an allowed glob, the fixed part before the first 
glob character is checked.

A refused source fails the exchange with an {{IllegalArgumentException}} whose 
message names the option to set.

h3. Classes and code changes

* {{DuckDbProducer}}: a check in {{doCopy}} / {{resolveFilePath}}, before 
{{buildCopySql}}, for the URL scheme, glob characters and base directory 
containment.
* {{DuckDbEndpoint}}: the three new {{@UriParam}} options (label {{producer}}), 
{{allowUrlCopySource}} marked as {{insecure:dev}}.
* {{duckdb-component.adoc}}: document that {{copy}} loads one local file, the 
new options, and optionally DuckDB's own settings ({{enable_external_access}}, 
{{allowed_directories}}, {{autoinstall_known_extensions}}) for deployments that 
want DuckDB to enforce the same limits for all operations.
* Regenerated files: component JSON, endpoint configurer and URI factory, 
endpoint DSL builder, catalog JSON and doc copy.
* Upgrade guide: 4.24 entry (and a 4.22.3 entry if backported) stating that 
{{copy}} no longer accepts a URL or glob pattern unless {{allowUrlCopySource}} 
/ {{allowGlobCopySource}} is set.

h3. Tests

In {{DuckDbProducerTest}}, without network access:
* a URL source is refused by default;
* a glob pattern is refused by default, and loads all matching files (two CSV 
files in a temporary directory) when {{allowGlobCopySource=true}};
* a {{File}} / {{Path}} body whose name contains glob characters is refused by 
default;
* with {{copyBaseDirectory}}: a file inside the directory loads; an absolute 
path outside it, a {{../}} path and a symbolic link pointing outside it are 
refused;
* the existing {{copyFromCsvFile}}, {{copyFromPathBody}} and 
{{copyFromBytesBody}} tests keep passing.

The camel-4.22.x branch has the component without the {{Path}} / {{byte[]}} 
body support, so a backport there covers {{String}} and {{File}} bodies and 
regenerates the endpoint files on the branch.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to