[
https://issues.apache.org/jira/browse/FLINK-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Wang updated FLINK-13838:
------------------------------
Description:
Currently we could use --yarnship to transfer jars, files and directory for
cluster and add them to classpath. However, compressed package could not be
supported. If we have a compressed package including some config files, so
files and jars, the --yarnshipArchives will be very useful.
What’s the difference between -yt and -yta?
'-yt' [file:///tmp|file:///tmp/a.tar.gz] Only directory could be supported. And
the file will be transferred by Yarn and keep the original compressed file(not
be unpacked) in the workdir of jobmanager/taskmanager container. If we have a
{{a.tar.gz}} under /tmp directory, it will be shipped in the original name and
format.
'-yta' [file:///tmp/a.tar.gz#dict1] A tar or zip file could be supported. The
file will be transferred by Yarn and unpacked to a new directory with name
dict1 in the workdir. If no new name is specified, a directory with name same
will be used(a directory named with a.tar.gz).
'-yta/--yarnshipArchives' <arg> Ship archives for cluster (t for
transfer), Use ',' to separate
multiple files. The archives could be
in local file system or distributed
file system. Use URI schema to specify
which file system the file belongs. If
schema is missing, would try to get
the archives in local file system. Use
'#' after the file path to specify a
new name in workdir. (eg: -yta
[file:///tmp/a.tar.gz#dict1,hdfs:///$na]
menode_address/tmp/b.tar.gz)
was:
Currently we could use --yarnship to transfer jars, files and directory for
cluster and add them to classpath. However, compressed package could not be
supported. If we have a compressed package including some config files, so
files and jars, the --yarnshipArchives will be very useful.
What’s the difference between -yt and -yta?
-yt [file:///tmp/a.tar.gz] The file will be transferred by Yarn and keep the
original compressed file(not be unpacked) in the workdir of
jobmanager/taskmanager container.
-yta [file:///tmp/a.tar.gz#dict1] The file will be transferred by Yarn and
unpacked to a new directory with name dict1 in the workdir.
-yta,--yarnshipArchives <arg> Ship archives for cluster (t for
transfer), Use ',' to separate
multiple files. The archives could be
in local file system or distributed
file system. Use URI schema to specify
which file system the file belongs. If
schema is missing, would try to get
the archives in local file system. Use
'#' after the file path to specify a
new name in workdir. (eg: -yta
file:///tmp/a.tar.gz#dict1,hdfs:///$na
menode_address/tmp/b.tar.gz)
> Support -yta(--yarnshipArchives) arguments in flink run command line
> --------------------------------------------------------------------
>
> Key: FLINK-13838
> URL: https://issues.apache.org/jira/browse/FLINK-13838
> Project: Flink
> Issue Type: New Feature
> Components: Command Line Client
> Reporter: Yang Wang
> Assignee: Yang Wang
> Priority: Major
>
> Currently we could use --yarnship to transfer jars, files and directory for
> cluster and add them to classpath. However, compressed package could not be
> supported. If we have a compressed package including some config files, so
> files and jars, the --yarnshipArchives will be very useful.
>
> What’s the difference between -yt and -yta?
> '-yt' [file:///tmp|file:///tmp/a.tar.gz] Only directory could be supported.
> And the file will be transferred by Yarn and keep the original compressed
> file(not be unpacked) in the workdir of jobmanager/taskmanager container. If
> we have a {{a.tar.gz}} under /tmp directory, it will be shipped in the
> original name and format.
> '-yta' [file:///tmp/a.tar.gz#dict1] A tar or zip file could be supported. The
> file will be transferred by Yarn and unpacked to a new directory with name
> dict1 in the workdir. If no new name is specified, a directory with name same
> will be used(a directory named with a.tar.gz).
>
> '-yta/--yarnshipArchives' <arg> Ship archives for cluster (t for
> transfer), Use ',' to separate
> multiple files. The archives could
> be
> in local file system or distributed
> file system. Use URI schema to
> specify
> which file system the file belongs.
> If
> schema is missing, would try to get
> the archives in local file system.
> Use
> '#' after the file path to specify a
> new name in workdir. (eg: -yta
>
> [file:///tmp/a.tar.gz#dict1,hdfs:///$na]
> menode_address/tmp/b.tar.gz)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
