Since this error is happening in your IDE, I would recommend using the
IntelliJ debugger to follow the filesystem initialization process and see
where it fails to pick up the credentials.
On Fri, Mar 12, 2021 at 11:11 PM sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> Same error.
Same error.
On Fri, 12 Mar 2021 at 09:01, ChesnaSchepler wrote:
> From the exception I would conclude that your core-site.xml file is not
> being picked up.
>
> AFAIK fs.hdfs.hadoopconf only works for HDFS, not for S3 filesystems, so
> try setting HADOOP_CONF_DIR to the directory that the file
From the exception I would conclude that your core-site.xml file is not
being picked up.
AFAIK fs.hdfs.hadoopconf only works for HDFS, not for S3 filesystems, so
try setting HADOOP_CONF_DIR to the directory that the file resides in.
On 3/12/2021 5:10 PM, sri hari kali charan Tummala wrote:
If
If anyone working have flink version 1.8.1 code reading S3 in Intellij in
public GitHub please pass it on that will be huge help.
Thanks
Sri
On Fri, 12 Mar 2021 at 08:08, sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> Which I already did in my pin still its not working.
>
> Tha
Which I already did in my pin still its not working.
Thanks
Sri
On Fri, 12 Mar 2021 at 06:18, Chesnay Schepler wrote:
> The concept of plugins does not exist in 1.8.1. As a result it should be
> sufficient for your use-case to add a dependency on flink-s3-fs-hadoop to
> your project.
>
> On 3/1
The concept of plugins does not exist in 1.8.1. As a result it should be
sufficient for your use-case to add a dependency on flink-s3-fs-hadoop
to your project.
On 3/12/2021 4:33 AM, sri hari kali charan Tummala wrote:
Let's close this issue guys please answer my questions. I am using
Flink 1.
Let's close this issue guys please answer my questions. I am using Flink
1.8.1.
Thanks
Sri
On Wed, 10 Mar 2021 at 13:25, sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> Also I don't see ConfigConstants.ENV_FLINK_PLUGINS_DIR I only see
> ConfigConstants.ENV_FLINK_LIB_DIR will this
Also I don't see ConfigConstants.ENV_FLINK_PLUGINS_DIR I only see
ConfigConstants.ENV_FLINK_LIB_DIR will this work ?
Thanks
Sri
On Wed, Mar 10, 2021 at 1:23 PM sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> I am not getting what you both are talking about lets be clear.
>
> Plug
I am not getting what you both are talking about lets be clear.
Plugin ? what is it ? Is it a Jar which I have to download from the
Internet and place it in a folder ? Is this the Jar which I have to
download ? (flink-s3-fs-hadoop) ?
Will this belo solution work ?
https://stackoverflow.com/questi
Well, you could do this before running the job:
// set the ConfigConstants.ENV_FLINK_PLUGINS_DIR environment variable,
pointing to a directory containing the plugins
PluginManager pluginManager =
PluginUtils.createPluginManagerFromRootFolder(new Configuration());
Filesystem.initialize(new Co
Hi.
I had the same problem. Flink use a plugins to access s3. When you run local it
starts a mini cluster and the mini cluster don’t load plugins. So it’s not
possible without modifying Flink. In my case I wanted to investigate save
points through Flink processor API and the workaround was to
Flink,
I am able to access Kinesis from Intellij but not S3 I have edited my stack
overflow question with kinesis code , Flink is still having issues reading
S3.
https://stackoverflow.com/questions/66536868/flink-aws-s3-access-issue-intellij-idea?noredirect=1#comment117656862_66536868
Thanks
Sr
my stack overflow question.
https://stackoverflow.com/questions/66536868/flink-aws-s3-access-issue-intellij-idea?noredirect=1#comment117626682_66536868
On Tue, Mar 9, 2021 at 11:28 AM sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> Here is my Intellij question.
>
>
> https://stac
Here is my Intellij question.
https://stackoverflow.com/questions/66536868/flink-aws-s3-access-issue-intellij-idea?noredirect=1#comment117626682_66536868
On Mon, Mar 8, 2021 at 11:22 AM sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
>
> Hi Flink Experts,
>>
>
> I am trying to read
> Hi Flink Experts,
>
I am trying to read an S3 file from my Intellij using Flink I am.comimg
> across Aws Auth error can someone help below are all the details.
>
> I have Aws credentials in homefolder/.aws/credentials
>
My Intellij Environment Variables:-
> ENABLE_BUILT_IN_PLUGINS=flink-s3-fs
15 matches
Mail list logo
