Hi Sinan, I recommend you to destroy the server which has been hacked. If someone wrote a mining program in it, maybe they've persisted some malicious code as well (rootkit, etc)
Definetly, If you detect a database node compromised you must destroy it and load a backup. On Thu, Dec 21, 2017 at 6:27 PM, Sinan Gabel <[email protected]> wrote: > PS Just want to say that by removing the cron job as described the crypto > mining problem is removed and the 100% CPU problem is resolved. (I have had > responses from other users experiencing the exact same problem.) > > $ sudo -i > $ su couchdb > $ crontab -e > > Then remove the line in the cron. If you use the nano editor the command > are "ctrl-k" to remove lines, and "ctrl-o" to write the file, and "ctrl-x" > to exit the crontab editing. And thus the problem should be resolved. Else > just write to me. > > > On 11 December 2017 at 17:49, Sinan Gabel <[email protected]> wrote: > >> I just found this, it may solve the problem by removing it. It is a >> script that starts the crypto mining program as user couchdb. >> >> [image: Inline images 1] >> >> On 11 December 2017 at 15:04, Sinan Gabel <[email protected]> wrote: >> >>> >>> >>> I have now checked it, and it is as you @Robert perceived, a crypto >>> mining program. >>> >>> It is a program "fs-manager" with config.json as given below that is >>> restarted every 3 hours or so (8:41, 11:41 etc.) unless it keeps running. >>> >>> I updated couchdb to latest version and changed all admin passwords but >>> that has not solved it, so it hidden somewhere on the server. >>> >>> Anyone have a clue to how to remove this hack completely? >>> >>> >>> >>> $ sudo find / -name "fs-manager" >>> >>> => >>> >>> /var/tmp/.X1M-Unix/fs-manager >>> >>> >>> $ ls -al >>> >>> => >>> >>> >>> $ less config.json >>> >>> => >>> >>> { "algo": "cryptonight", "av": 0, "background": true, "colors": false, >>> "cpu-affinity": null, "cpu-priority": null, "donate-level": 2, "log-file": >>> "xmrig.log", "max-cpu-usage": 85, "print-time": 60, "retries": 2, >>> "retry-pause": 3, "safe": false, "syslog": false, "threads": null, "pools": >>> [ { "url": "pool-proxy.com:8080", "user": "user", "pass": "x", >>> "keepalive": true, "nicehash": false } ]} >>> >>> >>> >>> >>> On 10 December 2017 at 18:11, Robert Samuel Newson <[email protected]> >>> wrote: >>> >>>> fs-manager is not part of couchdb. You should check that you've not >>>> been hacked. See https://justi.cz/security/2017 >>>> /11/14/couchdb-rce-npm.html <https://justi.cz/security/201 >>>> 7/11/14/couchdb-rce-npm.html>. >>>> >>>> I've seen another case where a user's couchdb installation was >>>> compromised and a bitcoin mining tool installed. This would (obviously) use >>>> all your cpu, and I think fs-manager was part of that. >>>> >>>> B. >>>> >>>> > On 10 Dec 2017, at 17:07, Sinan Gabel <[email protected]> wrote: >>>> > >>>> > Same problem with latest couchdb clustered version. What is the >>>> process " >>>> > */.fs-manager*" doing? >>>> > >>>> > Googling it returned something about a python package called >>>> "fs-manager"!? >>>> > >>>> > For now I have killed the *couchdb fs-manager* process, and couchdb >>>> still >>>> > seems to be working fine. >>>> > >>>> > On 10 December 2017 at 16:50, Sinan Gabel <[email protected]> >>>> wrote: >>>> > >>>> >> PS I have just now upgraded to latest couchdb clustered version, I >>>> will >>>> >> see if that solves the 100% CPU usage problem. >>>> >> >>>> >> On 10 December 2017 at 15:28, Sinan Gabel <[email protected]> >>>> wrote: >>>> >> >>>> >>> Hi, >>>> >>> >>>> >>> I do not have a solution but also experiencing the 100% CPU usage. >>>> Here's >>>> >>> a .png screen shot of the processes running (my case): >>>> >>> >>>> >>> >>>> >>> On 10 December 2017 at 02:00, Geoffrey Cox <[email protected]> >>>> wrote: >>>> >>> >>>> >>>> Well, I'm back at this and here is the latest info and I think it >>>> may be >>>> >>>> related to writes and the _global_changes database: >>>> >>>> >>>> >>>> 1. I run my production env and one of my nodes becomes the >>>> "workhorse" >>>> >>>> node with 100% CPU >>>> >>>> 2. I stop all my production code from generating any more CouchDB >>>> >>>> requests and eventually the workhorse node goes back to 0% CPU >>>> >>>> 3. I can then issue writes on a single database (really any >>>> database >>>> >>>> and >>>> >>>> ANY node--not just the workhorse node) and the workhorse node >>>> will >>>> >>>> kick >>>> >>>> back up to 100% CPU. If I stop the writes, the workhorse node >>>> will >>>> >>>> return >>>> >>>> to 0% CPU. >>>> >>>> 4. And now the punch line: if I delete the _global_changes >>>> database, >>>> >>>> the >>>> >>>> CPU drops down to 0% even if I am issuing writes! Pure cray cray >>>> >>>> >>>> >>>> Any thoughts? >>>> >>>> >>>> >>>> (Sorry, still working on a reproducible env for everyone) >>>> >>>> >>>> >>>> On Wed, Dec 6, 2017 at 6:56 AM Geoffrey Cox <[email protected]> >>>> wrote: >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>>> >> >>>> >>>> >>> >> > -- [image: Cabify - Your private Driver] <http://www.cabify.com/> *Victor Torre* DevOps Madrid, Spain [email protected] [image: Facebook] <http://cbify.com/fb_ES>[image: Twitter] <http://cbify.com/tw_ES>[image: Instagram] <http://cbify.com/in_ES> -- Este mensaje y cualquier archivo adjunto va dirigido exclusivamente a su destinatario, pudiendo contener información confidencial sometida a secreto profesional. No está permitida su reproducción o distribución sin la autorización expresa de Cabify. Si usted no es el destinatario final por favor elimínelo e infórmenos por esta vía. This message and any attached file are intended exclusively for the addressee, and it may be confidential. You are not allowed to copy or disclose it without Cabify's prior written authorization. If you are not the intended recipient please delete it from your system and notify us by e-mail.
