[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic closed this task as "Resolved". GoranSMilovanovic added a comment. Ok. In any case the fix to this script is easy if anything similar happens again. I will begin to monitor the Sqoop runs more closely. Thank you @elukey ! TASK DETAIL

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

elukey added a comment. I think we should be fine from now on, I wouldn't add more complexity to what we have :) TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey Cc: elukey,

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic added a comment. @elukey Thank you. I was thinking along the following lines: - if due to any updates, upgrades, or other changes, this turns out to be a persistent problem, - then is there a way to ask from some Analytics Client if `org.mariadb.jdbc.Driver` should

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

elukey added a comment. @GoranSMilovanovic sure! During the migration of the hosts where Hive Server/Metastore runs to Debian Buster, we encountered a lot of problems with the only available java lib for mysql, namely the one containing the `org.mariadb.jdbc.Driver` JDBC driver. We have now

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey, Maintenance_bot Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel,

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

gerritbot added a comment. Change 683801 **merged** by GoranSMilovanovic: [analytics/wmde/WD/WikidataAnalytics@master] T281316 https://gerrit.wikimedia.org/r/683801 TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: elukey, gerritbot Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek,

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

gerritbot added a comment. Change 683801 had a related patch set uploaded (by GoranSMilovanovic; author: GoranSMilovanovic): [analytics/wmde/WD/WikidataAnalytics@master] T281316 https://gerrit.wikimedia.org/r/683801 TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic added a comment. @WMDE-leszek @elukey I would like to learn from this. The following argument to `/usr/bin/sqoop` > --driver org.mariadb.jdbc.Driver seems to have been causing us trouble for some time already in WDCM_Sqoop_Clients.R

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic reopened this task as "Open". GoranSMilovanovic added a comment. @elukey Let's take a close look at this, if you agree. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To:

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

elukey closed this task as "Resolved". elukey claimed this task. elukey added a comment. No issues from our side, going to close, please reopen if necessary! TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic added a comment. @elukey No worries. Let me know if you need any external tests performed. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic Cc: elukey,

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

elukey added a comment. Nice! Let's keep it open since I want to understand if we need to use `--driver com.mysql.jdbc.Driver` or not, it will have some impact also for Analytics, thanks a lot for bringing this up and sorry for the trouble! TASK DETAIL

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic lowered the priority of this task from "High" to "Low". TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic Cc: elukey, Lydia_Pintscher, MisterSynergy, Aklapper,

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic added a comment. @elukey > As a quick workaround you should be able to unblock your queries just removing --driver org.mariadb.jdbc.Driver, can you try to see if it works? That would do, thank you Luca! I will keep the ticket open just in case, until the update

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic added a comment. @elukey Thank your for a prompt response, Luca! > The first error is unrelated, it is due to the fact that you have created the /tmp/wmde directory as analytics-privatedata (and by default permissions allow only user + group, not others) and your

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

elukey added a comment. Hi Goran! We moved the hive server nodes to Debian Buster recently (T231067 ) and we had a problem with Hive and Mariadb, highlights in https://issues.apache.org/jira/browse/HIVE-25020. As a quick workaround you should

[Wikidata-bugs] [Maniphest] T281316: WDCM_Sqoop_Clients.R fails from stat1004 (again)

GoranSMilovanovic triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T281316 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GoranSMilovanovic Cc: Lydia_Pintscher, MisterSynergy, Aklapper, Manuel, WMDE-leszek,