Hi,
Have you checked your JDBC connections from Spark to Oracle. What is Oracle
saying? Is it doing anything or hanging?
set pagesize 9999
set linesize 140
set heading off
select SUBSTR(name,1,8) || ' sessions as on '||TO_CHAR(CURRENT_DATE, 'MON
DD YYYY HH:MI AM') from v$database;
set heading on
column spid heading "OS PID" format a6
column process format a13 heading "Client ProcID"
column username format a15
column sid format 999
column serial# format 99999
column STATUS format a3 HEADING 'ACT'
column last format 9,999.99
column TotGets format 999,999,999,999 HEADING 'Logical I/O'
column phyRds format 999,999,999 HEADING 'Physical I/O'
column total_memory format 999,999,999 HEADING 'MEM/KB'
--
SELECT
substr(a.username,1,15) "LOGIN"
, substr(a.sid,1,5) || ','||substr(a.serial#,1,5) AS "SID/serial#"
, TO_CHAR(a.logon_time, 'DD/MM HH:MI') "LOGGED IN SINCE"
, substr(a.machine,1,10) HOST
, substr(p.username,1,8)||'/'||substr(p.spid,1,5) "OS PID"
, substr(a.osuser,1,8)||'/'||substr(a.process,1,5) "Client PID"
, substr(a.program,1,15) PROGRAM
--,ROUND((CURRENT_DATE-a.logon_time)*24) AS "Logged/Hours"
, (
select round(sum(ss.value)/1024) from v$sesstat ss,
v$statname sn
where ss.sid = a.sid and
sn.statistic# = ss.statistic# and
-- sn.name in ('session pga memory')
sn.name in ('session pga memory','session uga
memory')
) AS total_memory
, (b.block_gets + b.consistent_gets) TotGets
, b.physical_reads phyRds
, decode(a.status, 'ACTIVE', 'Y','INACTIVE', 'N') STATUS
, CASE WHEN a.sid in (select sid from v$mystat where rownum = 1)
THEN '<-- YOU' ELSE ' ' END "INFO"
FROM
v$process p
,v$session a
,v$sess_io b
WHERE
a.paddr = p.addr
AND p.background IS NULL
--AND a.sid NOT IN (select sid from v$mystat where rownum = 1)
AND a.sid = b.sid
AND a.username is not null
--AND (a.last_call_et < 3600 or a.status = 'ACTIVE')
--AND CURRENT_DATE - logon_time > 0
--AND a.sid NOT IN ( select sid from v$mystat where rownum=1) -- exclude me
--AND (b.block_gets + b.consistent_gets) > 0
ORDER BY a.username;
exit
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Fri, 10 Apr 2020 at 17:37, Ruijing Li <liruijin...@gmail.com> wrote:
> Hi all,
>
> I am on spark 2.4.4 and using scala 2.11.12, and running cluster mode on
> mesos. I am ingesting from an oracle database using spark.read.jdbc. I am
> seeing a strange issue where spark just hangs and does nothing, not
> starting any new tasks. Normally this job finishes in 30 stages but
> sometimes it stops at 29 completed stages and doesn’t start the last stage.
> The spark job is idling and there is no pending or active task. What could
> be the problem? Thanks.
> --
> Cheers,
> Ruijing Li
>
