overview.json is a generated file that is placed in the local directory
controlled by /historyserver.web.tmpdir/.
Have you configured this option to point to some non-local filesystem?
(Or if not, is the java.io.tmpdir property pointing somewhere funny?)
On 24/04/2020 18:24, Hailu, Andreas wrote:
I’m having a further look at the code in
HistoryServerStaticFileServerHandler - is there an assumption about
where overview.json is supposed to be located?
*// *ah**
*From:*Hailu, Andreas [Engineering]
*Sent:* Wednesday, April 22, 2020 1:32 PM
*To:* 'Chesnay Schepler' <ches...@apache.org>; Hailu, Andreas
[Engineering] <andreas.ha...@ny.email.gs.com>; user@flink.apache.org
*Subject:* RE: History Server Not Showing Any Jobs - File Not Found?
Hi Chesnay, thanks for responding. We’re using Flink 1.9.1. I enabled
DEBUG level logging and this is something relevant I see:
2020-04-22 13:25:52,566 [Flink-HistoryServer-ArchiveFetcher-thread-1]
DEBUG DFSInputStream - Connecting to datanode 10.79.252.101:1019
2020-04-22 13:25:52,567 [Flink-HistoryServer-ArchiveFetcher-thread-1]
DEBUG SaslDataTransferClient - SASL encryption trust check:
localHostTrusted = false, remoteHostTrusted = false
2020-04-22 13:25:52,567 [Flink-HistoryServer-ArchiveFetcher-thread-1]
DEBUG SaslDataTransferClient - SASL client skipping handshake in
secured configuration with privileged port for addr = /10.79.252.101,
datanodeId = DatanodeI
nfoWithStorage[10.79.252.101:1019,DS-7f4ec55d-7c5f-4a0e-b817-d9e635480b21,DISK]
*2020-04-22 13:25:52,571 [Flink-HistoryServer-ArchiveFetcher-thread-1]
DEBUG DFSInputStream - DFSInputStream has been closed already*
*2020-04-22 13:25:52,573 [nioEventLoopGroup-3-6] DEBUG
HistoryServerStaticFileServerHandler - Unable to load requested file
/jobs/overview.json from classloader*
2020-04-22 13:25:52,576 [IPC Parameter Sending Thread #0] DEBUG
Client$Connection$3 - IPC Client (1578587450) connection to
d279536-002.dc.gs.com/10.59.61.87:8020 from d...@gs.com
<mailto:d...@gs.com> sending #1391
Aside from that, it looks like a lot of logging around datanodes and
block location metadata. Did I miss something in my classpath,
perhaps? If so, do you have a suggestion on what I could try?
*// *ah**
*From:*Chesnay Schepler <ches...@apache.org <mailto:ches...@apache.org>>
*Sent:* Wednesday, April 22, 2020 2:16 AM
*To:* Hailu, Andreas [Engineering] <andreas.ha...@ny.email.gs.com
<mailto:andreas.ha...@ny.email.gs.com>>; user@flink.apache.org
<mailto:user@flink.apache.org>
*Subject:* Re: History Server Not Showing Any Jobs - File Not Found?
Which Flink version are you using?
Have you checked the history server logs after enabling debug logging?
On 21/04/2020 17:16, Hailu, Andreas [Engineering] wrote:
Hi,
I’m trying to set up the History Server, but none of my
applications are showing up in the Web UI. Looking at the console,
I see that all of the calls to /overview return the following 404
response: {"errors":["File not found."]}.
I’ve set up my configuration as follows:
JobManager Archive directory:
*jobmanager.archive.fs.dir*:
hdfs:///user/p2epda/lake/delp_qa/flink_hs/
-bash-4.1$ hdfs dfs -ls /user/p2epda/lake/delp_qa/flink_hs
Found 44282 items
-rw-r----- 3 delp datalake_admin_dev 50569 2020-03-21 23:17
/user/p2epda/lake/delp_qa/flink_hs/000144dba9dc0f235768a46b2f26e936
-rw-r----- 3 delp datalake_admin_dev 49578 2020-03-03 08:45
/user/p2epda/lake/delp_qa/flink_hs/000347625d8128ee3fd0b672018e38a5
-rw-r----- 3 delp datalake_admin_dev 50842 2020-03-24 15:19
/user/p2epda/lake/delp_qa/flink_hs/0004be6ce01ba9677d1eb619ad0fa757
...
...
History Server will fetch the archived jobs from the same location:
*historyserver.archive.fs.dir*:
hdfs:///user/p2epda/lake/delp_qa/flink_hs/
So I’m able to confirm that there are indeed archived applications
that I should be able to view in the histserver. I’m not able to
find out what file the overview service is looking for from the
repository – any suggestions as to what I could look into next?
Best,
Andreas
------------------------------------------------------------------------
Your Personal Data: We may collect and process information about
you that may be subject to data protection laws. For more
information about how we use and disclose your personal data, how
we protect your information, our legal basis to use your
information, your rights and who you can contact, please refer to:
www.gs.com/privacy-notices <http://www.gs.com/privacy-notices>
------------------------------------------------------------------------
Your Personal Data: We may collect and process information about you
that may be subject to data protection laws. For more information
about how we use and disclose your personal data, how we protect your
information, our legal basis to use your information, your rights and
who you can contact, please refer to: www.gs.com/privacy-notices
<http://www.gs.com/privacy-notices>
