simicd commented on issue #627:
URL: https://github.com/apache/arrow-ballista/issues/627#issuecomment-1911128842
Hi all, while working on #957 I faced the same issue, here's how to
reproduce it:
1. In the API handler, `get_job_svg_graph` function replace `.map_err(|_|
warp::reject())` with `.map_err(|err| { info!("Issue: {:?}", err);
warp::reject()})`
https://github.com/apache/arrow-ballista/blob/9903ab27f121f16717a70de7a30f643c4c45dd34/ballista/scheduler/src/api/handlers.rs#L343-L348
2. Launch scheduler and executor
3. Submit a query, e.g. the `standalone-sql.rs` but with a call to remote:
```rust
use ballista::prelude::{BallistaConfig, BallistaContext, Result};
use ballista_examples::test_util;
use datafusion::execution::options::ParquetReadOptions;
#[tokio::main]
async fn main() -> Result<()> {
let config = BallistaConfig::builder()
.set("ballista.shuffle.partitions", "1")
.build()?;
let ctx = BallistaContext::remote("localhost", 50050,
&config).await?;
let testdata = test_util::examples_test_data();
// register parquet file with the execution context
ctx.register_parquet(
"test",
&format!("{testdata}/alltypes_plain.parquet"),
ParquetReadOptions::default(),
)
.await?;
let df = ctx.sql("select count(1) from test").await?;
df.show().await?;
Ok(())
}
```
4. Wait until query is executed
5. Look up the job ID, e.g. GET request to `http://localhost:50050/api/jobs`
6. Call `http://localhost:50050/api/job/{job_id}/dot_svg` with the retrieved
job_id (e.g. `http://localhost:50050/api/job/bvppZ4r/dot_svg`)
7. Go to the terminal of the scheduler and monitor the logs - I got the
following error:
```
2024-01-25T22:28:24.800607Z INFO tokio-runtime-worker ThreadId(19)
ballista_scheduler::api::handlers: Issue: Error { kind: NotFound, message:
"program not found" }
```
Hope that helps to narrow it down!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
