phanikumv commented on code in PR #69527:
URL: https://github.com/apache/airflow/pull/69527#discussion_r3534310388
##########
java-sdk/sdk/build.gradle.kts:
##########
@@ -150,14 +155,67 @@ abstract class GenerateDiscriminatorTask : DefaultTask() {
}
}
+abstract class SyncSupervisorSchemaTask : DefaultTask() {
+ @get:Input
+ abstract val schemaVersion: Property<String>
+
+ @get:Input
+ abstract val baseUrl: Property<String>
+
+ @get:Internal
+ abstract val schemaFile: RegularFileProperty
+
+ private fun apiVersionOf(file: File): String =
+ if (file.exists()) {
+ com.fasterxml.jackson.databind
+ .ObjectMapper()
+ .readTree(file)
+ .path("api_version")
+ .asText()
+ } else {
+ ""
+ }
+
+ @TaskAction
+ fun sync() {
+ val file = schemaFile.get().asFile
+ val version = schemaVersion.get()
+ if (apiVersionOf(file) == version) {
+ logger.lifecycle("Supervisor Schema is up-to-date
(api_version=$version).")
+ return
+ }
+ val url = "${baseUrl.get()}/$version.json"
+ logger.lifecycle("Refreshing Supervisor Schema with $url")
+ file.parentFile.mkdirs()
+ URI(url).toURL().openStream().use { input ->
+ Files.copy(input, file.toPath(),
StandardCopyOption.REPLACE_EXISTING)
+ }
+ val downloaded = apiVersionOf(file)
+ if (downloaded != version) {
+ throw GradleException(
+ "Downloaded schema declares api_version='$downloaded' but
expected '$version' ($url)",
+ )
+ }
Review Comment:
Let's download to a temp file, validate, then move it into place:
`openStream()` has no connect/read timeout, so a slow host hangs the build;
and `Files.copy(..., REPLACE_EXISTING)` streams onto `schema/schema.json` in
place, so a mid-stream
failure truncates it — and since `apiVersionOf` reads it with no
try/catch, the next run throws at the up-to-date check instead of
re-downloading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
