Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
damccorm commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4693821281 @kennknowles could you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4501099840 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewers` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
damccorm commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4501082360 R: @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4478204338 Reminder, please take a look at this pr: @Abacn @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4421058593 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4388181460 Reminder, please take a look at this pr: @ahmedabu98 @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4335420045 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @ahmedabu98 for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4313193787 Reminder, please take a look at this pr: @chamikaramj @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4260112176 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @chamikaramj for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4243916555 Reminder, please take a look at this pr: @Abacn @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4198984517 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4183264480 Reminder, please take a look at this pr: @chamikaramj @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4134340873 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @chamikaramj for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4117874278 Reminder, please take a look at this pr: @Abacn @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4074616739 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4054736956 Reminder, please take a look at this pr: @ahmedabu98 @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-4004703929 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @ahmedabu98 for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3990678299 Reminder, please take a look at this pr: @chamikaramj @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3944474406 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @chamikaramj for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3926950300 Reminder, please take a look at this pr: @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3884125740 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
ahmedabu98 commented on code in PR #36296:
URL: https://github.com/apache/beam/pull/36296#discussion_r2774960466
##
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java:
##
@@ -649,20 +632,30 @@ static void verifyDatasetPresence(DatasetService
datasetService, TableReference
}
static String getDatasetLocation(
- DatasetService datasetService, String projectId, String datasetId) {
-Dataset dataset;
+ DatasetService datasetService, String projectId, String datasetId)
+ throws IOException, InterruptedException {
Review Comment:
nit: why can't we throw a RuntimeException instead? and avoid having to
try-catch each time we call `getDatasetLocation` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
ahmedabu98 commented on code in PR #36296:
URL: https://github.com/apache/beam/pull/36296#discussion_r2774960466
##
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java:
##
@@ -649,20 +632,30 @@ static void verifyDatasetPresence(DatasetService
datasetService, TableReference
}
static String getDatasetLocation(
- DatasetService datasetService, String projectId, String datasetId) {
-Dataset dataset;
+ DatasetService datasetService, String projectId, String datasetId)
+ throws IOException, InterruptedException {
Review Comment:
nit: why can't we throw a RuntimeException instead? and avoid having to
try-catch each time we call `getDatasetLocation` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3853325358 Reminder, please take a look at this pr: @ahmedabu98 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3817314707 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @ahmedabu98 for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3804907967 Reminder, please take a look at this pr: @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3768059643 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3754454058 Reminder, please take a look at this pr: @ahmedabu98 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3718608313 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @ahmedabu98 for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3703637751 Reminder, please take a look at this pr: @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
Suvrat1629 commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3691072494 @kennknowles I have addressed your comments and changed some of the pr logic. I went through some of the code and realised that the `IOException` being wrapped in the `RuntimeException` was causing the infinite retries(according to my understanding), so I introduced a new typed exception that explicitly specified that the retries had exhausted and then threw a `IllegalStateException` wrapped in a `RuntimeException`. I think the logs and the `IllegalStateException` should act as a terminal error for dataflow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
Suvrat1629 commented on code in PR #36296:
URL: https://github.com/apache/beam/pull/36296#discussion_r2646648401
##
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java:
##
@@ -632,20 +632,15 @@ static void verifyDatasetPresence(DatasetService
datasetService, TableReference
}
static String getDatasetLocation(
- DatasetService datasetService, String projectId, String datasetId) {
-Dataset dataset;
+ DatasetService datasetService, String projectId, String datasetId)
+ throws IOException, InterruptedException {
try {
- dataset = datasetService.getDataset(projectId, datasetId);
-} catch (Exception e) {
- if (e instanceof InterruptedException) {
-Thread.currentThread().interrupt();
- }
- throw new RuntimeException(
- String.format(
- "unable to obtain dataset for dataset %s in project %s",
datasetId, projectId),
- e);
+ Dataset dataset = datasetService.getDataset(projectId, datasetId);
+ return dataset.getLocation();
+} catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
Suvrat1629 commented on code in PR #36296:
URL: https://github.com/apache/beam/pull/36296#discussion_r2646648206
##
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteRename.java:
##
@@ -329,14 +331,39 @@ private BigQueryHelpers.PendingJob startCopy(
new EncryptionConfiguration().setKmsKeyName(kmsKey));
}
- String bqLocation =
- BigQueryHelpers.getDatasetLocation(
- datasetService, ref.getProjectId(), ref.getDatasetId());
+ String bqLocation;
+ try {
Review Comment:
Abstracted the logic out into `BigQueryHelper.getDatasetLocation`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
Suvrat1629 commented on code in PR #36296: URL: https://github.com/apache/beam/pull/36296#discussion_r2646647342 ## sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/DebeziumReadSchemaTransformProvider.java: ## @@ -23,7 +23,6 @@ import java.util.Collection; import java.util.Collections; import java.util.List; -import java.util.Objects; Review Comment: This is an unintended change, this happened automatically when I ran `./gradlew spotlessApply`. But I have reverted it back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
kennknowles commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3661913311 waiting on author -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3660235665 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3646254015 Reminder, please take a look at this pr: @chamikaramj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
kennknowles commented on code in PR #36296:
URL: https://github.com/apache/beam/pull/36296#discussion_r2591170411
##
sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/DebeziumReadSchemaTransformProvider.java:
##
@@ -23,7 +23,6 @@
import java.util.Collection;
import java.util.Collections;
import java.util.List;
-import java.util.Objects;
Review Comment:
Unrelated file - just leave it
##
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteRename.java:
##
@@ -329,14 +331,39 @@ private BigQueryHelpers.PendingJob startCopy(
new EncryptionConfiguration().setKmsKeyName(kmsKey));
}
- String bqLocation =
- BigQueryHelpers.getDatasetLocation(
- datasetService, ref.getProjectId(), ref.getDatasetId());
+ String bqLocation;
+ try {
Review Comment:
You've repeated this block in each place. Instead, it should be abstracted
out. Or perhaps the exception should just be propagated. That is usually the
right thing to do unless it absolutely must be converted into RuntimeException.
##
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java:
##
@@ -632,20 +632,15 @@ static void verifyDatasetPresence(DatasetService
datasetService, TableReference
}
static String getDatasetLocation(
- DatasetService datasetService, String projectId, String datasetId) {
-Dataset dataset;
+ DatasetService datasetService, String projectId, String datasetId)
+ throws IOException, InterruptedException {
try {
- dataset = datasetService.getDataset(projectId, datasetId);
-} catch (Exception e) {
- if (e instanceof InterruptedException) {
-Thread.currentThread().interrupt();
- }
- throw new RuntimeException(
- String.format(
- "unable to obtain dataset for dataset %s in project %s",
datasetId, projectId),
- e);
+ Dataset dataset = datasetService.getDataset(projectId, datasetId);
+ return dataset.getLocation();
+} catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
Review Comment:
Log here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
Suvrat1629 commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3603776128 @chamikaramj @kennknowles Gentle nudge could you please take a look at this pr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3589112107 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @chamikaramj for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3575376413 Reminder, please take a look at this pr: @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3547354825 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3527545433 Reminder, please take a look at this pr: @ahmedabu98 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3490894814 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @ahmedabu98 for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3476330101 Reminder, please take a look at this pr: @chamikaramj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3442830265 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @chamikaramj for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3432074698 Reminder, please take a look at this pr: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3389775624 Reminder, please take a look at this pr: @chamikaramj -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3401496093 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label java. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
Suvrat1629 commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3362123466 @chamikaramj Gentle ping, please take a look. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3342576018 Assigning reviewers: R: @chamikaramj for label java. Note: If you would like to opt out of this review, comment `assign to next reviewer`. Available commands: - `stop reviewer notifications` - opt out of the automated review tooling - `remind me after tests pass` - tag the comment author after tests pass - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers) The PR bot will only process comments in the main thread (not review comments). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
github-actions[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3338899841 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [Java]Fix infinite retry issue in BigQuery getDatasetLocation() for non-existent datasets [beam]
gemini-code-assist[bot] commented on PR #36296: URL: https://github.com/apache/beam/pull/36296#issuecomment-3338317421 ## Summary of Changes Hello @Suvrat1629, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical flaw in BigQuery integration where attempts to query or manipulate non-existent datasets resulted in an infinite retry loop, consuming resources and masking the true error. The changes refactor the exception handling within the `BigQueryHelpers.getDatasetLocation()` method and its consumers, ensuring that dataset-not-found errors are now correctly identified and immediately propagated as specific exceptions, thereby preventing indefinite retries and providing more actionable feedback to users. ### Highlights * **Resolved Infinite Retry Issue**: Fixed a bug where BigQuery operations would endlessly retry when attempting to access a non-existent dataset, previously identified as issue #28438. * **Improved Exception Handling**: The `BigQueryHelpers.getDatasetLocation()` method now explicitly throws `IOException` and `InterruptedException`, instead of wrapping all exceptions in a generic `RuntimeException`. * **Specific Error Messaging**: Calling methods (`UpdateSchemaDestination`, `WriteRename`, `WriteTables`) now catch these specific exceptions and provide clearer error messages, particularly for "dataset not found" scenarios, preventing infinite retries. * **New Unit Tests**: Added unit tests for `BigQueryHelpers.getDatasetLocation()` to verify correct exception propagation for non-existent datasets and proper location retrieval for valid ones. Using Gemini Code Assist The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either `/gemini ` or `@gemini-code-assist `. Below is a summary of the supported commands on the current page. Feature | Command | Description --- | --- | --- Code Review | `/gemini review` | Performs a code review for the current pull request in its current state. Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state. Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. Help | `/gemini help` | Displays a list of available commands. Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG). You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist). [^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
