[
https://issues.apache.org/jira/browse/BEAM-5071?focusedWorklogId=134743&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-134743
]
ASF GitHub Bot logged work on BEAM-5071:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Aug/18 21:51
Start Date: 14/Aug/18 21:51
Worklog Time Spent: 10m
Work Description: yifanzou commented on a change in pull request #6167:
[BEAM-5071] Replace the bigquery with restful APIs to query dependenc…
URL: https://github.com/apache/beam/pull/6167#discussion_r210103254
##########
File path:
.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
##########
@@ -105,18 +108,26 @@ def prioritize_dependencies(deps, sdk_type):
try:
logging.info("\n\nStart processing: " + dep)
dep_name, curr_ver, latest_ver = extract_single_dep(dep)
- curr_release_date, latest_release_date =
query_dependency_release_dates(bigquery_client,
-
dep_name,
-
curr_ver,
-
latest_ver)
+ curr_release_date = None
+ latest_release_date = None
group_id = None
+
if sdk_type == 'Java':
# extract the groupid and artifactid
group_id, artifact_id = dep_name.split(":")
dep_details_url = "{0}g:\"{1}\" AND
a:\"{2}\"".format(_MAVEN_CENTRAL_URL, group_id, artifact_id)
+ curr_release_date = find_release_time_from_maven_central(group_id,
artifact_id, curr_ver)
+ latest_release_date = find_release_time_from_maven_central(group_id,
artifact_id, latest_ver)
else:
dep_details_url = _PYPI_URL + dep_name
-
+ curr_release_date =
find_release_time_from_python_compatibility_checking_service(dep_name, curr_ver)
+ latest_release_date =
find_release_time_from_python_compatibility_checking_service(dep_name, curr_ver)
+
+ if not curr_release_date or not latest_release_date:
+ curr_release_date, latest_release_date =
query_dependency_release_dates_from_bigquery(bigquery_client,
Review comment:
Yes.
1. Need the bigquery table to record release dates of non-MavenCentral
packages.
2. The Python compatibility checking services is not mature, 10% of requests
will timeout even run out 3 retries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 134743)
Time Spent: 3h (was: 2h 50m)
> Using the restful API in Beam dependency check system, get rid of bigquery
> --------------------------------------------------------------------------
>
> Key: BEAM-5071
> URL: https://issues.apache.org/jira/browse/BEAM-5071
> Project: Beam
> Issue Type: Bug
> Components: dependencies
> Reporter: yifan zou
> Assignee: yifan zou
> Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
