betodealmeida commented on PR #32520: URL: https://github.com/apache/superset/pull/32520#issuecomment-2755095524
> My concern is that by using a simple API, this feature will not work or be highly inefficient for companies with big datasets which is our case. How separating the API will work in practice? Are you going to launch the feature just to a subset of companies? If at the end we'll need to support big datasets, why pay the cost twice? Well, what I'm advocating is that we solve the problem we have today — where updating a dataset with tens of thousands of columsn takes too long — in a separate PR. The API is already slow, so I don't see the point in optimizing the folder updates when we would still be sending all the columns and all the metrics when we do an update. Here's my proposal: it seems like we need a new `PATCH` API for dataset updates. We should add the endpoint, and use a standardized DSL for deltas, like [JSON Patch](https://en.wikipedia.org/wiki/JSON_Patch), which has Python and npm implementations. It would be used for updateing columns, metrics, and folders., and the frontend would be updated to use it, since it's important to be responsive in an interactive environment like the browser. But we should keep the original `PUT` API for updates that exists today (and with the changes in this PR) because (1) it preserves backwards compatibility, and because (2) it's useful for non-interactive clients where response time is irrelevant. This way, someone who wants to sync datasets from dbt or Cube could use the much simpler `PUT` API, where the whole payload can be sent at once. So what I'm asking is: let's merge this PR, and I'll work on the `PATCH` endpoint that will solve all the problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
