Hi,
Sorry Frazer for not replying earlier.
For the benefit of the list, I'd like to point out that this proposal
came mostly out of this IRC conversation[0], and is aims to solve the
problem discussed last week[1], with regards to issue 408[2], while at
the same time addressing the issue of the collect_manifst plugin[3],
i.e. exposing API (on the CLI and on Source objects) which potentially
make it possible to really implement solutions to the collect_manifest
problem which are not complete hacks.
Some comments on this proposal so far:
* Could we have at least a small blurb in this proposal explaining the
problem statement and how these changes will solve said problem
statement ?
* I think that this is not really very specified, I would like to see
more specifics here, even if the resulting merge request might not
match the proposed APIs to the letter, it's better if we agree on the
expected APIs here first.
For this I'd like to see specifics such as:
- How will we be formatting the stdout of `bst source show` in order
to properly parse the multiple [0...N] sources per element, when
running `bst source show element.bst` ?
- Are we allowed to address a specific source belonging to an element
on the command line, instead of listing information for all of the
sources of an element ? if so, how will we interpret this on the
command line ?
- What are the new API contracts we will be exposing on the plugin
facing python API surface for the Source object ?
Can we please have a preliminary concept of what the calling
signatures will look like for these APIs, and an explanation of
whether it will be mandatory to implement these new methods
in a BuildStream 2 Source plugin ?
* Let's please be consistent with other show commands, I think that
for the original `bst show` interface and for `bst artifact show`
and `bst source show`, we should be supporting the `--format`
style argument to select what is displayed in the output and how.
This allows scripts to add newlines and additional formatting text
to `show` command outputs so that scripts can hopefully more easily
consume the output in any custom way they like (e.g. with `bst show`
currently, it's fairly simple to write some loops which end up
composing some output as json or yaml or whatever the caller wants
to use).
Because of the listy nature of sources and their URIs, maybe this
is not entirely possible ?
Cheers,
-Tristan
[0]:
https://irclogs.baserock.org/buildstream/%23buildstream.2020-07-02.log.html#t2020-07-02T08:33:47
[1]:
https://lists.apache.org/thread.html/r57a99da565e09fcae75857de1b212d86ad5ed3f49ae1d537964bdfd6%40%3Cdev.buildstream.apache.org%3E
[2]: https://gitlab.com/BuildStream/buildstream/-/issues/408
[3]: https://gitlab.com/BuildStream/bst-plugins-experimental/-/issues/2
On Tue, 2020-07-07 at 00:39 +0200, Sander Striker wrote:
> Hi Frazer,
>
> On Sat, Jul 4, 2020 at 6:19 PM Frazer Clews <
> [email protected]>
> wrote:
>
> > This is a proposal for `bst source show`
> >
> > The idea is to show the sources in a BuildStream project
> >
> > I think a normal `bst source show` should display the sources,
> > along
> > with their directories/URLs
> >
> > `bst source show`
> >
> > ```
> >
> > local: <example/dir>
> >
> > git: <git_url>
> >
> > https: <https_url>
> >
> > ```
> >
>
> Can you expand the above a bit more?
> What will `bst source show` actually show?
> All the sources for all elements?
> Will we be able to see which source belongs to which element, or
> should one
> run `bst source show ELEMENT` for that?
>
>
> > Further goals would be to have certain flags to show only the
> > directory/URL or the protocols in use
> >
> > `bst source show -p|--protocol`?
> >
> > ```
> >
> > local
> >
> > git
> >
> > https
> >
> > ```
> >
>
> This seems very specific. I'm a bit concerned we are catering for a
> very
> specific use case, and given that we are dealing with URLs here, for
> which
> a wide spectrum of tools is available to parse them, I don't know if
> this
> is the best thing to do. That is, I don't really want us to have a
> --protocol option to maintain.
>
> `bst source show -u|--url`?
> > ```
> >
> > <example/dir>
> >
> > <git_url>
> >
> > <https_url>
> >
> > ```
> >
> > Or one that only show a certain protocol, for example only git
> > sources
> >
> > `bst source show -g|--git`
> >
> > ```
> >
> > <git_url>
> >
> > ```
> >
>
> Let's get this into a full proposal? I'd like to see the flags you
> propose
> and what this will mean for the UI/UX, so that we can come up with
> something consistent and intuitive. This should likely look a lot
> like
> what we do for artifacts.
>
>
> > Finally and not sure if this should be the default or one with
> > flags,
> > but one that shows the file names as headings, similar to `git grep
> > --break --heading`
> >
> > `bst source show -f|--file-name`?
> >
> > ```
> >
> > hello.bst
> >
> > local: <example/dir>
> >
> > git: <git_url>
> >
> > https: <https_url>
> >
> > ```
> >
>
> I certainly think that having sources be associated with elements is
> useful.
>
>
> > Open to suggestions/improvements to start work on `bst source
> > show`,
> > could also use arguements from `bst source fetch` which work well
> > during
> > testing
> >
>
> Also please take into account
> https://gitlab.com/BuildStream/buildstream/-/issues/1273 and
> https://gitlab.com/BuildStream/buildstream/-/issues/1274 as well.
>
> HTH.
>
> Thanks
> > Frazer
> >
>
> Cheers,
>
> Sander
