Re: rescuecd.spec - problems with sending build request
On Saturday 29 March 2008 17:52, Patryk Zawadzki wrote: If you want to fix anything, I'd suggest fixing the spec by using an unambiguous URI. Its not unambiguous, its correct. If I write at the beginning http then its obvious that this is a http:// URI. If I will write ftp:// then its obvious that this will be a ftp URI. Builder also has this information and could even decide what to do when getting files from distfiles. The URI is correct but wget's behavior is non-standard. It should not include the question mark and the params in the file name and distfiles/builder should not depend on it. we could enforce -O outfile to wget, and i think we should. not sure how it would be included on ~/.builderrc due different downloaders [0]. and of course different df download schema is already supported [1]. $ cat ~/.builderrc PROTOCOL=http # [1] USE_AXEL=yes # [0] AXEL_OPTS=-n 6 $ -- glen ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
Elan Ruusamäe wrote: and always possible to upload file with custom name to distfiles and use nourl sources in such cases. How? For example I need this for make-request.spec. But this is still ugly workaround. -- Regards, Kamil Dziedzic signature.asc Description: This is a digitally signed message part. ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
Dnia sobota 29 marzec 2008, Patryk Zawadzki napisał: Do we need this? Where and for what? For using distfiles. But where is a problem? Please be more verbose. Why it will fail if we will use my method? Some example. See above. wget is not the only downloader on the planet. Ok I understand this, but right now on builders it is used. I understand also that wget's method to get files is bad but right now we are using it. So I want simply fix/workaround this method so this will start work right now. Ok eot from me. I only want to say that builder script doesn't work well right now and this is highly uncomfortable to use workarounds for Sources which uses some special characters. This is really ugly;) -- Regards, Kamil Dziedzic signature.asc Description: This is a digitally signed message part. ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
Kamil Dziedzic wrote: Elan Ruusam=C3=A4e wrote: and always possible to upload file with custom name to distfiles and use nourl sources in such cases. How? For example I need this for make-request.spec. Manually using dropin test.spec ? -- === Andrzej M. Krzysztofowicz [EMAIL PROTECTED] phone (48)(58) 347 19 36 Faculty of Applied Phys. Math., Gdansk University of Technology ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
Patryk Zawadzki wrote: It's not a valid URI to put in spec. Not if you want to be able to freely switch between different URI schemas (http:// vs. ftp:// is a good enough example). Do we need this? Where and for what? In http schema the ? character serves the purpose of separating a document name from a query string. Therefore the question mark is not part of the document requested, it's there to pass additional parameters to the http server. It should also not be saved as part of the file name. The file name should be derived from either response headers (for example Disposition headers allows one to set the desired file name) or the server's document name (the part between the last / and ?). wget behaves differently and saves the question mark as part of the file name which is required for wget's teleport mode to work (where you request a full website mirror locally). Even if you asked wget to behave like a usual web browser (and thus have the correct file name on distfiles), the builder script still has no idea what name to search for (as the web server is free to pass any file name in the response headers). I know that and this only explains why wget saves files with everything after ? and also explains why we don't use name given by webbrowser. This is obvious. But this still doesn't explain why you refuse to serve wget encoded name part of URL when downloading from distfiles (especially when builder script could even distinguish if file was downloaded from http or ftp because it has such information in spec - but this is not required). Please tell me what this will brake. What will/could fail after encoding name when getting files from distfiles? For me this is correct fix i don't understand why You telling its not? I just want some example;) If you want to fix anything, I'd suggest fixing the spec by using an unambiguous URI. Its not unambiguous, its correct. If I write at the beginning http then its obvious that this is a http:// URI. If I will write ftp:// then its obvious that this will be a ftp URI. Builder also has this information and could even decide what to do when getting files from distfiles. -- Regards, Kamil Dziedzic signature.asc Description: This is a digitally signed message part. ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
2008/3/29 Kamil Dziedzic [EMAIL PROTECTED]: Patryk Zawadzki wrote: It's not a valid URI to put in spec. Not if you want to be able to freely switch between different URI schemas (http:// vs. ftp:// is a good enough example). Do we need this? Where and for what? For using distfiles. I know that and this only explains why wget saves files with everything after ? and also explains why we don't use name given by webbrowser. This is obvious. But this still doesn't explain why you refuse to serve wget encoded name part of URL when downloading from distfiles (especially when builder script could even distinguish if file was downloaded from http or ftp because it has such information in spec - but this is not required). Soon we will get rpm that is capable of downloading files on its own. It's likely that it will behave in a more standardized way than wget. Such URIs will pose a problem at some point in time. Please tell me what this will brake. What will/could fail after encoding name when getting files from distfiles? For me this is correct fix i don't understand why You telling its not? I just want some example;) See above. wget is not the only downloader on the planet. If you want to fix anything, I'd suggest fixing the spec by using an unambiguous URI. Its not unambiguous, its correct. If I write at the beginning http then its obvious that this is a http:// URI. If I will write ftp:// then its obvious that this will be a ftp URI. Builder also has this information and could even decide what to do when getting files from distfiles. The URI is correct but wget's behavior is non-standard. It should not include the question mark and the params in the file name and distfiles/builder should not depend on it. -- Patryk Zawadzki PLD Linux Distribution ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
On Tuesday 18 March 2008 22:58:53 Kamil Dziedzic wrote: Dnia wtorek 18 marzec 2008, Elan Ruusamäe napisał: this could be either by the fact that it is not fetched, or something doesn't like filename in form 'rcdmod?x86_64' (containing question mark) Ok I know what is the problem. Builder should encode urls when fetching from distfiles. Now it try to get: ftp://distfiles.pld-linux.org/distfiles/by-md5/d/7/d712792e3216e49aec85bf50 46d1e212/rcdmod?x86 This doesn't work. But it should try to get: ftp://distfiles.pld-linux.org/distfiles/by-md5/d/7/d712792e3216e49aec85bf50 46d1e212/rcdmod%3fx86 This works. Any reasons to not do that? Do we have any simple script for url encoding? this all depends how you fetch the file, via squid proxy both urls work, and the file is saved as 'rcdmod?x86' on distfiles. and all the other things apply that patrys said. sometimes you can workaround and append /filename in url: /rcdmod?x86/x86 or even: /rcdmod?x86foo=/x86 depending on web application giving you that url and always possible to upload file with custom name to distfiles and use nourl sources in such cases. -- glen ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
On Tuesday 18 March 2008 01:37:37 Kamil Dziedzic wrote: http://wklej.org/id/0868358518 What I break?;) And how to fix that? builders accept files only from distfiles, but first url is not available from df. this could be either by the fact that it is not fetched, or something doesn't like filename in form 'rcdmod?x86_64' (containing question mark) -- glen ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
Dnia wtorek 18 marzec 2008, Elan Ruusamäe napisał: this could be either by the fact that it is not fetched, or something doesn't like filename in form 'rcdmod?x86_64' (containing question mark) Ok I know what is the problem. Builder should encode urls when fetching from distfiles. Now it try to get: ftp://distfiles.pld-linux.org/distfiles/by-md5/d/7/d712792e3216e49aec85bf5046d1e212/rcdmod?x86 This doesn't work. But it should try to get: ftp://distfiles.pld-linux.org/distfiles/by-md5/d/7/d712792e3216e49aec85bf5046d1e212/rcdmod%3fx86 This works. Any reasons to not do that? Do we have any simple script for url encoding? -- Regards, Kamil Dziedzic signature.asc Description: This is a digitally signed message part. ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
2008/3/18 Kamil Dziedzic [EMAIL PROTECTED]: Dnia wtorek 18 marzec 2008, Elan Ruusamäe napisał: this could be either by the fact that it is not fetched, or something doesn't like filename in form 'rcdmod?x86_64' (containing question mark) Ok I know what is the problem. Builder should encode urls when fetching from distfiles. Now it try to get: ftp://distfiles.pld-linux.org/distfiles/by-md5/d/7/d712792e3216e49aec85bf5046d1e212/rcdmod?x86 This doesn't work. But it should try to get: ftp://distfiles.pld-linux.org/distfiles/by-md5/d/7/d712792e3216e49aec85bf5046d1e212/rcdmod%3fx86 This works. Any reasons to not do that? Yes, how do you differentiate between query params and parts of document name? It's the spec file where encoding should happen (as currently it's not a valid URI, I believe FTP schema does not even allow one to use query strings). -- Patryk Zawadzki PLD Linux Distribution ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
Dnia wtorek 18 marzec 2008, Patryk Zawadzki napisał: Yes, how do you differentiate between query params and parts of document name? It's the spec file where encoding should happen (as currently it's not a valid URI, I believe FTP schema does not even allow one to use query strings). Why in spec? It is a valid URI. I don't understand or you;) Could you give me a live example when this will not work? For better undertanding I will try to explain this more verbose. Lets drop for now problems with rescuecd.spec (because it has more problems). Lets take for example dmg2img.spec. Logs from test build: http://wklej.org/id/479c5dfff2 I'm saying that builder script is now broken. You can't pass as SourceX urls with special characters like for example ? . Why not? Because when distfiles fetches files it makes something like that: wget -O download.pl?dmg2img.tar.gz http://vu1tur.eu.org/tools/download.pl?dmg2img.tar.gz From logs: 19:46:11 (0.00 B/s) - `download.pl?dmg2img.tar.gz' saved [0] So it get file download.pl and saves it as download.pl?dmg2img.tar.gz on ftp. Now when we want try to build a package from spec then builder script tries to download file from distfiles. He uses same method like distfiles to get file: wget -O download.pl?dmg2img.tar.gz ftp://distfiles.pld-linux.org/distfiles/by-md5/e/3/e3fa1bc5f38e961230100c1c2274bd28/download.pl?dmg2img.tar.gz From logs: # No matches on pattern `download.pl?dmg2img.tar.gz'. But on ftp there is no file download.pl, there is a file download.pl%3fdmg2img.tar.gz ftp://distfiles.pld-linux.org/distfiles/by-md5/e/3/e3fa1bc5f38e961230100c1c2274bd28/download.pl%3fdmg2img.tar.gz So to resolve this problem we should encode url (just name - download.pl?dmg2img.tar.gz) when geting file from distfiles like in example above. -- Regards, Kamil Dziedzic signature.asc Description: This is a digitally signed message part. ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
Re: rescuecd.spec - problems with sending build request
2008/3/18 Kamil Dziedzic [EMAIL PROTECTED]: Dnia wtorek 18 marzec 2008, Patryk Zawadzki napisał: Yes, how do you differentiate between query params and parts of document name? It's the spec file where encoding should happen (as currently it's not a valid URI, I believe FTP schema does not even allow one to use query strings). Why in spec? It is a valid URI. It's not a valid URI to put in spec. Not if you want to be able to freely switch between different URI schemas (http:// vs. ftp:// is a good enough example). In http schema the ? character serves the purpose of separating a document name from a query string. Therefore the question mark is not part of the document requested, it's there to pass additional parameters to the http server. It should also not be saved as part of the file name. The file name should be derived from either response headers (for example Disposition headers allows one to set the desired file name) or the server's document name (the part between the last / and ?). wget behaves differently and saves the question mark as part of the file name which is required for wget's teleport mode to work (where you request a full website mirror locally). Even if you asked wget to behave like a usual web browser (and thus have the correct file name on distfiles), the builder script still has no idea what name to search for (as the web server is free to pass any file name in the response headers). If you want to fix anything, I'd suggest fixing the spec by using an unambiguous URI. -- Patryk Zawadzki PLD Linux Distribution ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
rescuecd.spec - problems with sending build request
http://wklej.org/id/0868358518 What I break?;) And how to fix that? -- Regards, Kamil Dziedzic signature.asc Description: This is a digitally signed message part. ___ pld-devel-en mailing list pld-devel-en@lists.pld-linux.org http://lists.pld-linux.org/mailman/listinfo/pld-devel-en
