[lfs-support] Howto keep track....
...of changed/new files on http only sites like sourceforge.net? Keeping track of changed and new files on FTP sites is relative easy. However, HTTP sites is differently and made complicated because no page is the same and directory listings are not easy. In fact, I do not remember anymore how to get directory listings from HTTP servers. If that is possible and if it works mostly the same on every HTTP server, I can make a script to get it done. Since authors on LFS are probably using such a tool, what tool or mechanism I can try? Regards, Frans. -- http://linuxfromscratch.org/mailman/listinfo/lfs-support FAQ: http://www.linuxfromscratch.org/lfs/faq.html Unsubscribe: See the above information page
Re: [lfs-support] Howto keep track....
Frans de Boer wrote: ...of changed/new files on http only sites like sourceforge.net? Keeping track of changed and new files on FTP sites is relative easy. However, HTTP sites is differently and made complicated because no page is the same and directory listings are not easy. In fact, I do not remember anymore how to get directory listings from HTTP servers. If that is possible and if it works mostly the same on every HTTP server, I can make a script to get it done. Since authors on LFS are probably using such a tool, what tool or mechanism I can try? It's not easy. I have custom scripts that basically address each package. Sometimes upstream blocks directory listings completely. Sometimes you get into a situation where odd releases are stable and even development or vice versa. You can look at http://anduin.linuxfromscratch.org/~bdubbs/lfs-latest-files.phps but that is just an example and slightly out of date. -- Bruce -- http://linuxfromscratch.org/mailman/listinfo/lfs-support FAQ: http://www.linuxfromscratch.org/lfs/faq.html Unsubscribe: See the above information page
Re: [lfs-support] Howto keep track....
On 10/20/2013 04:43 PM, Bruce Dubbs wrote: Frans de Boer wrote: ...of changed/new files on http only sites like sourceforge.net? Keeping track of changed and new files on FTP sites is relative easy. However, HTTP sites is differently and made complicated because no page is the same and directory listings are not easy. In fact, I do not remember anymore how to get directory listings from HTTP servers. If that is possible and if it works mostly the same on every HTTP server, I can make a script to get it done. Since authors on LFS are probably using such a tool, what tool or mechanism I can try? It's not easy. I have custom scripts that basically address each package. Sometimes upstream blocks directory listings completely. Sometimes you get into a situation where odd releases are stable and even development or vice versa. You can look at http://anduin.linuxfromscratch.org/~bdubbs/lfs-latest-files.phps but that is just an example and slightly out of date. -- Bruce Thanks Bruce, Like I said no page is the same and I noticed that you handle every site in a different way using customized regex's. That is exactly the thing I try to avoid, but given the nature of things I assume that would be hard to accomplished using HTTP. I stay on the lookout of a more generic tool and am happy that my scripts can handle normal FTP transfers. I augmented the tables in my scripts to handle also the LFS site, since that can be reached by FTP too. Regards, Frans. -- http://linuxfromscratch.org/mailman/listinfo/lfs-support FAQ: http://www.linuxfromscratch.org/lfs/faq.html Unsubscribe: See the above information page
Re: [lfs-support] Howto keep track....
On Sun, 20 Oct 2013 18:33:53 +0200 Frans de Boer fr...@fransdb.nl wrote: Like I said no page is the same and I noticed that you handle every site in a different way using customized regex's. That is exactly the thing I try to avoid, but given the nature of things I assume that would be hard to accomplished using HTTP. I just had a flashback to Gopher. Gopher proponents usualy cite that exact reason as the reason the World should use Gopher instead of HTTP. Even though that is not HTTPs fault. It's really HTMLs fault. As for automated package tracking, I did an experiment using the source revision tools that various packages use (git, subversion, mercurial and others..) but had the mother of mixed success. While for some packages this works so well you would swear God gave his personal blessing, for other packages this is the worst kind of a nightmare. Using source revision tools also adds the aditional problem that in most cases you need to rebuild the ./configure script and that is often very difficult, if not impossible. And then there is the problem of both initial seeding and continuous maintenance of the 350+GB repository of repositories containing 300+ individual packages. Definitely not for the faint of heart. -- You don't need an AI for a robot uprising. Humans will do just fine. signature.asc Description: PGP signature -- http://linuxfromscratch.org/mailman/listinfo/lfs-support FAQ: http://www.linuxfromscratch.org/lfs/faq.html Unsubscribe: See the above information page
