Re: [RDD] RDCatch Questions - Weird Wildcards

[Fred Gleason]

On Dec 18, 2020, at 16:44, wa7skg  wrote:

> That takes care of the date issues. Now I need to figure out how to deal with 
> the changing episode names. Apparently, the traditional wildcards of * and ? 
> do not work in rdcatch. In testing, I find that in ftp, the wildcards do not 
> work with the get command, but they do work with mget. Is there possibly any 
> way to get rdcatch to use the mget command when fetching files?

It’s been looked into. The problem with it in an rdcatch(1) context is: what to 
do if the wildcard matches *multiple* remote files? (Multiple matches are not 
only possible, but rather likely in many contexts.)

The bottom line is that putting arbitrary strings into the file names is a 
really Bad Idea on the part of the program supplier. It makes downloads 
effectively impossible to ingest with any kind of reliability.

Cheers!


|-|
| Frederick F. Gleason, Jr. | Chief Developer |
|   | Paravel Systems |
|-|
| A room without books is like a body without a soul. |
| |
| -- Cicero   |
|-|___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

We all live within certain restrictions and limitations. I must work 
within the caveats of my position. The staff is being trained on the 
various functions of Rivendell. They are able to handle certain things 
in outside programs, such as downloading files via Filezilla, renaming 
them, and putting them in a folder to be ingested by the automation. The 
hope was to automate as much as possible. If program changes need to be 
made, the staff needs to be able to do that without my involvement. That 
will leave out any kind of scripts or cron jobs or pretty much anything 
else command line related. If rdcatch is not up to the task for these 
particular programs, then they will have to go to the old method of 
manual downloads. Fortunately, there are only a couple programs that 
fall into this category. They will only have to do this once per week 
with two programs, not a half dozen daily programs every day of the week.


Michael


Robert Jeffares wrote on 12/20/20 10:17 PM:

tut tut

On 18/12/20 12:50 pm, wa7skg wrote:
I need to keep this all within rdcatch, no scripts or exotic 
solutions. If it can't be done in rdcatch, then the staff will have to 
go back to manual daily downloads like they were doing prior to 
Rivendell. 


you write a shell script that downloads every week on the same day to a 
local folder



___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[Robert Jeffares]

On 18/12/20 1:04 pm, wa7skg wrote:

Whatever I set up needs to be within the documented system


for each automated cronjob write the manual page which is stored in a 
local printed folder


this is what is needed

this is how it happens

this is the result

this is the email that gets sent

do not monkey with this unless you know what you are doing

find and train one first responder.


--
Communication Consultants 2020 Limited
64 Warner Park Avenue
Laingholm
Auckland 0604
New Zealand

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[Robert Jeffares]

tut tut

On 18/12/20 12:50 pm, wa7skg wrote:
I need to keep this all within rdcatch, no scripts or exotic 
solutions. If it can't be done in rdcatch, then the staff will have to 
go back to manual daily downloads like they were doing prior to 
Rivendell. 


you write a shell script that downloads every week on the same day to a 
local folder


renaming as necessary

then carting each daily episode

and sending you a 'done' message; or a 'squawk'

all done as a cron job in the background no user intervention.

the producer has to be consistent in their labeling. I have one who is a 
bit erratic. There are some "but if" in that script.


you can wildcard anything in a filename in bash.

rd catch does well with the sane stuff.

some providers, especially ones whose 1st language is not English can be 
somewhat fey.


bash is very powerful.

if you need a hand send me the url of the download site, the names of 
the files you want and the GROUP and Cart Numbers you want them in. New 
cart each day not new cut each day although that can be done.


I can send you a script with comments so you can see how it's done and 
use the same ideas over again.


rdcatch and dropbox are both very useful but the syndicated stuff or the 
stuff you are leaching from somewhere may need something  a little more 
serious.


regards

Robert


--
Communication Consultants 2020 Limited
64 Warner Park Avenue
Laingholm
Auckland 0604
New Zealand

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

Turns out there are two Rivendell Facebook pages. Only one shows up in a 
search. That one, Rivendell, radio broadcast automation solution, the 
last post was March 20, 2018.


The other one, Rivendell Open Source Automation Users, is a private 
group that cannot be seen unless someone gives you a link. It says there 
have been 24 posts this month, but I cannot see any of them. I requested 
to join the group, but have received no reply.


Michael


Robert Jeffares wrote on 12/20/20 9:26 PM:


On 19/12/20 11:04 am, wa7skg wrote:
There hasn't been a post to the Rivendell Facebook page in 2 1/2 years. 



not true.

R


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[Robert Jeffares]

On 19/12/20 11:04 am, wa7skg wrote:
There hasn't been a post to the Rivendell Facebook page in 2 1/2 years. 



not true.

R

--
Communication Consultants 2020 Limited
64 Warner Park Avenue
Laingholm
Auckland 0604
New Zealand

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

Funny, this group never showed up in a search for Rivendell. Now I have 
the full name and found the right page. Membership in group applied for.


Thanks,
Michael


Andy Higginson wrote on 12/18/20 3:36 PM:

Hi,

You may be looking at the wrong Facebook group.  The one to look at is 
"Rivendell Open Source Automation Users”.  The last post to this group was 3 
days ago, with the last comment 5 hours ago.

Andy



___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[Lorne Tyndale]

Michael,

I'm not sure if you're looking at the correct Facebook page.  There are
2 for Rivendell, for the one with a lot of traffic search for:

Rivendell Open Source Automation Users

Its most recent post (with active replies) was just from a few days ago.
 I've assisted others on that page several times within the past week.

There is another Rivendell page on Facebook that has no traffic as you
mention.  I agree that Facebook is not the best forum for technical
support, but a lot of people do use it.

Lorne Tyndale


> 
> 
> There hasn't been a post to the Rivendell Facebook page in 2 1/2 years. 
> Not a lot of help there. Facebook is a horrible place for user support.
> 
> Michael
> 
> 
> Mike Carroll wrote on 12/18/20 12:50 PM:
> 
> > If you have a Facebook presence you might also check the Rivendell group 
> > there; it's geared more towards Rivendell users than this mailing list 
> > (which is geared more to the development of Rivendell).
> > 
> > Mike
> > 
> ___
> Rivendell-dev mailing list
> Rivendell-dev@lists.rivendellaudio.org
> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[Andy Higginson]

Hi,

You may be looking at the wrong Facebook group.  The one to look at is 
"Rivendell Open Source Automation Users”.  The last post to this group was 3 
days ago, with the last comment 5 hours ago.

Andy




> On 18 Dec 2020, at 22:04, wa7skg  wrote:
> 
> There hasn't been a post to the Rivendell Facebook page in 2 1/2 years. Not a 
> lot of help there. Facebook is a horrible place for user support.
> 
> Michael
> 
> 
> Mike Carroll wrote on 12/18/20 12:50 PM:
> 
>> If you have a Facebook presence you might also check the Rivendell group 
>> there; it's geared more towards Rivendell users than this mailing list 
>> (which is geared more to the development of Rivendell).
>> Mike
> ___
> Rivendell-dev mailing list
> Rivendell-dev@lists.rivendellaudio.org
> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

There hasn't been a post to the Rivendell Facebook page in 2 1/2 years. 
Not a lot of help there. Facebook is a horrible place for user support.


Michael


Mike Carroll wrote on 12/18/20 12:50 PM:

If you have a Facebook presence you might also check the Rivendell group 
there; it's geared more towards Rivendell users than this mailing list 
(which is geared more to the development of Rivendell).


Mike


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

After rereading this about nine times, the light finally came on. 
Looking at the wildcard specs again, I see the  variable, 
and understand what you mean.


First, my offset list:

Monday - 0
Tuesday - 86400
Wednesday - 172800
Thursday - 259200
Friday - 345600

That would make my strings:
Mon: ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode.mp3
Tue: ftp.hostname.com/program/%Y-%m%d/%m%86400d_program_episode.mp3
Wed: ftp.hostname.com/program/%Y-%m%d/%m%172800d_program_episode.mp3
Thu: ftp.hostname.com/program/%Y-%m%d/%m%259200d_program_episode.mp3
Fri: ftp.hostname.com/program/%Y-%m%d/%m%345600d_program_episode.mp3

By setting the Event Offset for 3 days and running all of these on 
Friday, that should give me all the programs for the next week.


I'll try these as macros as suggested below to see if they actually work.

That takes care of the date issues. Now I need to figure out how to deal 
with the changing episode names. Apparently, the traditional wildcards 
of * and ? do not work in rdcatch. In testing, I find that in ftp, the 
wildcards do not work with the get command, but they do work with mget. 
Is there possibly any way to get rdcatch to use the mget command when 
fetching files?


Michael


John Boles wrote on 12/17/20 5:29 PM:
Not knowing if you download everything on the same day or if everything 
is downloaded the day of

This is if you Download on a Friday (tomorrow 2020-12-18) for the next week
ftp.hostname.com/program/%Y-%m%259200d/%m%259200d_program_episode-name.mp3 

= ftp.hostname.com/program/2020-12-21/1221_program_episode-name.mp3 



So the 25900 equates to 3 days in seconds
1 day is 86400 seconds

If you download the day of:  Today of this week for this Weeks Monday in 
this example
ftp.hostname.com/program/%Y-%m%-259200d/%m%d_program_episode-name.mp3 

  = ftp.hostname.com/program/2020-12-14/1217_program_episode-name.mp3 



If you want to test what your metadata values will turn out to.
Use the Host Variables in RDAdmin->Manage Hosts->Host Variables.
Create your Host Variable:
Var Name: %DownloadDate%
Var Value:  %Y-%m%-259200d
Remark: Variable Tests

Then create a macros to display the value in RDAirplay.
LB Folder Date: %DownloadData%!
SP 5000!
LB !

Save and run the macro from the library and you should see the values 
equate in RDAirplay Message Box



Hope this makes sense
John

On Thu, Dec 17, 2020 at 5:04 PM wa7skg > wrote:


To reiterate - "I need to keep this all within rdcatch, no scripts or
exotic solutions. If it can't be done in rdcatch, then the staff will
have to go back to manual daily downloads like they were doing prior to
Rivendell."

I am only providing part time assistance to this station. The staff is
doing well with Rivendell. However, anything else is out of their
realm.
They want to be able to make their own changes with programming and
retrieval. That mandates staying within whatever Rivendell does. They
have training and documentation to use Rivendell and I can walk them
through pretty much anything over the phone. External cron jobs,
scripts, and other convoluted gymnastics goes against my support plan.
They were in the pickle they were in previously with an engineer who
did
many things with "creative solutions" that were undocumented and
outside
the normal operations of things over the last many years, then
evaporated leaving them hanging.

Whatever I set up needs to be within the documented system, so if I get
hit by a truck next week, they are not again left hanging.

Michael


chris cottingham wrote on 12/17/20 3:52 PM:
 > Maybe you can do an intermediary download with a program that can
deal with the file structure to a folder on your RD server? Then Set
RD To import from your folder?
 >
 > Just thinking off the top of my head.
 >
 >
 >
 >
 >
 > Sent from my iPhone
 >
 >> On Dec 17, 2020, at 3:50 PM, wa7skg mailto:wa7...@wa7skg.com>> wrote:
 >>
 >> No RSS feed. I tried
 >>
 >> ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3

 >>
 >> but both %d wildcards pull the same day each time. The first one
needs to keep Monday's date while the second one does Monday through
Friday.
 >>
 >> I need to keep this all within rdcatch, no scripts or exotic
solutions. If it can't be done in rdcatch, then the staff will have
to go back to manual daily downloads like they were doing prior to
Rivendell.
 >>
 >> Michael
 >>
 >> David Klann wrote on 12/17/20 2:32

Re: [RDD] RDCatch Questions - Weird Wildcards

[Mike Carroll]

RDCatch requires an exact filename - which can be constructed using various
metadata pieces, such as the date.  Mask characters aren't allowed.

What you might be able to do is create a shell script that uses curl or
wget to retrieve the program file and place it into a local directory
specific to the program.  Then set up a Rivendell dropbox to monitor that
directory and ingest any file it finds there into the correct Rivendell
cart.  Finally, set up an RDCatch event to run the script.  The script will
run at the specified time and download the file.  Shortly after that the
dropbox will "see" the new file and ingest it.  This is sort of sloppy
(two-stage transfer) but it should be enough to let the staff work on
other things.

Rivendell dropboxes don't care about filenames, except to try to extract
program information from them if requested. If a file shows up in the
dropbox folder, it gets ingested.

We've just started exploring dropboxes, so that's about all I know. Others
can probably help more.

If you have a Facebook presence you might also check the Rivendell group
there; it's geared more towards Rivendell users than this mailing list
(which is geared more to the development of Rivendell).

Mike


On Fri, Dec 18, 2020 at 9:47 AM wa7skg  wrote:

> Well, the date issue notwithstanding. how do you deal with random
> portions of the filename? In the below example, while "program" is
> always the same, the "episode-name" is unique to each file, like
> "C4701_Paul_trip_1", "C7604_Acts_Chapter_3", "C7605_Acts_Chapter_4",
> etc. How can you automate that?
>
> I thought using a "*" or "?" in the filename would take care of that,
> but it apparently is not allowed.
>
> Michael
>
>
> David Klann wrote on 12/17/20 2:32 PM:
> > Hi Michael
> >
> > On 12/17/20 1:02 PM, you wrote:
> >>
> >> Yes, there are more.
> >>
> >> One program I need to download has a troublesome file structure.
> >>
> >> There is a folder for the program, then a folder for each week beginning
> >> on Monday. The weekly folder holds the files for that week. For example:
> >>
> >> ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3
> >>
> >> ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
> >> ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3
> >>
> >
> > This looks to me like:
> >
> > ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3
> >
> > This only works if "program_episode-name" does not change from episode
> > to episode. At this time, Rivendell does not support arbitrary
> > differences in download filenames. Does this program have an RSS feed?
> > If so, I might be able to help with a script I wrote specifically for
> > downloading and ingesting audio from an RSS feed.
> >
> > ~David Klann
> >
> >> etc.
> >>
> >> I can't figure out how to set up a wildcard to deal with the weekly
> >> folder with Monday's date.
> >>
> >> This is a popular program that quite a few stations carry, so I'm sure
> >> there must be a way to automate it.
> >>
> >> Thanks for any ideas.
> >>
> >> Michael
> >>
> ___
> Rivendell-dev mailing list
> Rivendell-dev@lists.rivendellaudio.org
> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
>
___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

Well, the date issue notwithstanding. how do you deal with random 
portions of the filename? In the below example, while "program" is 
always the same, the "episode-name" is unique to each file, like 
"C4701_Paul_trip_1", "C7604_Acts_Chapter_3", "C7605_Acts_Chapter_4", 
etc. How can you automate that?


I thought using a "*" or "?" in the filename would take care of that, 
but it apparently is not allowed.


Michael


David Klann wrote on 12/17/20 2:32 PM:

Hi Michael

On 12/17/20 1:02 PM, you wrote:


Yes, there are more.

One program I need to download has a troublesome file structure.

There is a folder for the program, then a folder for each week beginning
on Monday. The weekly folder holds the files for that week. For example:

ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3

ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3



This looks to me like:

ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3

This only works if "program_episode-name" does not change from episode
to episode. At this time, Rivendell does not support arbitrary
differences in download filenames. Does this program have an RSS feed?
If so, I might be able to help with a script I wrote specifically for
downloading and ingesting audio from an RSS feed.

~David Klann


etc.

I can't figure out how to set up a wildcard to deal with the weekly
folder with Monday's date.

This is a popular program that quite a few stations carry, so I'm sure
there must be a way to automate it.

Thanks for any ideas.

Michael


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[David Klann]

Using RDCatch to download media, the wildcards are expanded with *the 
current values*. So, using date wildcards, you can download the episode 
for December 14th only *on December 14th*. One option is to code the 
*exact name* of the remote file in the RDCatch job, but this is what I 
would call "anti-automation".

As chris cottingham writes, and as far as I know, you'll need to use a 
facility external to Rivendell to do anything else.

   ~David


On 12/17/20 5:50 PM, wa7skg wrote:
> 
> No RSS feed. I tried
> 
> ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3
> 
> but both %d wildcards pull the same day each time. The first one needs
> to keep Monday's date while the second one does Monday through Friday.
> 
> I need to keep this all within rdcatch, no scripts or exotic solutions.
> If it can't be done in rdcatch, then the staff will have to go back to
> manual daily downloads like they were doing prior to Rivendell.
> 
> Michael
> 
> David Klann wrote on 12/17/20 2:32 PM:
>> Hi Michael
>>
>> On 12/17/20 1:02 PM, you wrote:
>>>
>>> Yes, there are more.
>>>
>>> One program I need to download has a troublesome file structure.
>>>
>>> There is a folder for the program, then a folder for each week beginning
>>> on Monday. The weekly folder holds the files for that week. For example:
>>>
>>> ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3
>>>
>>> ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3
>>>
>>
>> This looks to me like:
>>
>> ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3
>>
>> This only works if "program_episode-name" does not change from episode
>> to episode. At this time, Rivendell does not support arbitrary
>> differences in download filenames. Does this program have an RSS feed?
>> If so, I might be able to help with a script I wrote specifically for
>> downloading and ingesting audio from an RSS feed.
>>
>>  ~David Klann
>>
>>> etc.
>>>
>>> I can't figure out how to set up a wildcard to deal with the weekly
>>> folder with Monday's date.
>>>
>>> This is a popular program that quite a few stations carry, so I'm sure
>>> there must be a way to automate it.
>>>
>>> Thanks for any ideas.
>>>
>>> Michael
>>>
>>>
>>> ___
>>> Rivendell-dev mailing list
>>> Rivendell-dev@lists.rivendellaudio.org
>>> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
>>>
>>

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[David Klann]

Hi Michael

On 12/17/20 1:02 PM, you wrote:
> 
> Yes, there are more.
> 
> One program I need to download has a troublesome file structure.
> 
> There is a folder for the program, then a folder for each week beginning
> on Monday. The weekly folder holds the files for that week. For example:
> 
> ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
> ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
> ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
> ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
> ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3
> 
> ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
> ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
> ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
> ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
> ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3
> 

This looks to me like:

ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3

This only works if "program_episode-name" does not change from episode 
to episode. At this time, Rivendell does not support arbitrary 
differences in download filenames. Does this program have an RSS feed? 
If so, I might be able to help with a script I wrote specifically for 
downloading and ingesting audio from an RSS feed.

   ~David Klann

> etc.
> 
> I can't figure out how to set up a wildcard to deal with the weekly
> folder with Monday's date.
> 
> This is a popular program that quite a few stations carry, so I'm sure
> there must be a way to automate it.
> 
> Thanks for any ideas.
> 
> Michael
> 
> 
> ___
> Rivendell-dev mailing list
> Rivendell-dev@lists.rivendellaudio.org
> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
> 

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[John Boles]

Couple of things to note:
The example is using 0 padded wildcards, change if not your use case.
LB Folder Date: %DownloadData%!
Should Be:
LB Folder Date: %DownloadDate%!

On Thu, Dec 17, 2020 at 6:29 PM John Boles  wrote:

> Not knowing if you download everything on the same day or if everything is
> downloaded the day of
> This is if you Download on a Friday (tomorrow 2020-12-18) for the next week
> ftp.hostname.com/program/%Y-%m%259200d/%m%259200d_program_episode-name.mp3
> = ftp.hostname.com/program/2020-12-21/1221_program_episode-name.mp3
>
> So the 25900 equates to 3 days in seconds
> 1 day is 86400 seconds
>
> If you download the day of:  Today of this week for this Weeks Monday in
> this example
> ftp.hostname.com/program/%Y-%m%-259200d/%m%d_program_episode-name.mp3
>  = ftp.hostname.com/program/2020-12-14/1217_program_episode-name.mp3
>
> If you want to test what your metadata values will turn out to.
> Use the Host Variables in RDAdmin->Manage Hosts->Host Variables.
> Create your Host Variable:
> Var Name: %DownloadDate%
> Var Value:  %Y-%m%-259200d
> Remark: Variable Tests
>
> Then create a macros to display the value in RDAirplay.
> LB Folder Date: %DownloadData%!
> SP 5000!
> LB !
>
> Save and run the macro from the library and you should see the values
> equate in RDAirplay Message Box
>
>
> Hope this makes sense
> John
>
> On Thu, Dec 17, 2020 at 5:04 PM wa7skg  wrote:
>
>> To reiterate - "I need to keep this all within rdcatch, no scripts or
>> exotic solutions. If it can't be done in rdcatch, then the staff will
>> have to go back to manual daily downloads like they were doing prior to
>> Rivendell."
>>
>> I am only providing part time assistance to this station. The staff is
>> doing well with Rivendell. However, anything else is out of their realm.
>> They want to be able to make their own changes with programming and
>> retrieval. That mandates staying within whatever Rivendell does. They
>> have training and documentation to use Rivendell and I can walk them
>> through pretty much anything over the phone. External cron jobs,
>> scripts, and other convoluted gymnastics goes against my support plan.
>> They were in the pickle they were in previously with an engineer who did
>> many things with "creative solutions" that were undocumented and outside
>> the normal operations of things over the last many years, then
>> evaporated leaving them hanging.
>>
>> Whatever I set up needs to be within the documented system, so if I get
>> hit by a truck next week, they are not again left hanging.
>>
>> Michael
>>
>>
>> chris cottingham wrote on 12/17/20 3:52 PM:
>> > Maybe you can do an intermediary download with a program that can deal
>> with the file structure to a folder on your RD server? Then Set RD To
>> import from your folder?
>> >
>> > Just thinking off the top of my head.
>> >
>> >
>> >
>> >
>> >
>> > Sent from my iPhone
>> >
>> >> On Dec 17, 2020, at 3:50 PM, wa7skg  wrote:
>> >>
>> >> No RSS feed. I tried
>> >>
>> >> ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3
>> >>
>> >> but both %d wildcards pull the same day each time. The first one needs
>> to keep Monday's date while the second one does Monday through Friday.
>> >>
>> >> I need to keep this all within rdcatch, no scripts or exotic
>> solutions. If it can't be done in rdcatch, then the staff will have to go
>> back to manual daily downloads like they were doing prior to Rivendell.
>> >>
>> >> Michael
>> >>
>> >> David Klann wrote on 12/17/20 2:32 PM:
>> >>> Hi Michael
>>  On 12/17/20 1:02 PM, you wrote:
>> 
>>  Yes, there are more.
>> 
>>  One program I need to download has a troublesome file structure.
>> 
>>  There is a folder for the program, then a folder for each week
>> beginning
>>  on Monday. The weekly folder holds the files for that week. For
>> example:
>> 
>>  ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3
>> 
>>  ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
>>  ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3
>> 
>> >>> This looks to me like:
>> >>> ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3
>> >>> This only works if "program_episode-name" does not change from episode
>> >>> to episode. At this time, Rivendell does not support arbitrary
>> >>> differences in download filenames. Does this program have an RSS feed?
>> >>> If so, I might be able to help with a script I wrote specifica

Re: [RDD] RDCatch Questions - Weird Wildcards

[John Boles]

Not knowing if you download everything on the same day or if everything is
downloaded the day of
This is if you Download on a Friday (tomorrow 2020-12-18) for the next week
ftp.hostname.com/program/%Y-%m%259200d/%m%259200d_program_episode-name.mp3
= ftp.hostname.com/program/2020-12-21/1221_program_episode-name.mp3

So the 25900 equates to 3 days in seconds
1 day is 86400 seconds

If you download the day of:  Today of this week for this Weeks Monday in
this example
ftp.hostname.com/program/%Y-%m%-259200d/%m%d_program_episode-name.mp3
 = ftp.hostname.com/program/2020-12-14/1217_program_episode-name.mp3

If you want to test what your metadata values will turn out to.
Use the Host Variables in RDAdmin->Manage Hosts->Host Variables.
Create your Host Variable:
Var Name: %DownloadDate%
Var Value:  %Y-%m%-259200d
Remark: Variable Tests

Then create a macros to display the value in RDAirplay.
LB Folder Date: %DownloadData%!
SP 5000!
LB !

Save and run the macro from the library and you should see the values
equate in RDAirplay Message Box


Hope this makes sense
John

On Thu, Dec 17, 2020 at 5:04 PM wa7skg  wrote:

> To reiterate - "I need to keep this all within rdcatch, no scripts or
> exotic solutions. If it can't be done in rdcatch, then the staff will
> have to go back to manual daily downloads like they were doing prior to
> Rivendell."
>
> I am only providing part time assistance to this station. The staff is
> doing well with Rivendell. However, anything else is out of their realm.
> They want to be able to make their own changes with programming and
> retrieval. That mandates staying within whatever Rivendell does. They
> have training and documentation to use Rivendell and I can walk them
> through pretty much anything over the phone. External cron jobs,
> scripts, and other convoluted gymnastics goes against my support plan.
> They were in the pickle they were in previously with an engineer who did
> many things with "creative solutions" that were undocumented and outside
> the normal operations of things over the last many years, then
> evaporated leaving them hanging.
>
> Whatever I set up needs to be within the documented system, so if I get
> hit by a truck next week, they are not again left hanging.
>
> Michael
>
>
> chris cottingham wrote on 12/17/20 3:52 PM:
> > Maybe you can do an intermediary download with a program that can deal
> with the file structure to a folder on your RD server? Then Set RD To
> import from your folder?
> >
> > Just thinking off the top of my head.
> >
> >
> >
> >
> >
> > Sent from my iPhone
> >
> >> On Dec 17, 2020, at 3:50 PM, wa7skg  wrote:
> >>
> >> No RSS feed. I tried
> >>
> >> ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3
> >>
> >> but both %d wildcards pull the same day each time. The first one needs
> to keep Monday's date while the second one does Monday through Friday.
> >>
> >> I need to keep this all within rdcatch, no scripts or exotic solutions.
> If it can't be done in rdcatch, then the staff will have to go back to
> manual daily downloads like they were doing prior to Rivendell.
> >>
> >> Michael
> >>
> >> David Klann wrote on 12/17/20 2:32 PM:
> >>> Hi Michael
>  On 12/17/20 1:02 PM, you wrote:
> 
>  Yes, there are more.
> 
>  One program I need to download has a troublesome file structure.
> 
>  There is a folder for the program, then a folder for each week
> beginning
>  on Monday. The weekly folder holds the files for that week. For
> example:
> 
>  ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3
> 
>  ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
>  ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3
> 
> >>> This looks to me like:
> >>> ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3
> >>> This only works if "program_episode-name" does not change from episode
> >>> to episode. At this time, Rivendell does not support arbitrary
> >>> differences in download filenames. Does this program have an RSS feed?
> >>> If so, I might be able to help with a script I wrote specifically for
> >>> downloading and ingesting audio from an RSS feed.
> >>> ~David Klann
>  etc.
> 
>  I can't figure out how to set up a wildcard to deal with the weekly
>  folder with Monday's date.
> 
>  This is a popular program that quite a few stations carry, so I'm sure
>  there must be a way to automate it.
> 
>  Thanks for

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

To reiterate - "I need to keep this all within rdcatch, no scripts or 
exotic solutions. If it can't be done in rdcatch, then the staff will 
have to go back to manual daily downloads like they were doing prior to 
Rivendell."


I am only providing part time assistance to this station. The staff is 
doing well with Rivendell. However, anything else is out of their realm. 
They want to be able to make their own changes with programming and 
retrieval. That mandates staying within whatever Rivendell does. They 
have training and documentation to use Rivendell and I can walk them 
through pretty much anything over the phone. External cron jobs, 
scripts, and other convoluted gymnastics goes against my support plan. 
They were in the pickle they were in previously with an engineer who did 
many things with "creative solutions" that were undocumented and outside 
the normal operations of things over the last many years, then 
evaporated leaving them hanging.


Whatever I set up needs to be within the documented system, so if I get 
hit by a truck next week, they are not again left hanging.


Michael


chris cottingham wrote on 12/17/20 3:52 PM:

Maybe you can do an intermediary download with a program that can deal with the 
file structure to a folder on your RD server? Then Set RD To import from your 
folder?

Just thinking off the top of my head.





Sent from my iPhone


On Dec 17, 2020, at 3:50 PM, wa7skg  wrote:

No RSS feed. I tried

ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3

but both %d wildcards pull the same day each time. The first one needs to keep 
Monday's date while the second one does Monday through Friday.

I need to keep this all within rdcatch, no scripts or exotic solutions. If it 
can't be done in rdcatch, then the staff will have to go back to manual daily 
downloads like they were doing prior to Rivendell.

Michael

David Klann wrote on 12/17/20 2:32 PM:

Hi Michael

On 12/17/20 1:02 PM, you wrote:

Yes, there are more.

One program I need to download has a troublesome file structure.

There is a folder for the program, then a folder for each week beginning
on Monday. The weekly folder holds the files for that week. For example:

ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3

ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3


This looks to me like:
ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3
This only works if "program_episode-name" does not change from episode
to episode. At this time, Rivendell does not support arbitrary
differences in download filenames. Does this program have an RSS feed?
If so, I might be able to help with a script I wrote specifically for
downloading and ingesting audio from an RSS feed.
~David Klann

etc.

I can't figure out how to set up a wildcard to deal with the weekly
folder with Monday's date.

This is a popular program that quite a few stations carry, so I'm sure
there must be a way to automate it.

Thanks for any ideas.

Michael


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[chris cottingham]

Maybe you can do an intermediary download with a program that can deal with the 
file structure to a folder on your RD server? Then Set RD To import from your 
folder?

Just thinking off the top of my head. 





Sent from my iPhone

> On Dec 17, 2020, at 3:50 PM, wa7skg  wrote:
> 
> No RSS feed. I tried
> 
> ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3
> 
> but both %d wildcards pull the same day each time. The first one needs to 
> keep Monday's date while the second one does Monday through Friday.
> 
> I need to keep this all within rdcatch, no scripts or exotic solutions. If it 
> can't be done in rdcatch, then the staff will have to go back to manual daily 
> downloads like they were doing prior to Rivendell.
> 
> Michael
> 
> David Klann wrote on 12/17/20 2:32 PM:
>> Hi Michael
>>> On 12/17/20 1:02 PM, you wrote:
>>> 
>>> Yes, there are more.
>>> 
>>> One program I need to download has a troublesome file structure.
>>> 
>>> There is a folder for the program, then a folder for each week beginning
>>> on Monday. The weekly folder holds the files for that week. For example:
>>> 
>>> ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3
>>> 
>>> ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
>>> ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3
>>> 
>> This looks to me like:
>> ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3
>> This only works if "program_episode-name" does not change from episode
>> to episode. At this time, Rivendell does not support arbitrary
>> differences in download filenames. Does this program have an RSS feed?
>> If so, I might be able to help with a script I wrote specifically for
>> downloading and ingesting audio from an RSS feed.
>>~David Klann
>>> etc.
>>> 
>>> I can't figure out how to set up a wildcard to deal with the weekly
>>> folder with Monday's date.
>>> 
>>> This is a popular program that quite a few stations carry, so I'm sure
>>> there must be a way to automate it.
>>> 
>>> Thanks for any ideas.
>>> 
>>> Michael
>>> 
>>> 
>>> ___
>>> Rivendell-dev mailing list
>>> Rivendell-dev@lists.rivendellaudio.org
>>> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
>>> 
> ___
> Rivendell-dev mailing list
> Rivendell-dev@lists.rivendellaudio.org
> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

No RSS feed. I tried

ftp.hostname.com/program/%Y-%m%d/%m%d_program_*.mp3

but both %d wildcards pull the same day each time. The first one needs 
to keep Monday's date while the second one does Monday through Friday.


I need to keep this all within rdcatch, no scripts or exotic solutions. 
If it can't be done in rdcatch, then the staff will have to go back to 
manual daily downloads like they were doing prior to Rivendell.


Michael

David Klann wrote on 12/17/20 2:32 PM:

Hi Michael

On 12/17/20 1:02 PM, you wrote:


Yes, there are more.

One program I need to download has a troublesome file structure.

There is a folder for the program, then a folder for each week beginning
on Monday. The weekly folder holds the files for that week. For example:

ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3

ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3



This looks to me like:

ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3

This only works if "program_episode-name" does not change from episode
to episode. At this time, Rivendell does not support arbitrary
differences in download filenames. Does this program have an RSS feed?
If so, I might be able to help with a script I wrote specifically for
downloading and ingesting audio from an RSS feed.

~David Klann


etc.

I can't figure out how to set up a wildcard to deal with the weekly
folder with Monday's date.

This is a popular program that quite a few stations carry, so I'm sure
there must be a way to automate it.

Thanks for any ideas.

Michael


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev




___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

Yes, that is exactly the problem. The week folder has  Monday's date. 
The provider has over a hundred stations taking their program and nobody 
else seems to have a problem, so they don't want to change just for one 
station.


If it didn't use the date twice in the overall path, I could do it with 
offsets, but that doesn't work either.


I'd like to keep this all inside rdcatch and not get into exotic cron 
jobs or scripts.


Michael

Mark Murdock wrote on 12/17/20 2:36 PM:

It looks like you would need to change the number for the weekly folder every 
time you wanted files for a new week.

This is close, but won't work:

ftp.hostname.com/program/%Y-%m%d/%m%d_program_episode-name.mp3

because the first "%d" would change to the current date every day and thus mess 
up the file folder name. In the past when I had an issue like this with a provider that 
created a new folder for each month, I discussed the issue with them, and they decided to 
put their files into the same folder each month. Sometimes if you negotiate with these 
people they will modify their practices.


Mark Murdock
KAMB
90 E. 16th St.
Merced, CA 95340
(209) 723-1015
m...@celebrationradio.com
Website



-Original Message-
From: rivendell-dev-boun...@lists.rivendellaudio.org 
 On Behalf Of wa7skg
Sent: Thursday, December 17, 2020 11:03 AM
To: rivendell-dev@lists.rivendellaudio.org
Subject: [RDD] RDCatch Questions - Weird Wildcards

Yes, there are more.

One program I need to download has a troublesome file structure.

There is a folder for the program, then a folder for each week beginning on 
Monday. The weekly folder holds the files for that week. For example:

ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3

ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3

etc.

I can't figure out how to set up a wildcard to deal with the weekly folder with 
Monday's date.

This is a popular program that quite a few stations carry, so I'm sure there 
must be a way to automate it.

Thanks for any ideas.

Michael


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

[RDD] RDCatch Questions - Weird Wildcards

[wa7skg]

Yes, there are more.

One program I need to download has a troublesome file structure.

There is a folder for the program, then a folder for each week beginning 
on Monday. The weekly folder holds the files for that week. For example:


ftp.hostname.com/program/2020-1214/1214_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1215_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1216_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1217_program_episode-name.mp3
ftp.hostname.com/program/2020-1214/1218_program_episode-name.mp3

ftp.hostname.com/program/2020-1221/1221_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1222_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1223_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1224_program_episode-name.mp3
ftp.hostname.com/program/2020-1221/1225_program_episode-name.mp3

etc.

I can't figure out how to set up a wildcard to deal with the weekly 
folder with Monday's date.


This is a popular program that quite a few stations carry, so I'm sure 
there must be a way to automate it.


Thanks for any ideas.

Michael


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev