[RDD] EBUR128 normalize - was RE: re-normalize a batch of carts?

[Lorne Tyndale]

Andy,

The script you are thinking of is python-rivendell, at:

https://github.com/sebleblanc/python-rivendell

It was referenced in a message:

http://caspian.paravelsystems.com/pipermail/rivendell-dev/2016-February/023931.html

It can use libebur128 to do a normalization.

At one point there was some discussion surrounding including R128
normalization as an option in Rivendell, but I don't believe it ever got
much past the discussion phase.

Lorne Tyndale


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Fred Gleason]

On Nov 2, 2018, at 12:29, David Klann  wrote:

> You would likely need to export the CSV data and use an app to coerce it
> into the acceptable XML format used in Rivendell. Not hard, but would
> require attention to the details.

You can also pass many of those fields in through the use of --set-string-* 
switches —e.g. --set-string-title=, --set-string-artist=, etc. See the 
rdimport(1) man page for the full list. This is the technique that many of the 
scripts in that Git repo use.


> Speaking of which, Fred do you have a formal schema (DTD) for the XML
> used in the export data? I recently had a partially broken Rivendell
> installation (my "lab" system) and wanted to preserve the library, so I
> made my own "exporter" and created the XML with a script (attached for
> reference).

Alas, no. I have been gradually extending my knowledge of XML/XSL theology over 
the past year or so, to the point where I can now more-or-less write DocBook 
markup reasonably well, and even hack a stylesheet or two. One of these days 
I’ll get around to DTDs.

Cheers!


|--|
| Frederick F. Gleason, Jr. |  Chief Developer |
|   |  Paravel Systems |
|--|
| "No, `Eureka!' is Greek for `This bath is too hot!'" |
| -- Dr. Who   |
|--|___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Jonathan Cohen]

Hi Fred,

 
In my tests I found that some metadata (the carts notes field springs to mind) 
did not transfer using the rdexport/rdimport method with its XML file. As a 
result I wrote a set of PHP scripts to export and import carts, or groups of 
carts, with also allows them to be moved between groups and renumbered etc. The 
code is rather messy but might be of use to someone:


 
http://radiotools.uk/download/rivendell-export-import-101.zip 
 

 
Best regards,

Jonathan

 
From: rivendell-dev-boun...@lists.rivendellaudio.org 
[mailto:rivendell-dev-boun...@lists.rivendellaudio.org] On Behalf Of Fred 
Gleason
Sent: 02 November 2018 15:59
To: rivendell-dev@lists.rivendellaudio.org
Subject: Re: [RDD] Importing and Exporting ranges of cuts from the library

 
On Nov 2, 2018, at 11:40, Andy Higginson mailto:a...@higgles.co.uk> > wrote:

 
The export and import would need to include a full set of meta data, including 
all of the talk times, segues etc.  It would however be a part of the library, 
and not the whole library.  Should this be done using rdexport/rdimport or 
should rivendell_filter be used?

 
You’re best bet would be to use rdexport(1) and then rdimport(1). Assuming that 
you use reasonably recent Rivendell (2.19.x) on both ends, you should get 100% 
transfer of the relevant metadata.

 



If using rdexport/rdimport, does some kindly soul have the correct command line 
syntax that I should be using for this?

 
The specifics will depend on precisely what you need to export and where you 
need it to go on the destination. There are man pages for both commands that 
should point you in the right direction. One suggestion: you probably want to 
defeat normalization [--normalization-level=0] on both the import and export 
(unless you have gross level errors in the original that you want to correct, 
in which case applying normalization on the import pass would be appropriate).

 
Cheers!

 
 
|--|


| Frederick F. Gleason, Jr. |              Chief Developer             |

|                           |              Paravel Systems             |

|--|


|          A room without books is like a body without a soul.         |

|                                         -- Cicero                    |

|--|



___



Rivendell-dev mailing list



Rivendell-dev@lists.rivendellaudio.org 
 



http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev 
 


___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[David Klann]

On 11/2/18 11:13 AM, Andy Higginson wrote:
> Thank you Fred and David.  You have both been most helpful.
> 
> One other quick question while I'm thinking about music libraries.  Is
> there a way to use a spreadsheet of music library data (talktime, segue
> points, hooks etc) with a Rivendell library?  For instance the data
> provided by Charlie Davy (http://www.charliedavy.co.uk/music-library/)
> and others.  It would be useful to take out some of the grunt work from
> setting up a library.
> 

You would likely need to export the CSV data and use an app to coerce it
into the acceptable XML format used in Rivendell. Not hard, but would
require attention to the details.

Speaking of which, Fred do you have a formal schema (DTD) for the XML
used in the export data? I recently had a partially broken Rivendell
installation (my "lab" system) and wanted to preserve the library, so I
made my own "exporter" and created the XML with a script (attached for
reference).

Thanks!

  ~David
#!/bin/zsh

zmodload zsh/datetime

. /usr/local/bin/zsh-functions

# mysql -e "describe CART"
# NUMBER  int(10) unsignedNO  PRI NULL
# TYPEint(10) unsignedNO  NULL
# GROUP_NAME  char(10)NO  MUL NULL
# TITLE   char(255)   YES MUL NULL
# ARTIST  char(255)   YES MUL NULL
# ALBUM   char(255)   YES MUL NULL
# YEARdateYES NULL
# CONDUCTOR   char(64)YES MUL NULL
# LABEL   char(64)YES MUL NULL
# CLIENT  char(64)YES MUL NULL
# AGENCY  char(64)YES MUL NULL
# PUBLISHER   char(64)YES MUL NULL
# COMPOSERchar(64)YES MUL NULL
# USER_DEFINEDchar(255)   YES MUL NULL
# SONG_ID char(32)YES MUL NULL
# BPM int(10) unsignedYES 0
# USAGE_CODE  int(11) YES 0
# FORCED_LENGTH   int(10) unsignedYES NULL
# AVERAGE_LENGTH  int(10) unsignedYES NULL
# LENGTH_DEVIATIONint(10) unsignedYES 0
# AVERAGE_SEGUE_LENGTHint(10) unsignedYES NULL
# AVERAGE_HOOK_LENGTH int(10) unsignedYES 0
# CUT_QUANTITYint(10) unsignedYES NULL
# LAST_CUT_PLAYED int(10) unsignedYES NULL
# PLAY_ORDER  int(10) unsignedYES NULL
# VALIDITYint(10) unsignedYES 2
# START_DATETIME  datetimeYES NULL
# END_DATETIMEdatetimeYES NULL
# ENFORCE_LENGTH  enum('N','Y')   YES N
# USE_WEIGHTING   enum('N','Y')   YES Y
# PRESERVE_PITCH  enum('N','Y')   YES N
# ASYNCRONOUS enum('N','Y')   YES N
# OWNER   char(64)YES MUL NULL
# MACROS  textYES NULL
# SCHED_CODES varchar(255)YES NULL
# NOTES   textYES NULL
# METADATA_DATETIME   datetimeYES MUL NULL
# USE_EVENT_LENGTHenum('N','Y')   YES N
# PENDING_STATION char(64)YES MUL NULL
# PENDING_PID int(11) YES NULL
# PENDING_DATETIMEdatetimeYES MUL NULL

# mysql -e "describe CUTS"
# CUT_NAMEchar(12)NO  PRI NULL
# CART_NUMBER int(10) unsignedNO  MUL NULL
# EVERGREEN   enum('N','Y')   YES N
# DESCRIPTION char(64)YES MUL NULL
# OUTCUE  char(64)YES MUL NULL
# ISRCchar(12)YES MUL NULL
# ISCIchar(32)YES MUL NULL
# LENGTH  int(10) unsignedYES NULL
# SHA1_HASH   char(40)YES MUL NULL
# ORIGIN_DATETIME datetimeYES MUL NULL
# START_DATETIME  datetimeYES MUL NULL
# END_DATETIMEdatetimeYES MUL NULL
# SUN enum('N','Y')   YES Y
# MON enum('N','Y')   YES Y
# TUE enum('N','Y')   YES Y
# WED enum('N','Y')   YES Y
# THU enum('N','Y')   YES Y
# FRI enum('N','Y')   YES Y
# SAT enum('N','Y')   YES Y
# START_DAYPART   timeYES NULL
# END_DAYPART timeYES NULL
# ORIGIN_NAME char(64)YES NULL
# ORIGIN_LOGIN_NAME   char(255)   YES NULL
# SOURCE_HOSTNAME char(255)   YES NULL
# WEIGHT  int(10) unsignedYES 1
# PLAY_ORDER  int(11) YES NULL
# LAST_PLAY_DATETIME  datetimeYES NULL
# UPLOAD_DATETIME datetimeYES NULL
# PLAY_COUNTERint(10) unsignedYES 0
# LOCAL_COUNTER   int(10) unsignedYES 0
# VALIDITYint(10) unsignedYES 2
# CODING_FORMAT   int(10) unsignedYES   

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Andy Higginson]

Food for thought there.  I'll look over it when I have some free time (and when 
the brain is a little less fuzzy). Thanks Andy  On Fri, 02 Nov 2018 
16:20:53 + Fred Gleason  wrote  On Nov 2, 
2018, at 12:13, Andy Higginson  wrote: One other quick 
question while I'm thinking about music libraries.  Is there a way to use a 
spreadsheet of music library data (talktime, segue points, hooks etc) with a 
Rivendell library?  For instance the data provided by Charlie Davy 
(http://www.charliedavy.co.uk/music-library/) and others.  It would be useful 
to take out some of the grunt work from setting up a library. There is, but it 
generally involves a bit of Perl ju-jitsu to make it practical. You might want 
to see https://github.com/ElvishArtisan/rivendell-importers for some ideas 
(particularly the ‘audio/‘ directory). Cheers! 
|--| | 
Frederick F. Gleason, Jr. |              Chief Developer             | |        
                   |              Paravel Systems             | 
|--| |      
    A room without books is like a body without a soul.         | |             
                            -- Cicero                    | 
|--| 
___ Rivendell-dev mailing list 
Rivendell-dev@lists.rivendellaudio.org 
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Fred Gleason]

On Nov 2, 2018, at 12:13, Andy Higginson  wrote:

> One other quick question while I'm thinking about music libraries.  Is there 
> a way to use a spreadsheet of music library data (talktime, segue points, 
> hooks etc) with a Rivendell library?  For instance the data provided by 
> Charlie Davy (http://www.charliedavy.co.uk/music-library/ 
> ) and others.  It would be 
> useful to take out some of the grunt work from setting up a library.

There is, but it generally involves a bit of Perl ju-jitsu to make it 
practical. You might want to see 
https://github.com/ElvishArtisan/rivendell-importers for some ideas 
(particularly the ‘audio/‘ directory).

Cheers!


|--|
| Frederick F. Gleason, Jr. |  Chief Developer |
|   |  Paravel Systems |
|--|
|  A room without books is like a body without a soul. |
| -- Cicero|
|--|___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[David Klann]

On 11/2/18 11:12 AM, Fred Gleason wrote:
> On Nov 2, 2018, at 12:03, David Klann  > wrote:
> 
>> rdexport --xml --format=FLAC --carts=START1:END1 --carts=START2:END2
>> /tmp/foo
>> ...
> 
> If the intended destination of the exported files is rdimport(1), then
> the --xml switch is superfluous. The exported files contain the very
> same XML embedded in the metadata fields, and rdimport(1) will see that
> and populate the metadata in newly created carts accordingly. The --xml
> switch is primarily intended for situations where you need the metadata
> for some external, non-Rivendell system.

Oh! Thanks for that reminder Fred! I'd forgotten that you added that to
the export a while back.

  ~David



signature.asc
Description: OpenPGP digital signature
___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Fred Gleason]

On Nov 2, 2018, at 12:03, David Klann  wrote:

> I have also imported those previously-exported files into a different
> Rivendell instance. My experience has been that the GROUP and CART
> number in the XML files are ignored when using `rdimport --xml ...`.
> There may be a switch I'm missing to force use of those specific XML
> values, but it has not been a problem for me...

GROUP and CART are both determined by the parameters provided to rdimport(1), 
hence the values in the XML are intentionally ignored.

Cheers!


|--|
| Frederick F. Gleason, Jr. |  Chief Developer |
|   |  Paravel Systems |
|--|
|  A room without books is like a body without a soul. |
| -- Cicero|
|--|___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Andy Higginson]

Thank you Fred and David.  You have both been most helpful. One other quick 
question while I'm thinking about music libraries.  Is there a way to use a 
spreadsheet of music library data (talktime, segue points, hooks etc) with a 
Rivendell library?  For instance the data provided by Charlie Davy 
(http://www.charliedavy.co.uk/music-library/) and others.  It would be useful 
to take out some of the grunt work from setting up a library. Thanks Andy  
On Fri, 02 Nov 2018 16:03:39 + David Klann  wrote  On 
11/2/18 10:40 AM, Andy Higginson wrote: > Hi, > > I've been asked to look at 
the options of exporting a range of audio > cuts from one library and then 
importing them into a different stand > alone machine.  The export and import 
would need to include a full set > of meta data, including all of the talk 
times, segues etc.  It would > however be a part of the library, and not the 
whole library.  Should > this be done using rdexport/rdimport or should 
rivendell_filter be used? > > If using rdexport/rdimport, does some kindly soul 
have the correct > command line syntax that I should be using for this? > Hi 
Andy, I have not yet found a need to use rivendell_filter, but I *have* used 
(something like): rdexport --xml --format=FLAC --carts=START1:END1 
--carts=START2:END2 /tmp/foo (where /tmp/foo is the directory that will hold 
the FLAC and XML files) to export multiple, contiguous ranges of carts. The use 
of the "--xml" switch enables rdexport to save pretty much all of the metadata 
(including markers) in XML files next to the FLAC files (same output 
directory). I have also imported those previously-exported files into a 
different Rivendell instance. My experience has been that the GROUP and CART 
number in the XML files are ignored when using `rdimport --xml ...`. There may 
be a switch I'm missing to force use of those specific XML values, but it has 
not been a problem for me... Hope this helps! ~David Klann___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Fred Gleason]

On Nov 2, 2018, at 12:03, David Klann  wrote:

> rdexport --xml --format=FLAC --carts=START1:END1 --carts=START2:END2
> /tmp/foo
> 
> (where /tmp/foo is the directory that will hold the FLAC and XML files)
> to export multiple, contiguous ranges of carts. The use of the "--xml"
> switch enables rdexport to save pretty much all of the metadata
> (including markers) in XML files next to the FLAC files (same output
> directory).

If the intended destination of the exported files is rdimport(1), then the 
--xml switch is superfluous. The exported files contain the very same XML 
embedded in the metadata fields, and rdimport(1) will see that and populate the 
metadata in newly created carts accordingly. The --xml switch is primarily 
intended for situations where you need the metadata for some external, 
non-Rivendell system.

Cheers!


|--|
| Frederick F. Gleason, Jr. |  Chief Developer |
|   |  Paravel Systems |
|--|
|  A room without books is like a body without a soul. |
| -- Cicero|
|--|___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[David Klann]

On 11/2/18 10:40 AM, Andy Higginson wrote:
> Hi,
> 
> I've been asked to look at the options of exporting a range of audio
> cuts from one library and then importing them into a different stand
> alone machine.  The export and import would need to include a full set
> of meta data, including all of the talk times, segues etc.  It would
> however be a part of the library, and not the whole library.  Should
> this be done using rdexport/rdimport or should rivendell_filter be used?
> 
> If using rdexport/rdimport, does some kindly soul have the correct
> command line syntax that I should be using for this?
> 

Hi Andy,

I have not yet found a need to use rivendell_filter, but I *have* used
(something like):

rdexport --xml --format=FLAC --carts=START1:END1 --carts=START2:END2
/tmp/foo

(where /tmp/foo is the directory that will hold the FLAC and XML files)
to export multiple, contiguous ranges of carts. The use of the "--xml"
switch enables rdexport to save pretty much all of the metadata
(including markers) in XML files next to the FLAC files (same output
directory).

I have also imported those previously-exported files into a different
Rivendell instance. My experience has been that the GROUP and CART
number in the XML files are ignored when using `rdimport --xml ...`.
There may be a switch I'm missing to force use of those specific XML
values, but it has not been a problem for me...

Hope this helps!

  ~David Klann




signature.asc
Description: OpenPGP digital signature
___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] Importing and Exporting ranges of cuts from the library

[Fred Gleason]

On Nov 2, 2018, at 11:40, Andy Higginson  wrote:

> The export and import would need to include a full set of meta data, 
> including all of the talk times, segues etc.  It would however be a part of 
> the library, and not the whole library.  Should this be done using 
> rdexport/rdimport or should rivendell_filter be used?

You’re best bet would be to use rdexport(1) and then rdimport(1). Assuming that 
you use reasonably recent Rivendell (2.19.x) on both ends, you should get 100% 
transfer of the relevant metadata.


> If using rdexport/rdimport, does some kindly soul have the correct command 
> line syntax that I should be using for this?

The specifics will depend on precisely what you need to export and where you 
need it to go on the destination. There are man pages for both commands that 
should point you in the right direction. One suggestion: you probably want to 
defeat normalization [--normalization-level=0] on both the import and export 
(unless you have gross level errors in the original that you want to correct, 
in which case applying normalization on the import pass would be appropriate).

Cheers!


|--|
| Frederick F. Gleason, Jr. |  Chief Developer |
|   |  Paravel Systems |
|--|
|  A room without books is like a body without a soul. |
| -- Cicero|
|--|___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] re-normalize a batch of carts?

[Marius Wikstøl]

There's a project called bs1770gain which normalizes audio to a set 
lufs, loudness.


Marius

On 11/2/18 4:27 PM, Andy Higginson wrote:
I seem to remember some discussion on using EBU R128 loudness 
measuring for normalization of audio.  It was mentioned a couple of 
years ago and I think someone on this list had found some open source 
code to enable this monitoring.  Has anything further happened on this?




 On Thu, 01 Nov 2018 15:43:07 + *Fred Gleason 
* wrote 


On Nov 1, 2018, at 11:27, Cowboy mailto:c...@cwf1.com>> wrote:

*THIS* is why 12 db or more of headroom is a good idea.
Personally, I would not have chosen 13, because I prefer even
numbers, but any
"reasonable" number should be fine.


There’s been lots of debate about this over the years. Many
American vendors have standardized with -12, whereas our friends
over the Atlantic seem to favor -18 or even -20.  In Rivendell,
the ‘reference level' is actually -16 dBFS (that being the ‘0’
point on Rivendell’s audio meters). That number was chosen mostly
because a certain proprietary automation system to which I had
access at the time (ca. 2002) used it as well, thus making it easy
to use audio from that system for test cases.

Cheers!


|--|
| Frederick F. Gleason, Jr. |              Chief Developer       |
|                 |              Paravel Systems       |
|--|
| A successful tool is one that was used to do something undreamed
of  |
| by its author.           |
|                               -- S.C. Johnson         |
|--|

___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org

http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev




___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev
___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

[RDD] Importing and Exporting ranges of cuts from the library

[Andy Higginson]

Hi, I've been asked to look at the options of exporting a range of audio cuts 
from one library and then importing them into a different stand alone machine.  
The export and import would need to include a full set of meta data, including 
all of the talk times, segues etc.  It would however be a part of the library, 
and not the whole library.  Should this be done using rdexport/rdimport or 
should rivendell_filter be used? If using rdexport/rdimport, does some kindly 
soul have the correct command line syntax that I should be using for this? 
Thanks Andy___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev

Re: [RDD] re-normalize a batch of carts?

[Andy Higginson]

I seem to remember some discussion on using EBU R128 loudness measuring for 
normalization of audio.  It was mentioned a couple of years ago and I think 
someone on this list had found some open source code to enable this monitoring. 
 Has anything further happened on this?  On Thu, 01 Nov 2018 15:43:07 + 
Fred Gleason  wrote  On Nov 1, 2018, at 11:27, 
Cowboy  wrote: *THIS* is why 12 db or more of headroom is a good 
idea. Personally, I would not have chosen 13, because I prefer even numbers, 
but any "reasonable" number should be fine. There’s been lots of debate about 
this over the years. Many American vendors have standardized with -12, whereas 
our friends over the Atlantic seem to favor -18 or even -20.  In Rivendell, the 
‘reference level' is actually -16 dBFS (that being the ‘0’ point on Rivendell’s 
audio meters). That number was chosen mostly because a certain proprietary 
automation system to which I had access at the time (ca. 2002) used it as well, 
thus making it easy to use audio from that system for test cases. Cheers! 
|--| | 
Frederick F. Gleason, Jr. |              Chief Developer             | |        
                   |              Paravel Systems             | 
|--| | A 
successful tool is one that was used to do something undreamed of  | | by its 
author.                                                       | |               
                         -- S.C. Johnson               | 
|--| 
___ Rivendell-dev mailing list 
Rivendell-dev@lists.rivendellaudio.org 
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev___
Rivendell-dev mailing list
Rivendell-dev@lists.rivendellaudio.org
http://caspian.paravelsystems.com/mailman/listinfo/rivendell-dev