With the patch this time...
Allan N. Snider wrote: > This patch adds the nextsub filter, (a replacement for extsub). It > allows better control over which subtitle indexes can be used, better > positioning control, allows opacity, zooming, rgb colours, etc. It's > well documented (something extsub was in need of). This is an old > patch but I see it still didn't make it in, so I'm resubmitting it. > > YES yes, I know. I need to determine these subtitle indices from > the VOB header and make them available to dvdrip. Been on my todo > list for two years now. That, and the "forced" flag. What can I say. > > Allan > >
diff -ruN old/filter/extsub/Makefile.am new/filter/extsub/Makefile.am --- old/filter/extsub/Makefile.am 2007-03-11 22:25:51.000000000 -0400 +++ new/filter/extsub/Makefile.am 2007-03-11 22:24:26.000000000 -0400 @@ -7,12 +7,12 @@ pkgdir = $(MOD_PATH) -pkg_LTLIBRARIES = filter_extsub.la filter_extsub2.la +pkg_LTLIBRARIES = filter_extsub.la filter_nextsub.la filter_extsub_la_SOURCES = filter_extsub.c subproc.c subtitle_buffer.c filter_extsub_la_LDFLAGS = -module -avoid-version -filter_extsub2_la_SOURCES = filter_extsub.c subproc.c subtitle_buffer.c -filter_extsub2_la_LDFLAGS = -module -avoid-version +filter_nextsub_la_SOURCES = filter_nextsub.c subproc.c subtitle_buffer.c +filter_nextsub_la_LDFLAGS = -module -avoid-version -EXTRA_DIST = subproc.h subtitle_buffer.h +EXTRA_DIST = subproc.h subtitle_buffer.h nextsub.h diff -ruN old/filter/extsub/filter_nextsub.c new/filter/extsub/filter_nextsub.c --- old/filter/extsub/filter_nextsub.c 1969-12-31 19:00:00.000000000 -0500 +++ new/filter/extsub/filter_nextsub.c 2007-03-11 22:43:35.000000000 -0400 @@ -0,0 +1,1272 @@ +/* + * filter_nextsub.c + * + * Copyright (C) Thomas Oestreich - January 2002 + * Copyright (C) Allan Snider - March 2007 + * + * This file is part of transcode, a video stream processing tool + * + * transcode is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2, or (at your option) + * any later version. + * + * transcode is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Make; see the file COPYING. If not, write to + * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA. + * + */ + +/* + * v0.3.5 -> v0.4.0: (Allan Snider) + * - code style cleanup + * - add opacity and rendering color for each index + * - add zoom flag + * - crop subtitle, (for consistent placement) + * - modified positioning behaviour + * - better (?) index guessing + */ + +#define MOD_NAME "filter_nextsub.so" +#define MOD_VERSION "0.4.0 (2007-03-01)" +#define MOD_CAP "DVD subtitle overlay plugin" +#define MOD_AUTHOR "Thomas Oestreich / Allan Snider" + +#include "nextsub.h" + +static MyFilterData *mfd; + + +/* + * help_opstr: + */ + +static void +help_optstr( void ) +{ + tc_log_info( MOD_NAME, "(%s) help:\n\n" + "* Overview:\n" + "\n" + " DVD subtitle overlay plugin.\n" + " Original code by Thomas Destreich,\n" + " Enhancements added by Allan Snider.\n" + "\n" + " A subtitle bitmap contains 4 index values, background, text body,\n" + " inner outline, and outer outline. The ordering of the indices vary\n" + " from DVD to DVD. The background will always be transparent. One can\n" + " specify opacity values (0-1) and rendering colors (hex rrggbb) for each\n" + " index (1-3). The bitmap can be anti-aliased, and zoomed (XxY). The\n" + " resulting bitmap is then cropped giving control of where the subtitle\n" + " is located, regardless of where the original bitmap had placed it.\n" + "\n" + " Occasionally, a subtitle component (eg. text body), will be composed\n" + " of more than one index. One could assign them the same opacity and\n" + " color, however, each index is anti-aliased and zoomed individually\n" + " which would produce poor results. Therefore, the 'map' option was\n" + " added, which allows one index value to be remapped to another. This\n" + " provides a mechanism to join two index values into one component.\n" + "\n" + " If no opacity arguments are given, the highest frequency (non-background)\n" + " subtitle index is assumed to be the body of the text, and the next\n" + " highest, the inner text outline. Both indices are assigned an opacity\n" + " of 1.0. The text body is assigned the default color and the outline is\n" + " assigned as black. The index frequencies are calculated from the first\n" + " subtitle rendered.\n" + "\n" + " For positioning, the default (pos=0,topalign=0) places the bottom line of\n" + " the subtitle at the bottom line of the video frame. Positive values move\n" + " the subtitle up by the given number of rows. If topalign=1, then the\n" + " top line of the subtitle is placed at the position given (and should be\n" + " at least the height of the subtitle, else it will be clipped).\n" + "\n" + " For boolean arguments, a value of TRUE is assumed if the option is\n" + " present but no value is given. For opacity arguments, a value\n" + " of 1.0 is assumed if the option is given with no value.\n" + "\n" + "* Options:\n" + "\n" + " 'track' Subtitle track (0-255) [0]\n" + " 'forced' Render only forced subtitles (0=off, 1=on) [0]\n" + " 'pos' Vertical position of subtitle (0-height) [0]\n" + " 'tshift' Start time correction (msec) [0]\n" + " 'pre' Run as a pre filter (0=no, 1=yes) [0]\n" + " 'aalias' Anti-aliasing (0=off, 1=on) [0]\n" + " 'zoom' Zoom subtitle image (X[xY]) [1x1]\n" + " 'o1' Index 1 opacity (0-1) [0]\n" + " 'o2' Index 2 opacity (0-1) [0]\n" + " 'o3' Index 3 opacity (0-1) [0]\n" + " 'c1' Index 1 rendering color (rr[ggbb]) [clr]\n" + " 'c2' Index 2 rendering color (rr[ggbb]) [clr]\n" + " 'c3' Index 3 rendering color (rr[ggbb]) [clr]\n" + " 'clr' Default rendering color (rr[ggbb]) [ffffff]\n" + " 'map' Remap an index value (ItoJ) [0to0]\n" + "\n" + "* Examples:\n" + "\n" + " -J nextsub\n" + "\n" + " Let the filter guess the indices, rendering text with the default\n" + " color (white) and the outline as black. Both are assigned an\n" + " opacity of 1.0. You should get something visible, either solid or\n" + " outlined text, rendered white, and at the very bottom of the screen.\n" + "\n" + " -J nextsub=clr=d0d000:zoom=0.8\n" + "\n" + " Similar to the above, but use yellow as the default color and shrink\n" + " the subtitle by 0.8 x 0.8.\n" + "\n" + " -J nextsub=o1:o2=0.9:c1=d0d000:c2=00:pos=75:top:aalias:zoom=0.9x0.75\n" + "\n" + " Specify explicit subtitle indices to use. Index 1 assumes an\n" + " opacity of 1.0, and index 2 is specified as 0.9. Render index 1\n" + " with yellow, and index 2 with black. Top align the subtitle at\n" + " row 75. Anti-alias it, and zoom to 0.9 x 0.75.\n" + "\n" + " The above zoom factors can be a good choice for some 16:9 films\n" + " when zooming a 720x480 frame. The subtitle is overlayed in the post\n" + " filter phase and sometimes we need a similar aspect ratio correction\n" + " applied to the subtitle bitmap as well. Alternatively, run it as\n" + " a pre-filter.\n" + "\n" + " -J nextsub=o2:map=3to2:forced\n" + "\n" + " Join indices 2 and 3 together to render the text body as index 2.\n" + " In addition, only render subtitles which have the forced flag set.\n" + "\n" + " -J nextsub=o1:o2:o3:c1=ff0000:c2=00ff00:c3=0000ff\n" + "\n" + " Render all indices with a different color. Encoding just a small\n" + " number of frames where a subtitle is present, this can be useful\n" + " for determining what components of the subtitle each index has\n" + " been assigned for a particular DVD.\n" + "\n" + , MOD_CAP ); +} + + +/* + * tc_filter: + * Single function interface. + */ + +int +tc_filter( frame_list_t *ptr_, char *opt ) +{ + vframe_list_t *ptr = (vframe_list_t*) ptr_; + + + //---------------------------------- + // + // filter init + // + //---------------------------------- + + if ( ptr->tag & TC_FILTER_INIT ) { + int n, i; + + mfd = tc_zalloc( sizeof(MyFilterData) ); + if ( !mfd ) { + tc_log_perror( MOD_NAME, "out of memory" ); + return( TC_EXPORT_ERROR ); + } + + mfd->vob = tc_get_vob(); + if ( !mfd->vob ) + return( -1 ); + + mfd->tcvhandle = tcv_init(); + if ( !mfd->tcvhandle ) { + tc_log_error( MOD_NAME, "tcv initialization failed" ); + return( TC_EXPORT_ERROR ); + } + + mfd->sub = tc_malloc( SIZE_RGB_FRAME ); + mfd->tmp = tc_malloc( SIZE_RGB_FRAME ); + if ( !mfd->sub || !mfd->tmp ) { + tc_log_perror( MOD_NAME, "out of memory" ); + return( TC_EXPORT_ERROR ); + } + + /* split sub and tmp into individual planes */ + n = TC_MAX_V_FRAME_WIDTH * TC_MAX_V_FRAME_HEIGHT; + for ( i=0; i<3; i++ ) { + mfd->sp[i] = mfd->sub + i*n; + mfd->dp[i] = mfd->tmp + i*n; + } + + mfd->clr_def.r = 255; + mfd->clr_def.g = 255; + mfd->clr_def.b = 255; + for ( i=0; i<3; i++ ) { + mfd->clr[i].r = -1; + mfd->clr[i].g = -1; + mfd->clr[i].b = -1; + } + + mfd->pts1 = -1; + mfd->pts2 = -1; + + if ( opt != NULL ) { + char buf[100]; + int n; + + if ( verbose ) + tc_log_info( MOD_NAME, "options=%s", opt ); + + /* booleans, default to TRUE if no value is given */ + n = optstr_get( opt, "forced", "%d", &mfd->forced ); + if ( !n ) + mfd->forced = TC_TRUE; + n = optstr_get( opt, "top", "%d", &mfd->topalign ); + if ( !n ) + mfd->topalign = TC_TRUE; + n = optstr_get( opt, "pre", "%d", &mfd->prefilter ); + if ( !n ) + mfd->prefilter = TC_TRUE; + n = optstr_get( opt, "aalias", "%d", &mfd->antialias ); + if ( !n ) + mfd->antialias = TC_TRUE; + + optstr_get( opt, "track", "%d", &mfd->vob->s_track ); + optstr_get( opt, "pos", "%d", &mfd->pos ); + optstr_get( opt, "tshift", "%d", &mfd->tshift ); + + if ( optstr_get(opt,"zoom","%[^:]",buf) >= 0 ) { + if ( get_zoom(buf,&mfd->z_x,&mfd->z_y) ) + mfd->do_zoom = 1; + } + + /* default these to 1.0 if no value is given */ + n = optstr_get( opt, "o1", "%lf", &mfd->fopc[0] ); + if ( !n ) + mfd->fopc[0] = 1.0; + n = optstr_get( opt, "o2", "%lf", &mfd->fopc[1] ); + if ( !n ) + mfd->fopc[1] = 1.0; + n = optstr_get( opt, "o3", "%lf", &mfd->fopc[2] ); + if ( !n ) + mfd->fopc[2] = 1.0; + + if ( optstr_lookup(opt,"o1") || + optstr_lookup(opt,"o2") || + optstr_lookup(opt,"o3") ) + mfd->have_indices = 1; + + if ( optstr_get(opt,"c1","%[^:]",buf) > 0 ) + mfd->clr[0] = get_rgb( buf ); + if ( optstr_get(opt,"c2","%[^:]",buf) > 0 ) + mfd->clr[1] = get_rgb( buf ); + if ( optstr_get(opt,"c3","%[^:]",buf) > 0 ) + mfd->clr[2] = get_rgb( buf ); + if ( optstr_get(opt,"clr","%[^:]",buf) > 0 ) + mfd->clr_def = get_rgb( buf ); + + optstr_get( opt, "map", "%dto%d", &mfd->remap[0], &mfd->remap[1] ); + + if ( optstr_lookup(opt,"help") ) + help_optstr(); + } + + /* assign default for unset colors */ + for ( i=0; i<3; i++ ) + if ( mfd->clr[i].r == -1 ) + mfd->clr[i] = mfd->clr_def; + + /* convert opacities from 0-1 to 0-255 */ + for ( i=0; i<3; i++ ) { + int o; + + o = mfd->fopc[i] * 255; + if ( o < 0 ) + o = 0; + if ( o > 255 ) + o = 255; + + mfd->opc[i] = o; + } + + if ( verbose & TC_DEBUG ) { + tc_log_info( MOD_NAME, "Option values:" ); + tc_log_info( MOD_NAME, " forced: %d", mfd->forced ); + tc_log_info( MOD_NAME, " pos: %d", mfd->pos ); + tc_log_info( MOD_NAME, " top: %d", mfd->topalign ); + tc_log_info( MOD_NAME, " tshift: %d", mfd->tshift ); + tc_log_info( MOD_NAME, " pre: %d", mfd->prefilter ); + tc_log_info( MOD_NAME, " aalias: %d", mfd->antialias ); + tc_log_info( MOD_NAME, " zoom: %s", pr_zoom() ); + for ( i=0; i<3; i++ ) + tc_log_info( MOD_NAME, " o%d: %g", i+1, mfd->fopc[i] ); + for ( i=0; i<3; i++ ) + tc_log_info( MOD_NAME, " c%d: %s", i+1, pr_clr(mfd->clr[i]) ); + tc_log_info( MOD_NAME, " clr: %s", pr_clr(mfd->clr_def) ); + tc_log_info( MOD_NAME, " map: %d to %d", mfd->remap[0], mfd->remap[1] ); + } + + // start subtitle stream + mfd->import.flag = TC_SUBEX; + if ( tcv_import(TC_IMPORT_OPEN,&mfd->import,mfd->vob) < 0 ) + tc_log_error( MOD_NAME, "popen subtitle stream" ); + + subproc_init( NULL, "title", 0, (unsigned short) mfd->vob->s_track ); + sframe_alloc( SUBTITLE_BUFFER, mfd->import.fd ); + + // start thread + if ( pthread_create(&mfd->thread,NULL,(void*)subtitle_reader,NULL) != 0 ) + tc_log_error( MOD_NAME, "failed to start subtitle import thread" ); + + mfd->f_time = 1.0 / (mfd->prefilter ? mfd->vob->fps : mfd->vob->ex_fps); + + if ( verbose ) + tc_log_info( MOD_NAME, "%s %s", MOD_VERSION, MOD_CAP ); + + return( 0 ); + } + + if ( ptr->tag & TC_FILTER_GET_CONFIG ) { + optstr_filter_desc( opt, MOD_NAME, MOD_CAP, MOD_VERSION, MOD_AUTHOR, "VRYOE", "1" ); + optstr_param( opt, "track", "Subtitle track", "%d", "0", "0", "255" ); + optstr_param( opt, "forced", "Render only forced subtitles", "%d", "0", "0", "1" ); + optstr_param( opt, "pos", "Vertical position of subtitle", "%d", "0", "0", "height" ); + optstr_param( opt, "tshift", "Start time correction (msec)", "%d", "0", "0", "-1" ); + optstr_param( opt, "pre", "Run as a pre filter", "%d", "0", "0", "1" ); + optstr_param( opt, "aalias", "Anti-aliasing", "%d", "1", "0", "1" ); + optstr_param( opt, "zoom", "Zoom subtitle image (XxY)", "%s", "1x1" ); + optstr_param( opt, "o1", "Index 1 opacity", "%f", "1", "0", "1" ); + optstr_param( opt, "o2", "Index 2 opacity", "%f", "1", "0", "1" ); + optstr_param( opt, "o3", "Index 3 opacity", "%f", "1", "0", "1" ); + optstr_param( opt, "c1", "Index 1 rendering color", "%s", "ffffff" ); + optstr_param( opt, "c2", "Index 2 rendering color", "%s", "ffffff" ); + optstr_param( opt, "c3", "Index 3 rendering color", "%s", "ffffff" ); + optstr_param( opt, "clr", "Default rendering color (hex rrggbb)", "%s", "ffffff" ); + optstr_param( opt, "map", "Remap an index value (ItoJ)", "%dto%d", "0to0" ); + + return( 0 ); + } + + + //---------------------------------- + // + // filter close + // + //---------------------------------- + + if ( ptr->tag & TC_FILTER_CLOSE ) { + static int close_done; + void *status; + + if ( !close_done ) { + pthread_cancel( mfd->thread ); + +#ifdef BROKEN_PTHREADS + pthread_cond_signal( &sframe_list_full_cv ); +#endif + + pthread_join( mfd->thread, &status ); + + mfd->import.flag = TC_SUBEX; + if ( mfd->import.fd != NULL ) + pclose( mfd->import.fd ); + + tcv_free( mfd->tcvhandle ); + free( mfd->sub ); + free( mfd->tmp ); + + free( mfd ); + + close_done = TC_TRUE; + } + + return( 0 ); + } + + //---------------------------------- + // + // filter frame routine + // + //---------------------------------- + + if ( ptr->tag & TC_VIDEO ) { + double f_pts; + + if ( mfd->prefilter ) { + if ( !(ptr->tag & TC_PRE_S_PROCESS) ) + return( 0 ); + } else { + if ( !(ptr->tag & TC_POST_S_PROCESS) ) + return( 0 ); + } + + if ( verbose & TC_STATS ) + tc_log_info( MOD_NAME, "%s/%s %s %s", + mfd->vob->mod_path, MOD_NAME, MOD_VERSION, MOD_CAP ); + + + //------------------------------------------------------------------------- + // + // below is a very fuzzy concept of rendering the subtitle on the movie + // + // (1) check if we have a valid subtitle and render it + // (2) if (1) fails try to get a new one + // (3) buffer and display the new one if it's showtime, if not return + // + // repeat steps throughout the movie + + // calculate current frame video PTS in [s] + // adjust for dropped frames so far + // adjust for user shift (in milliseconds) + + f_pts = mfd->f_time * (ptr->id-tc_get_frames_dropped()+mfd->vob->psu_offset) + + mfd->tshift / 1000.0; + + if ( verbose & TC_DEBUG ) + tc_log_info( MOD_NAME, "frame=%06d pts=%.3f sub1=%.3f sub2=%.3f", + ptr->id, f_pts, mfd->pts1, mfd->pts2 ); + + /* TODO: weird logic here, but going to leave it for now */ + + // overlay now? + if ( mfd->pts1<=f_pts && f_pts<=mfd->pts2 ) { + if ( !mfd->forced || mfd->fflg ) + subtitle_overlay( ptr->video_buf, ptr->v_width, ptr->v_height ); + return( 0 ); + } + + // get a new subtitle, if the last one has expired: + if ( f_pts > mfd->pts2 ) { + mfd->made_subtitle = 0; + if ( subtitle_retrieve() < 0 ) { + if ( verbose & TC_DEBUG ) + tc_log_info( MOD_NAME, "no subtitle available at this time" ); + return( 0 ); + } + } + + // overlay now? + if ( mfd->pts1<f_pts && f_pts<mfd->pts2 ) + if ( !mfd->forced || mfd->fflg ) + subtitle_overlay( ptr->video_buf, ptr->v_width, ptr->v_height ); + } + + return( 0 ); +} + + +/* + * get_zoom: + * Given a string of the form XxY, where X and Y are floating point + * numbers, return (via parameter) the two scaling values supplied as doubles. + * If only a single value is given, Y assumes the value of X. + */ + +static int +get_zoom( char *buf, double *px, double *py ) +{ + double x, y; + char *p; + + x = atof( buf ); + if ( x <= 0 ) + return( TC_FALSE ); + + p = strchr( buf, 'x' ); + if ( p ) { + y = atof( p + 1 ); + if ( y <= 0 ) + return( TC_FALSE ); + } else + y = x; + + *px = x; + *py = y; + + return( TC_TRUE ); +} + + +/* + * get_rgb: + * Given a string of the form (hex) "rrggbb", return an Rgb structure + * containing the (integer) tuplet [r,g,b]. If only an "rr" component is + * given, gg, and bb, assume the value of rr. For example, "ff" would be + * interpreted as white (ffffff). Return the structure by function value. + */ + +static Rgb +get_rgb( char *buf ) +{ + Rgb c; + int v; + + if ( strlen(buf) < 6 ) { + if ( strlen(buf) < 2 ) + v = 255; + else + v = htoi( buf ); + c.r = v; + c.g = v; + c.b = v; + + return( c ); + } + + c.r = htoi( buf+0 ); + c.g = htoi( buf+2 ); + c.b = htoi( buf+4 ); + + return( c ); +} + + +/* + * htoi: + * Convert a 2 digit hex number to integer, and return via function + * value. + */ + +static int +htoi( char *buf ) +{ + return( (htob(buf[0]) << 4) + htob(buf[1]) ); +} + + +/* + * htob: + * Convert a hex digit to integer, and return via function value. + */ + +static int +htob( char c ) +{ + c = tolower( c ); + if ( c>='a' && c<='f' ) + return( c-'a'+10 ); + return( c-'0' ); +} + + +/* + * pr_zoom: + * Return the zoom x, and y scaling values as a (static) string of + * the form "XxY", via function value. + */ + +static char * +pr_zoom( void ) +{ + static char buf[100]; + + if ( mfd->do_zoom ) + tc_snprintf( buf, 100, "%gx%g", mfd->z_x, mfd->z_y ); + else + strcpy( buf, "none" ); + + return( buf ); +} + + +/* + * pr_clr: + * Return the given Rgb structure as a string of the form (hex) "rrggbb". + */ + +static char * +pr_clr( Rgb c ) +{ + static char buf[100]; + + tc_snprintf( buf, 100, "%02x%02x%02x", c.r, c.g, c.b ); + return( buf ); +} + + +/* + * subtitle_retrieve: + * Retrieve the next subtitle from the subtitle reader thread. + * Store the results in mfd: + * + * sub - subtitle bitmap + * id - frame number + * pts1 - start time + * pts2 - end time + * fflg - forced flag + * xp - suggested x position + * xp - suggested y position + * sw - subtitle width + * sh - subtitle height + * + * The function returns 0 if successful, -1 otherwise. + */ + +static int +subtitle_retrieve( void ) +{ + sframe_list_t *sptr = NULL; + sub_info_t sub; + + // get a subtitle from buffer + pthread_mutex_lock( &sframe_list_lock ); + if ( sframe_fill_level(TC_BUFFER_EMPTY) ) { + pthread_mutex_unlock( &sframe_list_lock ); + return( -1 ); + } + + // not being empty does not mean ready to go!!!!! + if ( sframe_fill_level(TC_BUFFER_READY) ) { + pthread_mutex_unlock( &sframe_list_lock ); + if ( (sptr=sframe_retrieve()) == NULL ) { + // this shouldn't happen + tc_log_error( MOD_NAME, "internal error (S)" ); + return( -1 ); + } + } else { + pthread_mutex_unlock( &sframe_list_lock ); + return( -1 ); + } + + // conversion + sub.frame = mfd->sub; + if ( subproc_feedme(sptr->video_buf,sptr->video_size,sptr->id,sptr->pts,&sub) < 0 ) { + // problems, drop this subtitle + if ( verbose & TC_DEBUG ) + tc_log_warn( MOD_NAME, "subtitle dropped" ); + sframe_remove( sptr ); + pthread_cond_signal( &sframe_list_full_cv ); + return( -1 ); + } + + // save data + mfd->id = sptr->id; + mfd->pts1 = sptr->pts * mfd->f_time; + mfd->pts2 = mfd->pts1 + sub.time / 100.0; + + mfd->fflg = sub.forced; + mfd->orig.x = sub.x; + mfd->orig.y = sub.y; + mfd->sz.x = sub.w; + mfd->sz.y = sub.h; + + // release packet buffer + sframe_remove( sptr ); + pthread_cond_signal( &sframe_list_full_cv ); + + if ( verbose & TC_DEBUG ) + tc_log_info( MOD_NAME, "got SUBTITLE %d with forced=%d, pts=%.3f dtime=%.3f", + mfd->id, mfd->fflg, mfd->pts1, mfd->pts2-mfd->pts1 ); + + return( 0 ); +} + + +/* + * subtitle_overlay: + * The current subtitle is within the time window of the current + * video frame. Overlay it into the image. If this is the first time + * the subtitle is being rendered, anti-alias it, zoom it, and crop it. + * If this is the very first subtitle rendered, and no opacity arguments + * were given, then guess appropriate indices to use. Finally, determine + * the subtitle position and overlay the resulting bitmap by blending pixel + * values according to opacity. + */ + +static void +subtitle_overlay( uint8_t *vid, int w, int h ) +{ + if ( verbose & TC_DEBUG ) + tc_log_info( MOD_NAME, "SUBTITLE id=%d, x=%d, y=%d, w=%d, h=%d, t=%f", + mfd->id, mfd->orig.x, mfd->orig.y, mfd->sz.x, mfd->sz.y, + mfd->pts2 - mfd->pts1 ); + + if ( !mfd->have_indices ) { + /* this is only ever done once, for the first subtitle rendered */ + mfd->have_indices = TC_TRUE; + guess_indices(); + } + + /* anti-alias, zoom, and crop the subtitle */ + if ( !mfd->made_subtitle ) { + /* this is done once for each subtitle rendered */ + mfd->made_subtitle = TC_TRUE; + make_subtitle( w, h ); + } + + if ( mfd->vob->im_v_codec == CODEC_RGB ) + subtitle_overlay_rgb( vid, w, h ); + + if ( mfd->vob->im_v_codec == CODEC_YUV ) + subtitle_overlay_yuv( vid, w, h ); +} + + +/* + * guess_indices: + * This is called if no opacity arguments were given. The highest + * frequency (non-background) subtitle index is assumed to be the body of + * the text, and the next highest the inner text outline. Both indices + * are assigned an opacity of 1.0. The text body is assigned the default + * color, and the outline is assigned as black. + */ + +static void +guess_indices( void ) +{ + int subi[4]; + int ib, io; + int n, i; + + for ( i=0; i<4; i++ ) + subi[i] = 0; + + n = mfd->sz.x * mfd->sz.y; + for ( i=0; i<n; i++ ) + subi[ get_sub_index(i) ]++; + + ib = 1; + io = 2; + if ( subi[1]>subi[2] && subi[1]>subi[3] ) { + ib = 1; + io = (subi[2]>subi[3]) ? 2 : 3; + } + if ( subi[2]>subi[1] && subi[2]>subi[3] ) { + ib = 2; + io = (subi[1]>subi[3]) ? 1 : 3; + } + if ( subi[3]>subi[1] && subi[3]>subi[2] ) { + ib = 3; + io = (subi[1]>subi[2]) ? 1 : 2; + } + + --ib; + --io; + + mfd->opc[ib] = 255; + mfd->opc[io] = 255; + + mfd->clr[io].r = 0; + mfd->clr[io].g = 0; + mfd->clr[io].b = 0; + + if ( verbose & TC_INFO ) + tc_log_info( MOD_NAME, "index counts: 0=%d, 1=%d, 2=%d, 3=%d, body=%d, outline=%d", + subi[0], subi[1], subi[2], subi[3], + ib+1, io+1 ); +} + + +/* + * get_sub_index: + * Return, via function value, the subtitle index value at element i. + * Check for a remapped index, and clamp the result to [0,3]. + */ + +static int +get_sub_index( int i ) +{ + int c; + + c = mfd->sub[i]; + if ( c == mfd->remap[0] ) + c = mfd->remap[1]; + + if ( c < 0 ) + c = 0; + if ( c > 3 ) + c = 3; + + return( c ); +} + + +/* + * make_subtitle: + * We are given a single NxM plane of indices whose values can be 0 to 3. + * A value of 0 is always considered background and transparent. The values 1, 2, + * and 3 correspond to the text body, inner outline, and outer outline, (non resp). + * First, the indices are replaced by their corresponding opacity values, with each + * index written to a separate plane. These planes are then anti-aliased, and + * zoomed, Then, a minimum bounding box is computed over all planes (UxV), and the + * subtitle frame is replaced with the UxV frame (three planes). We are now capable + * of rendering the subtitle using different opacity values and rendering colors for + * each index. Since the frame has been cropped, we also control where the subtitle + * is located, regardless of where the original bitmap had placed it. + */ + +static void +make_subtitle( int w, int h ) +{ + uint8_t **sp, **dp; + Dim sz, p; + int n, c, i; + + sp = mfd->sp; + dp = mfd->dp; + + /* plane 0 is current subtitle */ + /* zero planes 1 and 2 */ + n = mfd->sz.x * mfd->sz.y; + memset( sp[1], 0, n ); + memset( sp[2], 0, n ); + + /* replace index values with opacity values */ + /* store opacities for each index in separate planes */ + for ( i=0; i<n; i++ ) { + c = get_sub_index( i ); + if ( c ) { + --c; + sp[c][i] = mfd->opc[c]; + } + } + + /* if anti-aliasing or zooming, crop to reduce work */ + if ( mfd->antialias || mfd->do_zoom ) { + /* crop, with margin 6 */ + crop_subtitle( sp, dp, mfd->sz, &sz, 6 ); + cpy3( sp, dp, sz ); + mfd->sz = sz; + } + + if ( mfd->antialias ) { + /* anti-alias each plane (individually) */ + for ( i=0; i<3; i++ ) + tcv_antialias( mfd->tcvhandle, sp[i], dp[i], mfd->sz.x, mfd->sz.y, 1, + mfd->vob->aa_weight, mfd->vob->aa_bias ); + cpy3( sp, dp, sz ); + } + + if ( mfd->do_zoom ) { + /* compute zoomed dimensions */ + sz.x = mfd->sz.x * mfd->z_x + 0.5; + sz.y = mfd->sz.y * mfd->z_y + 0.5; + + /* odd dimensions is probably not a good thing */ + if ( sz.x % 2 ) + sz.x++; + if ( sz.y % 2 ) + sz.y++; + + /* do not allow the zoom to exceed the dimensions of the video frame */ + if ( sz.x > w ) + sz.x = w; + if ( sz.y > h ) + sz.y = h; + + /* zoom each plane (individually) */ + for ( i=0; i<3; i++ ) + tcv_zoom( mfd->tcvhandle, sp[i], dp[i], mfd->sz.x, mfd->sz.y, + 1, sz.x, sz.y, mfd->vob->zoom_filter ); + + cpy3( sp, dp, sz ); + mfd->sz = sz; + } + + /* crop, with margin 0 */ + crop_subtitle( sp, dp, mfd->sz, &sz, 0 ); + cpy3( sp, dp, sz ); + mfd->sz = sz; + + /* find position of overlay region within frame */ + /* - zero corresponds to the bottom line of the frame */ + /* - if topalign is true, position the top line at pos */ + /* (pos should be at least the height of the subtitle) */ + /* - if topalign is false, position the bottom line at pos */ + p.x = (w - mfd->sz.x) / 2; + p.y = h - mfd->pos; + if ( !mfd->topalign ) + p.y -= mfd->sz.y; + + /* make the region even aligned */ + p.x -= p.x % 2; + p.y -= p.y % 2; + + /* top clip */ + mfd->ys = 0; + mfd->yn = mfd->sz.y; + if ( p.y < 0 ) { + mfd->ys = -p.y; + mfd->yn += p.y; + p.y = 0; + } + + /* bottom clip */ + n = p.y + mfd->sz.y - h; + if ( n > 0 ) + mfd->yn -= n; + + if ( mfd->yn <= 0 ) + tc_log_warn( MOD_NAME, "Subtitle is complety clipped. Check the 'pos' option." ); + + mfd->orig.x = p.x; + mfd->orig.y = p.y; +} + + +/* + * crop_subtitle: + * Find the minimum bounding box over the three opacity planes + * and copy that region into the dst buffer, (effectively cropping + * the image). Force both width and height to be a even dimension. + * Return the dimensions of the cropped image via parameter. + */ + +static void +crop_subtitle( uint8_t **sp, uint8_t **dp, Dim sd, Dim *dd, int margin ) +{ + Dim ll, ur; + int x, y, n, m; + int flg, v, i; + + ll.x = 0; + ll.y = 0; + ur.x = 0; + ur.y = 0; + + flg = TC_FALSE; + n = 0; + for ( y=0; y<sd.y; y++ ) + for ( x=0; x<sd.x; x++ ) { + v = sum3( sp, n ); + n++; + + if ( !v ) + continue; + + if ( !flg ) { + ll.x = x; + ur.x = x; + ll.y = y; + ur.y = y; + + flg = TC_TRUE; + continue; + } + + if ( x < ll.x ) + ll.x = x; + if ( x > ur.x ) + ur.x = x; + if ( y < ll.y ) + ll.y = y; + if ( y > ur.y ) + ur.y = y; + } + + /* add margin */ + ll.x -= margin; + ll.y -= margin; + ur.x += margin; + ur.y += margin; + + /* force even dimensions */ + dd->x = ur.x - ll.x + 1; + dd->y = ur.y - ll.y + 1; + if ( dd->x % 2 ) { + ur.x++; + dd->x++; + } + + if ( dd->y % 2 ) { + ur.y++; + dd->y++; + } + + /* extract bounded region to tmp, cropping, possibly padding */ + m = 0; + for ( y=ll.y; y<=ur.y; y++ ) + for ( x=ll.x; x<=ur.x; x++, m++ ) { + if ( x<0 || x>=sd.x || y<0 || y>=sd.y ) + n = -1; + else + n = y*sd.x + x; + + for ( i=0; i<3; i++ ) + dp[i][m] = (n<0) ? 0 : sp[i][n]; + } +} + + +/* + * sum3: + * Return the sum of the three opacity planes for element n. + */ + +static int +sum3( uint8_t **sp, int n ) +{ + int s, i; + + s = 0; + for ( i=0; i<3; i++ ) + s += sp[i][n]; + + return( s ); +} + + +/* + * max3: + * Return the maximum opacity value for element n, over the three + * opacity planes. + */ + +static int +max3( uint8_t **sp, int n ) +{ + int v, i; + + v = 0; + for ( i=0; i<3; i++ ) + if ( sp[i][n] > v ) + v = sp[i][n]; + + return( v ); +} + + +/* + * avg3: + * Return the weighted average for the RGB rendering colors at element n, + * as determined by the contributing opacity values on each plane. + */ + +static Rgb +avg3( uint8_t **sp, int n ) +{ + Rgb clr; + double s[3]; + int w, i; + + w = sum3( sp, n ); + for ( i=0; i<3; i++ ) + s[i] = sp[i][n] / (double) w; + + /* I'm thinking Rgb would have been better as an array */ + clr.r = 0; + clr.g = 0; + clr.b = 0; + for ( i=0; i<3; i++ ) { + clr.r += mfd->clr[i].r * s[i] + 0.5; + clr.g += mfd->clr[i].g * s[i] + 0.5; + clr.b += mfd->clr[i].b * s[i] + 0.5; + } + + if ( clr.r > 255 ) + clr.r = 255; + if ( clr.g > 255 ) + clr.g = 255; + if ( clr.b > 255 ) + clr.b = 255; + + return( clr ); +} + + +/* + * cpy3: + * Copy src to dst, for each of the three planes. + */ + +static void +cpy3( uint8_t **sp, uint8_t **dp, Dim sz ) +{ + int n, i; + + n = sz.x * sz.y; + for ( i=0; i<3; i++ ) + ac_memcpy( sp[i], dp[i], n ); +} + + +/* + * subtitle_overlay_rgb: + * Overlay the subtitle bitmap into the given video frame. For each pixel, + * the opacity used is the maximum value over all index planes. The color used is + * a weighted average as determined by the contributing opacity values on each + * plane. The resulting color is then blended with the pixel, ie: + * + * (1-opacity) * pixel_color + opacity * subtitle_color + */ + +static void +subtitle_overlay_rgb( uint8_t *vid, int w, int h ) +{ + uint8_t *v; + Rgb clr; + double s; + int x, y, n, m; + int opc, yo; + + for ( y=0; y<mfd->yn; y++ ) { + yo = mfd->orig.y + y; + m = (yo*w + mfd->orig.x) * 3; + + n = (mfd->ys + y) * mfd->sz.x; + for ( x=0; x<mfd->sz.x; x++ ) { + opc = max3( mfd->sp, n ); + if ( opc ) { + clr = avg3( mfd->sp, n ); + s = opc / 255.0; + if ( s > 1.0 ) + s = 1.0; + + v = vid + m; + blend_pixel( v+0, s, clr.r ); + blend_pixel( v+1, s, clr.g ); + blend_pixel( v+2, s, clr.b ); + } + + m += 3; + n++; + } + } +} + + +/* + * subtitle_overlay_yuv: + * Trying to blend in pixels in a non-linear color space just doesn't + * produce aesthetically pleasing results. Many variations were attempted. + * At edges of the text, even a slightly opaque pixel would cause the UV component + * for the entire 4x4 pixel cell to be modified, which generated very visable + * artifacts. Therefore, although quite expensive, the overlay region of the + * video frame is extracted to a temporary buffer, converted to rgb, overlayed + * with the subtitle, converted back to yuv, and re-inserted back into the + * video frame. The results are better, but still not as good as just running + * with -V rgb24. When the sub region is converted back to yuv, you still get + * some color blurring around edges of sharp contrast. You really have to be + * looking for it to notice though. + */ + +static void +subtitle_overlay_yuv( uint8_t *vid, int w, int h ) +{ + uint8_t *sp, *dp, *v; + Rgb clr; + double s; + int sw, sh, sn, sy, su, sv; + int dw, dh, dn, dy, du, dv; + int x, y, r; + int opc, n, m; + + if ( mfd->yn <= 0 ) + /* completely clipped */ + return; + + /* src, video frame */ + sp = vid; + sw = w; + sh = h; + sn = sw * sh; + + sy = mfd->orig.y * sw + mfd->orig.x; + su = sn + mfd->orig.y/2 * sw/2 + mfd->orig.x/2; + sv = su + sn/4; + + /* dst, extracted region */ + dp = mfd->tmp; + dw = mfd->sz.x; + dh = mfd->yn; + dn = dw * dh; + + dy = 0; + du = dn; + dv = du + dn/4; + + /* extract subtitle region */ + for ( y=0; y<dh; y++, sy+=sw, dy+=dw ) { + ac_memcpy( dp+dy, sp+sy, dw ); + + if ( !(y % 2) ) { + ac_memcpy( dp+du, sp+su, dw/2 ); + ac_memcpy( dp+dv, sp+sv, dw/2 ); + + su += sw/2; + sv += sw/2; + + du += dw/2; + dv += dw/2; + } + } + + /* convert to rgb */ + r = tcv_convert( mfd->tcvhandle, dp, dp, dw, dh, + IMG_YUV_DEFAULT, IMG_RGB_DEFAULT ); + if ( !r ) { + tc_log_error( MOD_NAME, "overlay, yuv to rgb conversion failed" ); + return; + } + + /* overlay subtitle */ + m = 0; + n = mfd->ys * dw; + for ( y=0; y<dh; y++ ) + for ( x=0; x<dw; x++, m+=3, n++ ) { + opc = max3( mfd->sp, n ); + if ( opc ) { + clr = avg3( mfd->sp, n ); + s = opc / 255.0; + if ( s > 1.0 ) + s = 1.0; + + v = dp + m; + blend_pixel( v+0, s, clr.r ); + blend_pixel( v+1, s, clr.g ); + blend_pixel( v+2, s, clr.b ); + } + } + + /* convert back to yuv */ + r = tcv_convert( mfd->tcvhandle, dp, dp, dw, dh, + IMG_RGB_DEFAULT, IMG_YUV_DEFAULT ); + if ( !r ) { + tc_log_error( MOD_NAME, "overlay, rgb to yuv conversion failed" ); + return; + } + + /* re-insert extracted region */ + sy -= dh * sw; + su -= dh * sw / 4; + sv -= dh * sw / 4; + + dy -= dh * dw; + du -= dh * dw / 4; + dv -= dh * dw / 4; + + for ( y=0; y<dh; y++, sy+=sw, dy+=dw ) { + ac_memcpy( sp+sy, dp+dy, dw ); + + if ( !(y % 2) ) { + ac_memcpy( sp+su, dp+du, dw/2 ); + ac_memcpy( sp+sv, dp+dv, dw/2 ); + + su += sw/2; + sv += sw/2; + + du += dw/2; + dv += dw/2; + } + } +} + + +/* + * blend_pixel: + * Blend in a subtitle color component against the current pixel color + * component, according to the subtitle opacity value. + */ + +static void +blend_pixel( uint8_t *v, double s, int color ) +{ + int t; + + t = (1-s)*(*v) + s*color + 0.5; + if ( t > 255 ) + t = 255; + + *v = t; +} diff -ruN old/filter/extsub/nextsub.h new/filter/extsub/nextsub.h --- old/filter/extsub/nextsub.h 1969-12-31 19:00:00.000000000 -0500 +++ new/filter/extsub/nextsub.h 2007-03-11 22:24:26.000000000 -0400 @@ -0,0 +1,105 @@ +#include "transcode.h" +#include "encoder.h" +#include "filter.h" +#include "libtc/optstr.h" +#include "libtcvideo/tcvideo.h" + +#include <stdint.h> +#include <ctype.h> + +#include "dl_loader.h" +#include "import/magic.h" + +#include "subtitle_buffer.h" +#include "subproc.h" + +#define BUFFER_SIZE SIZE_RGB_FRAME +#define SUBTITLE_BUFFER 100 + + +typedef struct rgb_t Rgb; +struct rgb_t + { + int r; + int g; + int b; + }; + +typedef struct dim_t Dim; +struct dim_t + { + int x; + int y; + }; + +typedef struct MyFilterData_t MyFilterData; +struct MyFilterData_t + { + /* filter options */ + int forced; /* render only forced subtitles */ + int pos; /* position, vertical offset */ + int topalign; /* align via top (not bottom) row of the subtitle */ + int tshift; /* time shift */ + int prefilter; /* run as a pre-filter */ + int antialias; /* anti-alias */ + int do_zoom; /* zoom the subtitle */ + double z_x, z_y; /* zoom factors */ + double fopc[3]; /* index opacities [0-1] */ + Rgb clr[3]; /* index rendering colors */ + Rgb clr_def; /* default rendering color */ + int remap[2]; /* remap an index value to a different index */ + + /* private data */ + transfer_t import; + pthread_t thread; + double f_time; + double pts1; + double pts2; + + uint8_t *sub; /* subtitle frame */ + uint8_t *tmp; /* temp buffer */ + + uint8_t *sp[3]; /* sub, decomposed to three planes */ + uint8_t *dp[3]; /* tmp, decomposed to three planes */ + + Dim orig; /* origin of clipped overlay region in video */ + Dim sz; /* dimension of subtitle */ + int ys; /* starting on-screen row of subtitle */ + int yn; /* number of on-screen subtitle rows */ + + int id; /* frame number */ + int fflg; /* subtitle has forced flag set */ + int opc[3]; /* opacities [0-255] */ + + int have_indices; /* indices to render are known */ + int made_subtitle; /* subtitle has been prepared (zoomed, etc) */ + + vob_t *vob; + TCVHandle tcvhandle; + }; + + +/* + * prototypes: + */ + +static int get_zoom( char*, double*, double* ); +static Rgb get_rgb( char* ); +static int htoi( char* ); +static int htob( char ); +static char *pr_zoom( void ); +static char *pr_clr( Rgb ); +static int subtitle_retrieve( void ); +static void subtitle_overlay( uint8_t*, int, int ); +static void guess_indices( void ); +static int get_sub_index( int ); +static void make_subtitle( int, int ); +static void crop_subtitle( uint8_t**, uint8_t**, Dim, Dim*, int ); +static int sum3( uint8_t**, int ); +static int max3( uint8_t**, int ); +static Rgb avg3( uint8_t**, int ); +static void cpy3( uint8_t**, uint8_t**, Dim ); +static void subtitle_overlay_rgb( uint8_t*, int, int ); +static void subtitle_overlay_yuv( uint8_t*, int, int ); +static void blend_pixel( uint8_t*, double, int ); + diff -ruN old/filter/extsub/subproc.c new/filter/extsub/subproc.c --- old/filter/extsub/subproc.c 2007-03-11 22:25:51.000000000 -0400 +++ new/filter/extsub/subproc.c 2007-03-11 22:24:26.000000000 -0400 @@ -178,7 +178,7 @@ //char filename[16]; //buffer for plugin - unsigned char *picture; + uint8_t *picture; picture = config.sub.frame; diff -ruN old/filter/extsub/subproc.h new/filter/extsub/subproc.h --- old/filter/extsub/subproc.h 2007-03-11 22:25:51.000000000 -0400 +++ new/filter/extsub/subproc.h 2007-03-11 22:24:26.000000000 -0400 @@ -39,7 +39,7 @@ int x, y; int w, h; - char *frame; + uint8_t *frame; int colour[4]; int alpha[4];