Hi Michael,
I have attached a patch with the mail.
With `avcodec_get_frame_defaults (&frame)` in the function readAudio
program was working fine with old version in my system. But this function
is not in the latest git repo so I used `av_frame_unref(&frame)` instead
and the program segfaults. I am not sure why or how to fix this.
Also, I haven't properly understood the input and output flow of
libavfilters (how to write input and output links? and overall flow). Could
you explain it to me?
On Thu, Oct 27, 2016 at 3:24 AM, Michael Niedermayer <mich...@niedermayer.cc
> wrote:
> On Thu, Oct 27, 2016 at 12:40:53AM +0530, Pallavi Kumari wrote:
> > I mean deciding a timeline for the opw project. Which is to be mentioned
> in
> > application
>
> its your application, you can choose whatever timeline you feel makes
> sense.
> As far as iam concered whats important is to finish the qualification
> task well before we have to choose the applicant(s) we accept this
> year.
>
> The timeline for the main project in dec-mar can be segmented as you
> like but leave plenty of time toward the end, problems always occur
> and things get delayed.
> Also both for qualification and the main project submit code early
> if you have questions ask early,
> if you dont get satisfactory reply from me on something ask again and
> louder/clearer ... i do sometimes also forget to reply until i then
> remember again days later ...
>
> also i think the architecture wasnt discussed much yet
> do you have some plans about it ?
>
> I mean for example there could be a avfilter that passes audio through
> and adds fingerint(s) as metadata into thf AVFrame
> (as with avpriv_frame_get_metadatap in vf_idet.c) and also print the
> fingerprints via av_log()
>
> a 2nd filter could then look these fingerprints from the metadata
> up in some file/database
>
> a 3rd one could add the fingerprints to the file/database
>
> or these also could all be in one filter of course
>
> this can be extended in the future to for example have a filter
> delay the audio until lookup succeeds and then once identified
> lookup lyrics online and add them or such (this of course would be a
> future thing after outreachy/opw)
>
> but maybe you have a different plan on how the components would
> interact ?
>
> thx
>
> >
> > On Wed, Oct 26, 2016 at 6:01 PM, Michael Niedermayer
> <mich...@niedermayer.cc
> > > wrote:
> >
> > > Hi
> > >
> > > On Mon, Oct 17, 2016 at 02:06:26PM +0530, Pallavi Kumari wrote:
> > > > Hi Michael,
> > > >
> > > > I figured out the use of fft. Help me with the time line setting.
> Thanks
> > >
> > > I dont understand the question,
> > > if its about AVFILTER_FLAG_SUPPORT_TIMELINE*, please ignore this for
> > > now, its not needed
> > >
> > > [...]
> > > --
> > > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC7
> 87040B0FAB
> > >
> > > Rewriting code that is poorly written but fully understood is good.
> > > Rewriting code that one doesnt understand is a sign that one is less
> smart
> > > then the original author, trying to rewrite it will not make it better.
> > >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> He who knows, does not speak. He who speaks, does not know. -- Lao Tsu
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>
From 60429fb7d83d11d6dd733c3477950a940bd0d84a Mon Sep 17 00:00:00 2001
From: Atana <at...@openmailbox.org>
Date: Mon, 24 Oct 2016 17:16:09 +0530
Subject: [PATCH] avfilter: add peakpoints filter
---
libavfilter/Makefile | 1 +
libavfilter/af_peakpoints.c | 263 ++++++++++++++++++++++++++++++++++++++++++++
libavfilter/allfilters.c | 1 +
libavfilter/version.h | 2 +-
4 files changed, 266 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/af_peakpoints.c
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 7ed4696..1a18902 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -96,6 +96,7 @@ OBJS-$(CONFIG_LADSPA_FILTER) += af_ladspa.o
OBJS-$(CONFIG_LOUDNORM_FILTER) += af_loudnorm.o
OBJS-$(CONFIG_LOWPASS_FILTER) += af_biquads.o
OBJS-$(CONFIG_PAN_FILTER) += af_pan.o
+OBJS-$(CONFIG_PEAKPOINTS_FILTER) += af_peakpoints.o
OBJS-$(CONFIG_REPLAYGAIN_FILTER) += af_replaygain.o
OBJS-$(CONFIG_RESAMPLE_FILTER) += af_resample.o
OBJS-$(CONFIG_RUBBERBAND_FILTER) += af_rubberband.o
diff --git a/libavfilter/af_peakpoints.c b/libavfilter/af_peakpoints.c
new file mode 100644
index 0000000..84b51da
--- /dev/null
+++ b/libavfilter/af_peakpoints.c
@@ -0,0 +1,263 @@
+/*
+ * Copyright (c) 2016 Pallavi Kumari
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavcodec/avcodec.h"
+#include "libavcodec/avfft.h"
+#include "libavformat/avformat.h"
+#include "libswscale/swscale.h"
+#include "avfilter.h"
+#include "libavutil/opt.h"
+
+
+/* Structure to contain peak points context */
+typedef struct {
+ const AVClass *class;
+ double *data;
+ int nsamples;
+ double *peaks;
+ int size; // number of peaks
+ int windowSize;
+ char *inputFile;
+} PeakPointsContext;
+
+/* returns maximum value from an array conditioned on start and end index */
+static double getMax(double *res_arr, int startIndex, int endIndex) {
+ int i;
+ double max = res_arr[startIndex];
+ for (i = startIndex; i <= endIndex; i++) {
+ if (res_arr[i] > max) {
+ max = res_arr[i];
+ }
+ }
+ return max;
+}
+
+/* Stores peak frequency for each window(of chunkSize) in peaks array */
+static void getPeakPointInChunk(int chunkSize, double *res_arr, int size, double *peaks) {
+ int endIndex, i = 0, peakIndex = 0;
+ int startIndex = 0;
+ double max;
+ // get a chunk and find max value in it
+ while (i < size) {
+ if (i % chunkSize-1 == 0) {
+ max = getMax(res_arr, startIndex, i);
+ peaks[peakIndex++] = max;
+ startIndex = startIndex + chunkSize;
+ }
+ i += 1;
+ }
+}
+
+/* read audio stream from input file */
+static void readAudio (int *sampleRate, PeakPointsContext *ppc) {
+ // Initialize FFmpeg
+ av_register_all ();
+
+ AVFormatContext *formatContext = NULL;
+ AVCodecContext *codecContext = NULL;
+ AVCodec *codec = NULL;
+ AVPacket packet;
+ AVFrame frame;
+
+ ssize_t nSamples = 0;
+
+ // open the file and autodetect the format
+ if (avformat_open_input (&formatContext, ppc->inputFile, NULL, NULL) < 0)
+ printf("Can not open input file %s", ppc->inputFile);
+
+ if (avformat_find_stream_info (formatContext, NULL) < 0)
+ printf("Can not find stream information");
+
+ // find audio stream (between video/audio/subtitles/.. streams)
+ int audioStreamId = av_find_best_stream (formatContext, AVMEDIA_TYPE_AUDIO,
+ -1, -1, &codec, 0);
+ if (audioStreamId < 0)
+ printf("Can not find audio stream in the input file");
+
+ codecContext = formatContext->streams[audioStreamId]->codec;
+
+ // init the audio decoder
+ if (avcodec_open2 (codecContext, codec, NULL) < 0)
+ printf("Can not open audio decoder");
+
+ // read all packets
+ int i = 0;
+ while (av_read_frame (formatContext, &packet) == 0) {
+ if (packet.stream_index == audioStreamId) {
+ //avcodec_get_frame_defaults (&frame);
+ av_frame_unref (&frame);
+ int gotFrame = 0;
+ if (avcodec_decode_audio4 (codecContext,&frame,&gotFrame,&packet) < 0){
+ printf("Error decoding audio");
+ }
+ if (gotFrame) {
+ // audio frame has been decoded
+ int size = av_samples_get_buffer_size (NULL,
+ codecContext->channels,
+ frame.nb_samples,
+ codecContext->sample_fmt,
+ 1);
+ if (size < 0) {
+ printf("av_samples_get_buffer_size invalid value");
+ }
+
+ // printf("%d\n", *frame.data[0]);
+ ppc->data[i] = (double)*frame.data[0];
+ i += 1;
+ nSamples += frame.nb_samples;
+ }
+ }
+ av_free_packet (&packet);
+ }
+ ppc->nsamples = i;
+
+ if (nSamples < 1)
+ printf("Decoded audio data is empty");
+
+ int sampleSize = av_get_bytes_per_sample (codecContext->sample_fmt);
+ if (sampleSize < 1)
+ printf("Invalid sample format");
+
+ // optional, return value with sample rate
+ if (sampleRate) *sampleRate = codecContext->sample_rate;
+
+ if (codecContext) avcodec_close (codecContext);
+
+ avformat_close_input (&formatContext);
+}
+
+/* Get peaks points from windowed frequency domain data*/
+static void getPeakPoints(PeakPointsContext *ppc) {
+ int i, k, size, chunkSize, chunkSampleSize, resSize, sample_rate;
+ double *fft_res;
+ int m;
+
+ ppc->data = malloc(sizeof(double)*10000);
+ readAudio (&sample_rate, ppc);
+
+ size = ppc->nsamples;
+ m = log2(ppc->windowSize);
+ chunkSize = ppc->windowSize;
+ chunkSampleSize = size/chunkSize;
+ resSize = chunkSize * chunkSampleSize;
+ fft_res = malloc(sizeof(double) * resSize);
+
+ RDFTContext *rdftC = av_rdft_init(m, DFT_R2C);
+ FFTSample *data;
+ data = malloc(sizeof(FFTSample)*chunkSize);
+ // FFT transform for windowed time domain data
+ // window is of size chunkSize
+ k = 0;
+ while (k < resSize) {
+ //copy data
+ for (i = 0; i < chunkSize; i++) {
+ data[i] = ppc->data[i+k];
+ }
+ //calculate FFT
+ av_rdft_calc(rdftC, data);
+ for (i = 0; i < chunkSize; i++) {
+ fft_res[i+k] = data[i];
+ }
+ k = k + chunkSize;
+ }
+
+ av_rdft_end(rdftC);
+
+ int pSize = resSize/chunkSize;
+ ppc->size = pSize;
+ ppc->peaks = malloc(sizeof(double)*pSize);
+ getPeakPointInChunk(chunkSize, fft_res, resSize, ppc->peaks);
+}
+
+
+#define OFFSET(x) offsetof(PeakPointsContext, x)
+
+static const AVOption peakpoints_options[] = {
+ { "input", "set input audio file", OFFSET(inputFile), AV_OPT_TYPE_STRING, {.str="sample.mp3"}, CHAR_MIN, CHAR_MAX },
+ { "wsize", "set window size", OFFSET(windowSize), AV_OPT_TYPE_INT, {.i64=16}, 0, INT_MAX},
+ { NULL },
+};
+
+static const char *peakpoints_get_name(void *ctx) {
+ return "peakpoints";
+}
+
+static const AVClass peakpoints_class = {
+ .class_name = "PeakPointsContext",
+ .item_name = peakpoints_get_name,
+ .option = peakpoints_options,
+};
+
+
+AVFILTER_DEFINE_CLASS(peakpoints);
+
+static av_cold int init(AVFilterContext *ctx)
+{
+ PeakPointsContext *p = ctx->priv;
+
+ if (!(p->inputFile)) {
+ av_log(ctx, AV_LOG_ERROR, "Input audio file must be passed\n");
+ return AVERROR(EINVAL);
+ }
+
+ if (p->windowSize < 16) {
+ av_log(ctx, AV_LOG_ERROR, "window size must be greater than or equal to 16");
+ return AVERROR(EINVAL);
+ }
+ //printf("Getting peak points..");
+ // get peaks
+ getPeakPoints(p);
+ //printf("Peak points collected.");
+ // print peaks
+ int i;
+ for (i = 0; i < p->size; i++) {
+ av_log(ctx, AV_LOG_DEBUG, "%f", p->peaks[i]);
+ }
+ return 0;
+}
+
+static av_cold void uninit(AVFilterContext *ctx)
+{
+ PeakPointsContext *p = ctx->priv;
+ int i;
+
+ // free memory allocated for audio data
+ for (i = 0; p->nsamples; i++) {
+ free(&p->data[i]);
+ }
+
+ // free memory allocated for peaks
+ for (i = 0; p->size; i++) {
+ free(&p->data[i]);
+ }
+}
+
+
+AVFilter ff_af_peakpoints = {
+ .name = "peakpoints",
+ .description = NULL_IF_CONFIG_SMALL("peak points from frequency domain windowed data."),
+ .init = init,
+ .uninit = uninit,
+ //.query_formats = query_formats,
+ .priv_size = sizeof(PeakPointsContext),
+ //.inputs = inputs,
+ //.outputs = outputs,
+ .priv_class = &peakpoints_class,
+};
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 82a65ee..759e63a 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -114,6 +114,7 @@ void avfilter_register_all(void)
REGISTER_FILTER(LOUDNORM, loudnorm, af);
REGISTER_FILTER(LOWPASS, lowpass, af);
REGISTER_FILTER(PAN, pan, af);
+ REGISTER_FILTER(PEAKPOINTS, peakpoints, af);
REGISTER_FILTER(REPLAYGAIN, replaygain, af);
REGISTER_FILTER(RESAMPLE, resample, af);
REGISTER_FILTER(RUBBERBAND, rubberband, af);
diff --git a/libavfilter/version.h b/libavfilter/version.h
index 8c6cf06..adf8c71 100644
--- a/libavfilter/version.h
+++ b/libavfilter/version.h
@@ -30,7 +30,7 @@
#include "libavutil/version.h"
#define LIBAVFILTER_VERSION_MAJOR 6
-#define LIBAVFILTER_VERSION_MINOR 64
+#define LIBAVFILTER_VERSION_MINOR 65
#define LIBAVFILTER_VERSION_MICRO 100
#define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \
--
1.9.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
