Re: [FFmpeg-devel] [GSoC] [WIP] [RFC] FLIF Encoder & Decoder Project

2020-05-05 Thread Jai Luthra
Hi Kartik,

On Mon, Mar 30, 2020, at 4:50 AM, Kartik K. Khullar wrote:
> This is my WIP patch for GSoC and I worked on transformations involved 
> in encoding/decoding FLIF images. I have created a module under 
> libavcodec and as guided by my mentors I have tried to use pixel data 
> from AVFrame structs.
> Module covers all the chunks of code for YCoCg Transformation that will 
> be used in final encoder/decoder. Necessary structs and methods have 
> been made as suggested by my mentors. The module compiles/builds 
> successfully.
> Also I have attached a small code 'transformtest.c' which I wrote for 
> testing the implementation of pixel transformation.
> The variable AVFrame::data[] is of type uint8_t* and I was initially 
> unaware of this so I stored the YCoCg values in 'data'. So the negative 
> values which were the output of transformation of RGB -> YCoCg were not 
> stored properly and thats where the output is wrong.

I tested your code, and it is good for an initial attempt (ofc the negative 
values are overflowing the uint8_t range, which is wrong). 

Your understanding of the problem is correct, when we transform an RGB value 
that could lie in (0-255, 0-255, 0-255) it can result in a YCoCg value that 
could be anywhere between (0-255, -255-255, -255-255) and thus not fit within 
AVFrame.data which is uint8_t *

> Just wanted to ask, if I should be using some other structs for storing 
> the YCoCg values and not AVFrame, because AVFrame seems to be the 
> standard struct in FFmpeg where the raw media resides.

The YCoCg doesn't need to go in AVFrame as your testcase (RGB->YCoCg) is the 
encoding phase, which reads RGB values from **AVFrame** and ultimately should 
output binary encoded data (after entropy coding) into **AVPacket**. Sorry if 
this was not clear before.

It is OK if you use a bigger buffer with 16-bits per color value for the 
intermediate transform stages. The only invariant is that original frame will 
have 8-bit RGB values, and final encoder output will be binary data. What the 
encoder uses in the interim to represent pixel values doesn't matter to FFmpeg 
api.

But theoretically you will never use all the 16x16x16 bits with YCoCg, as the 
Co range is conditional on Y, and Cg range conditional on Y & Co. It is 
*crucial* for your project that you thoroughly understand the "new ranges" 
section in the spec [1]

Unlike YCbCr (and other common transforms) which goes from 0-255 to 0-255 (or 
even shorter), YCoCg works differently. If we know that Y value is very low or 
very high, it means color components are roughly equal and thus Co and Cg will 
definitely be in a small range. This is what the animation [2] in the spec is 
about. The Y/Luminance varies from 0-255 and the working range of CoCg is shown 
as the size of the square.

I.e. the transform may take a Co value to -255-255 but that will not happen for 
every value of Y. It will only happen when `origmax4-1 <= Y <= 3*origmax4 - 1`. 
Similar rules apply for Cg.

So your next steps should be:
1. Use uint16_t to store interim pixel values for all transformations (doesn't 
need to be part of AVFrame, is internal structure to decoder)
2. Figure out how to implement the crange functions/api as this will be crucial 
for the MANIAC encoder phase (it needs to know the conditional ranges to 
effectively do entropy coding of the pixel values)

> Attaching some testcases of RGB and its corresponding YCoCg values for 
> testing purposes.
> 
> Thanks
> Kartik K. Khullar

Cheers,
Jai

[1]: https://flif.info/spec.html#_new_ranges
[2]: https://www.youtube.com/watch?v=-v-xoKZBnhI
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [GSoC] [WIP] [RFC] FLIF Encoder & Decoder Project

2020-03-29 Thread Kartik K. Khullar
This is my WIP patch for GSoC and I worked on transformations involved in
encoding/decoding FLIF images. I have created a module under libavcodec and
as guided by my mentors I have tried to use pixel data from AVFrame structs.
Module covers all the chunks of code for YCoCg Transformation that will be
used in final encoder/decoder. Necessary structs and methods have been made
as suggested by my mentors. The module compiles/builds successfully.
Also I have attached a small code 'transformtest.c' which I wrote for
testing the implementation of pixel transformation.
The variable AVFrame::data[] is of type uint8_t* and I was initially
unaware of this so I stored the YCoCg values in 'data'. So the negative
values which were the output of transformation of RGB -> YCoCg were not
stored properly and thats where the output is wrong.
Just wanted to ask, if I should be using some other structs for storing the
YCoCg values and not AVFrame, because AVFrame seems to be the standard
struct in FFmpeg where the raw media resides.
Attaching some testcases of RGB and its corresponding YCoCg values for
testing purposes.

Thanks
Kartik K. Khullar
#include "transformflif16.h"

int process(Transform transform, AVFrame *frame, int p){
switch(transform.transform_number){
case 1: //Transform number for YCoCg transformation is 1 officially.
if(!transform.done){
			TransformYCoCg transformYCoCg = initYCoCg(frame, p);
if(!processYCoCg(frame))
	transform.done = 1;
return 0;
}
		break;

/*
Rest of the cases will be implemented here.
*/

default:
break;
}
return 0;
}

TransformYCoCg initYCoCg(AVFrame *frame, int p){
	TransformYCoCg transform;
	transform.num_planes = p;
	transform.ranges = getRanges(transform.num_planes, frame);
transform.origmax4 = max(transform.ranges[0].max, transform.ranges[1].max, transform.ranges[2].max)/4 -1;
for(p=0; pdata[0][r*width + c];
G = frame->data[1][r*width + c];
B = frame->data[2][r*width + c];

Y = (((R + B)>>1) + G)>>1;
Co = R - B;
Cg = G - ((R + B)>>1);

frame->data[0][r*width + c] = Y;
frame->data[1][r*width + c] = Co;
frame->data[2][r*width + c] = Cg;
}
}
	return 0;
}

int invProcessYCoCg(AVFrame *frame){
	int r, c;
ColorVal R,G,B,Y,Co,Cg;

//Assuming all the channels will have same width and height
	int height = frame[0].height;
int width = frame[0].width;

	for (r=0; rdata[0][r*width + c];
Co= frame->data[1][r*width + c];
Cg= frame->data[2][r*width + c];

R = Co + Y + ((1-Cg)>>1) - (Co>>1);
G = Y - ((-Cg)>>1);
B = Y + ((1-Cg)>>1) - (Co>>1);

frame->data[0][r*width + c] = R;
frame->data[1][r*width + c] = G;
frame->data[2][r*width + c] = B;
}
}
	return 0;
}

ColorRanges* getRanges(int p, AVFrame *frame){
ColorRanges ranges[p];
int i, c, r, width, height;
uint8_t min, max;

for(i=0; idata[p][0];
max = frame->data[p][0];
for(r=0; r frame->data[p][r*width + c])
min = frame->data[p][r*width + c];
if(max < frame->data[p][r*width + c])
max = frame->data[p][r*width + c];
}
}
ranges[p].min = min;
ranges[p].max = max;
}
return ranges;
}

int max(int a, int b, int c){
	if(a > b){
		if(a > c)
			return a;
		else
			return c;
	}
	else
		return b;
}

int min(int a, int b){
	if(a < b)
		return a;
	else
		return b;
}

int get_min_y(int origmax4){
	return 0;
}

int get_max_y(int origmax4){
	return 4*origmax4-1;
}

int get_min_co(int origmax4, int yval){
	int newmax = 4*origmax4 - 1;
	if (yval < origmax4-1)
	return -3 - 4*yval; 
	else if (yval >= 3*origmax4)
  	return 4*(yval - newmax);
else
  	return -newmax;
}

int get_max_co(int origmax4, int yval){
	int newmax = 4*origmax4 - 1;
	if (yval < origmax4-1)
	return 3 + 4*yval; 
	else if (yval >= 3*origmax4)
  	return 4*(newmax - yval);
else
  	return newmax;
}

int get_min_cg(int origmax4, int yval, int coval){
	int newmax = 4*origmax4 - 1;
	if (yval < origmax4-1)
	return -2 - 2*yval; 
	else if (yval >= 3*origmax4)
  	return -2*(newmax-yval) + 2*((abs(coval)+1)/2);
else{
  	return min(2*yval + 1, 2*newmax - 2*yval - 2*abs(coval)+1)/2;
	}
}

int get_max_cg(int origmax4, int yval, int coval){
	int newmax = 4*origmax4 - 1;
	if (yval < origmax4-1)
	return 1 + 2*yval - 2*(abs(coval)/2); 
	else if (yval >= 3*origmax4)
  	return 2 * (newmax-yval);
else
  	return min(2*(yval- newmax), -2*yval - 1 + 2*(abs(coval)/2));
}

int min_range_YCoCg(int p, int origmax4){
	switch(p){
		case 0:
			return 0;
case 1:
			return -4*origmax4+1;
case 2:
			return -4*origmax4+1;
		default:
			return 0;
			break;
	}
}

int