[Libav-user] FFMPEG decoding H264 and Multithreading

[Jérôme SALAYET]

Hello, I'm using FFMPEG to decode video streaming from an IP Camera. all seems good but I try to decrease the CPU usage of my application when I decompress several differents streams.
In fact, I have a dual core processor, and if I decode one stream, my CPU usage is very low (so good). If I decompress two streams, it's ok too. But if I had a third decompression, my CPU increase from 2-5% to 25-30%.

So, I try to activate Mutlithreading in FFMPEG (compiling with --enable-w32threads) with Mingw.


But if I initialize my AVCodecContext with thread_count=3 (because I have a dual core processor) and thread_type = FF_THREAD_FRAME|FF_THREAD_SLICE, when I use avcodec_open2, I always got an result error -22: The maximum value for lowres supported by the decoder is 0. I read where this message appear in FFMPEG code source and it's in utils.c

if (avctx->codec->max_lowres < avctx->lowres || avctx->lowres < 0)

But in my code before call avcodec_open2( lpCodecCtx, lpCodec, ...) I have m_lpCodec->max_lowres=0 and m_lpCodecCtx->lowres=0, so I don't understa,nd why I always have this error.



After add some traces in Utils.c, it seems I have avctx->codec->max_lowres=0 and avctx->lowres=3 but I don't understand why...


Perhaps, trying Multithreading isn"t the good solution to decrease the CPU usage, I'm not very sure of this point.


Thanks for your answers,



Jérôme SALAYET
Ingénieur Informatique
Tél : [phone]
Fax : [phone]
Site web : www.hymatom.fr <http://www.hymatom.fr/>
Email : ***@hymatom.fr <mailto:***@hymatom.fr>

[Carl Eugen Hoyos]

Post by Jérôme SALAYET
if I decode one stream, my CPU usage is very low
(so good). If I decompress two streams, it's ok too.
But if I had a third decompression, my CPU increase
from 2-5% to 25-30%.So, I try to activate
Mutlithreading in FFMPEG

You should use multi-threading if you want
to decrease overall decoding time, if you
need minimal cpu usage, force -threads 1
(multithreading always has an overhead).

Post by Jérôme SALAYET
(compiling with --enable-w32threads) with Mingw

I believe this should not be necessary, if it is
there may be a bug (that should be reported).

Carl Eugen

[Camera Man]

Post by Carl Eugen Hoyos
You should use multi-threading if you want
to decrease overall decoding time, if you
need minimal cpu usage, force -threads 1
(multithreading always has an overhead).

Additionally, multithreading introduces a decoding delay of (number of
threads) frames - e.g., if you use 4 threads, the earliest time you'll
get your frame out is when you feed in the 5th. If you're decoding from
a file, that makes no difference - however, if you're showing a live
stream at (say) 15 fps, then a 4 frame delay = ~300ms which is very
noticeable - in this case you would also want to disable threads.

[René J.V. Bertin]

Post by Carl Eugen Hoyos
You should use multi-threading if you want
to decrease overall decoding time, if you
need minimal cpu usage, force -threads 1
(multithreading always has an overhead).

And, if it's still actual, multithreaded decoding only makes sense for H.264 with slices.

R

[Carl Eugen Hoyos]

Post by René J.V. Bertin
multithreaded decoding only makes sense for H.264 with slices.

This is not correct.

Carl Eugen

[Alex Cohn]

Post by Jérôme SALAYET
Hello, I'm using FFMPEG to decode video streaming from an IP Camera. all
seems good but I try to decrease the CPU usage of my application when I
decompress several differents streams.
In fact, I have a dual core processor, and if I decode one stream, my CPU
usage is very low (so good). If I decompress two streams, it's ok too. But
if I had a third decompression, my CPU increase from 2-5% to 25-30%.
So, I try to activate Mutlithreading in FFMPEG (compiling with
--enable-w32threads) with Mingw.****
Perhaps, trying Multithreading isn"t the good solution to decrease the CPU
usage, I'm not very sure of this point.

Other authors gave you important advice, but to address your specific
question: multithreading is definitely nice for multi-stream decoding,
especially when you have multi-core CPU. But this should not be
multithreaded decoding: the best overall performance can be achieved when
separate threads are attached to each input stream, and each thread uses a
"single-threaded" decoder instance.

BR,
Alex

[Jérôme SALAYET]

Hello,

In fact, in can't do test with Multithreading enable because I still have the message « The maximum value for lowres supported by the decoder is 0 », and I don't understand why (even if I set thread_count to 1).



If someone could tell me if there's a mistake in my code, thanks a lot...

Reagrds,



Part of my class code (one class used by decompression):



Define the lockmgr callback function:



static int ff_lockmgr(void **mutex, enum AVLockOp op)

// Callback

{ if (NULL == mutex)

return -1;



CRITICAL_SECTION **cSec = (CRITICAL_SECTION **)mutex;



switch (op)

{ case AV_LOCK_CREATE:

{ *cSec = NULL;

*cSec = new CRITICAL_SECTION();

if (*cSec == NULL) return 1;

InitializeCriticalSection(*cSec);

return 0;

}

case AV_LOCK_OBTAIN:

{ if (*cSec == NULL) return 1;

EnterCriticalSection(*cSec);

return 0;

}

case AV_LOCK_RELEASE:

{ if (*cSec == NULL) return 1;

LeaveCriticalSection(*cSec);

return 0;

}

case AV_LOCK_DESTROY:

{ if (*cSec == NULL) return 1;

DeleteCriticalSection(*cSec);

delete *cSec;

*cSec = NULL;

return 0;

}

}

return 0;

}



At the class constructor :

Just call once using a global check value :



av_register_all();

av_lockmgr_register(ff_lockmgr);



decoding part (call each once per frame):



if (m_lpCodecCtx ==NULL)

{ m_lpCodecCtx = avcodec_alloc_context3(NULL);

avcodec_get_context_defaults(m_lpCodecCtx);

}

// Init CodecContext

if (m_lpCodecCtx->width == 0)

{ m_lpCodecCtx->pix_fmt = AV_PIX_FMT_YUV420P;

m_lpCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;

m_lpCodecCtx->codec_id = CODEC_ID_H264;

m_lpCodecCtx->coded_width = iImageWidth; // values I already got from my camera stream

m_lpCodecCtx->coded_height = iImageHeight; // values I already got from my camera stream

m_lpCodecCtx->width = iImageWidth;

m_lpCodecCtx->height = iImageHeight;



if (m_lpCodecCtx->codec_id == AV_CODEC_ID_H264)

{ m_lpCodecCtx->thread_count = 1;

m_lpCodecCtx->thread_type = FF_THREAD_SLICE|FF_THREAD_FRAME;



}



if (m_lpCodec==NULL)

{ m_lpCodec = avcodec_find_decoder(m_lpCodecCtx->codec_id);



if(m_lpCodec->capabilities&CODEC_CAP_TRUNCATED) m_lpCodecCtx->flags|= CODEC_FLAG_TRUNCATED;



avcodec_open2(m_lpCodecCtx, m_lpCodec, NULL);

m_lpFrame = lpfnavcodec_alloc_frame();



m_lpFrame->width = m_lpCodecCtx->width;

m_lpFrame->height = m_lpCodecCtx->height;

}



// Prepare the packet to decode

av_init_packet(&m_lpPacket);



// Set the INPUT BUFFER PADDING

if (m_lpFFMPEGBuffer == NULL)

{ m_lpFFMPEGBuffer = new BYTE[MAX_MPEG4VIDEO_FRAMESIZE+FF_INPUT_BUFFER_PADDING_SIZE];

}

if (m_lpFFMPEGBuffer !=NULL)

{ ZeroMemory( m_lpFFMPEGBuffer, MAX_MPEG4VIDEO_FRAMESIZE+FF_INPUT_BUFFER_PADDING_SIZE);

CopyMemory( m_lpFFMPEGBuffer, lpImageData, dwImageSize);

// Set the buffer to packet data

m_lpPacket.data = m_lpFFMPEGBuffer;

// Set the size to packet size

m_lpPacket.size = dwImageSize;

}



// Decode the frame


do


{ iRes = avcodec_decode_video2( m_lpCodecCtx, m_lpFrame, &iFrameFinished, &m_lpPacket);

if (iRes<0)

{ break;

}

if (iRes>0)

{ m_lpPacket.data+=iRes;

m_lpPacket.size-=iRes;

}

}

while (m_lpPacket.size>0);



if (iFrameFinished)

...

I display my decoded image using swscale

...

// Free Packet

av_free_packet(&m_lpPacket);



When I finish my decompression work :



// Close the codec context

avcodec_close(m_lpCodecCtx);

av_free(m_lpCodecCtx);



avcodec_free_frame(&m_lpFrame);



if (m_lpFFMPEGBuffer != NULL)

{ delete [] m_lpFFMPEGBuffer;

}

m_lpFrame = NULL;

m_lpCodec = NULL;

m_lpCodecCtx = NULL;

m_lpFFMPEGBuffer = NULL;





Jérôme SALAYET
Ingénieur Informatique
Tél : [phone]
Fax : [phone]
Site web : www.hymatom.fr <http://www.hymatom.fr/>
Email : ***@hymatom.fr <mailto:***@hymatom.fr>