We use Gearman as we planning to distribute the encode to our server farm encoders in the future. Right now, we use Gearman to queue the submitted jobs by our user.
Previously, we just call our FFMPEG from PHP shell_exec() and...boom. This happened when our users uploaded 10 videos at once . Learned from our mistakes, we search the internet Online Video encoding best practises. We stumble upon BeanstalkD and Gearman and also Parallel,PPSS for crunching the server resources.
We deployed Gearman on monthly server patches and now working perfectly. To add some visibility to our users what's going on after they upload the video, we put live indicator on the VMS and they can see the encoding done in real time
|Web based indicator.|
FFMPEG already have the options to use all available CPU cores. So we use -threads 16 ( we have 8 cores ) and queue the job to run one by be using Gearman. So it is using FIFO concept. Fast In Fast Out. Peeking to our CPU usage after the changes, we're quite pleased withe results.
|SIngle video encoding using 700% CPU|
Once the encoding done, the videos will be sent to our storage server. Our Wowza VOD origin will fetch the video and serving them to Amazon Cloudfront CDN.
|The resulted video encoded by FFMPEG|
|MyKLIK Video streaming in MBR for Desktop and Mobile|
Softwares mentioned in this blog :
Gearman : http://gearman.org
PPSS : https://code.google.com/p/ppss
BeanstalkD : http://kr.github.io/beanstalkd
SupervisorD : http://supervisord.org
FFMPEG : https://www.ffmpeg.org
Wowza Streaming Engine : http://www.wowza.com
Amazon Cloudfront : http://aws.amazon.com/cloudfront