What’s next for MobileFFmpeg?
The Story
Back in early 2018, my friends and I were trying to build a social news application for mobile devices. We were using react-native. And, at one point we needed a react-native plugin to extract thumbnails from videos. We tried to use existing thumbnail plugins but they didn’t work us. They were neither updated nor supporting both Android and iOS. So, I though why not create an FFmpeg based react-native-thumbnail plugin. I was already familiar with FFmpeg and I knew that I can use it to extract thumbnails.
But to do that, I needed FFmpeg built for those platforms first. Unfortunately, again I wasn’t lucky with existing solutions. FFmpeg ports available for Android and iOS weren’t good enough to create a plugin relying on them. Thus, I decided to create another project to build FFmpeg on these two platforms. My aim was to support mobile platforms only, so I called it MobileFFmpeg.
I worked on the project for a few months, and released the first version of MobileFFmpeg in June 2018. That first release was supporting 10 architectures and 18 external libraries in total. A few months later I released the react-native plugin as well. But I didn’t build it as a thumbnail plugin as I planned. Instead, I developed it as a more general FFmpeg solution for react-native, and called it react-native-ffmpeg.
New Features
At first, I wasn’t too keen to add new features to MobileFFmpeg. The main idea was to run FFmpeg commands on mobile platforms as you run them on desktop. And that part was already working. But then realised that the wrapper library I created wasn’t easy to use. And some features weren’t working that well. I felt the need to make some changes in the implementation. First, I developed the log callbacks to redirect console output of FFmpeg. It took a few iterations to stabilise it but after that it became one of the core features of the library. Then, I spent quite a lot of time on shared libraries for iOS. I literally wasted weeks on them. They were working in Xcode but App Store Connect wasn’t accepting apps built using them. Apple documentation wasn’t helping me much. In the end, I decided to drop them and re-implemented the iOS library with static libraries. Meanwhile, I had already added support for GPL licensed libraries and created prebuilt packages.
After releasing these updates, I kept fixing bugs and continued to develop new features. Changelog of MobileFFmpeg includes all those changes implemented since 2018. The list below also shows some of the major features implemented since then.
The last release of MobileFFmpeg was published at the end of July 2020. Due to the jcenter() issues we faced, the Android library was republished on mavenCentral() in February 2021. It supports 14 architectures for 3 platforms and 34 external libraries. That release, v4.4, will be the final release of MobileFFmpeg. Because I will retire it soon.
Nevertheless, MobileFFmpeg is not going to disappear completely. I will continue to maintain it. But will release the new versions under a different name, FFmpegKit. I can say that FFmpegKit is the continuation of MobileFFmpeg. Its first release is already published 3 weeks ago. And it includes all bug fixes and features implemented on MobileFFmpeg since August 2020. Apple’s new architectures and Alex Cohn’s SAF (Storage Access Framework) implementation are also included in that release.
So, why am I retiring MobileFFmpeg and releasing new versions under FFmpegKit?
Well, there are two main reasons.
First things first, the project repository has become too big. It is not possible to clone the repository using some git clients. Even I, myself cannot do it. And there is no easy fix for it. So, creating a new project, starting with an empty git history is my solution to this problem.
The second reason is the project scope. Recently, I received a lot of requests about supporting non-mobile platforms like macOS, Linux, etc. Some of those requests came in flutter_ffmpeg, my Flutter port of MobileFFmpeg. Initially, I rejected them because they were not in the scope of the project. Also, I didn’t have native implementations for them. flutter_ffmpeg was backed by MobileFFmpeg and MobileFFmpeg was only Android and iOS (and tvOS). So, adding support for those platforms wasn’t that easy.
Moreover, I created MobileFFmpeg to fill a gap. Something that wasn’t available or good enough for mobile platforms. For desktop platforms ffmpeg.org was already releasing binaries. I wasn’t convinced that supporting these desktop platforms would make sense. Well, I’m still not convinced. But I must admit that Flutter makes it easy to develop cross platform applications. And adding the support for those platforms will help a lot of Flutter developers. So, I decided to give it a go and implement some of these non-mobile platforms. Since those platforms are not mobile, I wanted to release them under a different name. That’s why I’m retiring MobileFFmpeg and releasing new versions under FFmpegKit.
MobileFFmpeg was aiming to provide FFmpeg on mobile platforms. FFmpeg Kit aims to provide FFmpeg for applications.
What’s New
The first release of FFmpegKit already includes one new platform, macOS. It is one of the 4 platforms supported by FFmpegKit. Build scripts can build both x86–64 and the new arm64 architecture for macOS. There are new additions to the iOS and tvOS architectures too. Scripts are updated to support arm64-simulator and arm64-mac-catalyst on iOS, and arm64-simulator on tvOS.
Supported architectures are not the only thing that changed in the build scripts. FFmpegKit didn’t simply copy them from MobileFFmpeg. All scripts are optimised and re-organised under the scripts folder. Some of the existing ones are completely re-written to share more code between different platforms and allow more control during the build process. Additionally, some new scripts can be customised. source.sh file under the scripts folder is the script that stores the external library URLs and versions. You can edit that file and change the external library version that will be built by the scripts.
This is possible because, contrary to MobileFFmpeg, FFmpegKit repository does not include the source code of external libraries. The source code of an external library is downloaded only when it is enabled by “--enable-<library-name>” or “ --full” flag. This was a build model tested for a few libraries in the latest version of MobileFFmpeg. It worked well there so it was applied to all external libraries here. Using this approach, we can guarantee that the size of FFmpegKit repository will not grow too much as it happened in MobileFFmpeg.
This also means that you need to have an active internet connection and access to github.com to build FFmpegKit. But I believe this is a compromise we can all live with.
FFmpegKit API is mostly the same as the MobileFFmpeg API. It is again written in Java for Android and in Objective-C for Apple platforms. There are a lot of enhancement under the hood, also some new features that must be explained in detail. But, with regard to backward compatibility there are two breaking changes in the new API.
The first breaking change is the new Session concept introduced in FFmpegKit. It was developed to access execution information more easily. In MobileFFmpeg, synchronous execute methods return the return code and asynchronous ones return the execution id. In FFmpegKit all execute() methods return a Session. Every time an execute() method is called a new Session is created. And the Session object created is returned back to the caller. Caller can use it to get the session id, the return code, logs, statistics, etc. everything associated with that execution.
The second breaking change is made in the Objective-C API. In MobileFFmpeg, Objective-C callbacks were defined as Delegates, whereas in FFmpegKit they are implemented as blocks. This was a necessary change to allow define inline callbacks. Using Delegates defining inline callbacks wasn’t possible. So, In FFmpegKit Objective-C callbacks are re-implemented as blocks.
Conclusion
This is a wrap up of what’s happening around MobileFFmpeg. The next step is to release react-native and flutter plugins under FFmpegKit. It may not seem like a necessary change. But it will help me to explain the relationship between these libraries more easily. It will save us time too.
If you’re using MobileFFmpeg give FFmpegKit a try and let me know if you encounter any issues! FFmpegKit is here.
Update: As of September 14, 2022, I moved both FFmpegKit and FFmpegKitTest repositories under the ARTHENICA organisation on GitHub. It is just a cosmetic change to separate FFmpegKit from my other projects.
