Emerging from the long history of the video broadcast industry, there are legacy standards which specify rectangular pixels instead of square pixels. Yes, really! According to Wikipedia, non-square pixels originate in early digital TV standards:
The term pixel aspect ratio was first coined when ITU-R BT.601 (commonly known as “Rec. 601“) specified that standard-definition television pictures are made of lines of exactly 720 non-square pixels.
Today, we’re announcing that we will no longer support rendering non-square pixels. It will still be possible to use videos with non-square pixels in your projects, though.
GStreamer Editing Services (GES), the library used by Pitivi for video processing, is very flexible and allows using videos of any video format in the same project. However, normally, in a “pro” setup, most video editing applications are very strict about the formats they accept as input, so Pitivi and GES were a bit unconventional with the “anything goes” approach.
This might give a clue as to why it was rather complicated and not very rewarding to “properly” fix the rectangular pixels problems in GES. The following question arose in our IRC channel (#pitivi on freenode):
[2017-10-30 14:37:38] <thiblahute> nekohayo: So my main question would be, do we really care these days to be able to produce non square output?
And we mostly came to the conclusion that only dinosaurs are producing content with rectangular pixels, and that with our limited resources we can’t afford to spend time and effort on this:
[2017-10-30 14:44:06] <nekohayo> so… it seems that non-square only matters for older formats
[2017-10-30 14:44:45] <nekohayo> and since we gave up on handling DV tapes/firewire anyway… we could as well declare PAR dead. Because f*** the 1990-2000’s
[2017-10-30 14:45:29] <nekohayo> Mathieu_Du’s only got love for you if you were born in the 80’s
[2017-10-30 14:46:58] <Mathieu_Du> the what again ?
[2017-10-30 16:37:28] <Mathieu_Du> aha, knew that song from lubosz though 🙂
As a result of this simplification, the aspect ratio controls in the project settings dialog have been removed—one less thing for the user to worry about—and so the project width and height are now the only settings defining the display aspect ratio.
Three years ago we switched our bug tracker from Bugzilla to Freedesktop’s Phabricator instance. As very few projects were using it, the maintenance cost was too high for the gain, so the current plan is to obsolete it. Phabricator worked well for us, but now we say bye.
We’re excited to have three students that will contribute to Pitivi this summer. Congratulations, Fabián Orccón, Suhas Nayak and Ștefan-Adrian Popa!
Fabián already had an internship with us two years ago. He’ll focus on a plugin system with great documentation for video hackers, and a first plugin for inspecting the timeline of the currently opened project in a Python console (T3193).
Suhas will focus on a modern color correction UI for the corresponding GStreamer plugin, and lay the path for similar changes to other effects (T2372).
You can follow the linked blogs or subscribe to the Phabricator tasks to stay up-to-date with the progress, or keep an eye on the Pitivi planet. Feel free to make suggestions about these projects on the corresponding tasks, or chat with us about them in our IRC channel. The official start of the coding period is May 30.
Besides mentoring for GSoC, the Pitivi maintainers are busy ironing out the last stability problems we’re aware of. We made great progress, expect version 0.99 sometime soon, followed by some more testing towards 1.0!
Thanks to Google for GSoC and to the GNOME Foundation for allowing us to have these projects under their umbrella this year!
This is another release focused on fixing bugs and improving stability.
We switched our official build to use GTK 3.22 and the framework did not like how we were using it, spamming us with warnings in the console. We fixed those and improved the timeline in the process and added more unit tests.
It was quite a journey. Initially, the GTK Inspector was useful to figure out which widgets the widget IDs in the warnings identified. Then we had to go back between the GTK documentation and the #gtk+ IRC channel until we figured out we were changing the size of some containers in their do_draw methods, which was not good.
Taking advantage of an opportunity, with the last money from our coffers (the remaining 2014 fundraiser donations and the GSoC mentor stipends) we hired Alexandru Băluț, a long-time Pitivi contributor, so he can focus better on fixing issues blocking the 1.0 release. Alex worked on Pitivi in Oct, Nov, and will allocate more time in Dec 2016. Thanks again to everybody who donated!
Distributing apps as packages (deb, rpm, etc) is problematic. For example, the Pitivi package depends on the GTK package and Pitivi 0.95 broke in the distributions which updated to GTK version 3.20, because of the incorrect way we were using a virtual method. This is not the first time something like this happens. To avoid the slippery dependencies problem, two years ago we started making universal daily builds. They allowed everybody to run the latest Pitivi easily by downloading a large binary containing the app and all the dependencies.
A few problems still remained:
It was complicated to set up the Pitivi development environment. We had a huge script for setting up the development environment and new contributors always had problems. We tried to reuse the daily builds for preparing a development environment but it was hacky and we failed to make it work correctly.
Users were notified about available updates through a custom mechanism and updating was not ideal, requiring to download a huge binary blob.
Maintaining the AppImageKit based bundle required following many dependencies (security) updates and maintaining the build for them (even though we were sharing the burden with the whole GStreamer community, as we were using the Cerbero build system).
Recently Flatpak got our attention as it set out to fix these problems. Flatpak’s sandboxing allowed us to create a nice development environment. Our new dev env script reuses the one which flatpaks Pitivi so we got rid of the old ones. Also, Flatpak allows the user to only download binary diffs for updates. Moreover, the Gnome community already provides builds for their SDK and Runtimes, which contains almost everything we need for Pitivi; now, we only have to build a few libraries, which makes it much simpler for us to maintain the new bundles.
We’re enthusiastic about Flatpak. Jakub Steiner summarized Flatpak’s advantages very nicely:
Flatpak aims to solve the painful problem of the Linux distribution—the fact that the OS is intertwined with the applications. It is a pain to decouple the two to be able to:
Keep a particular version of an app around (and working, our note), regardless of OS updates. Or vice versa, be able to run an uptodate application on an older OS.
Allow application authors distribute binaries they built themselves. Binaries they can support and accept useful bug reports for. Binaries they can keep updated.
For now you need to use the command line to install Pitivi with Flatpak. In the near future, it should be trivial for you to install and manage our Flatpak build of Pitivi in your favorite software manager. GNOME Software is a good example.
Besides the usual cleanup and bug squashing, this release brings a fundamental feature which greatly improves the edit experience. Now you can edit and render a video project with any video files with great accuracy, thanks to proxy files.
Proxy editing: fast and accurate editing with any video format
The harsh reality of this world is that there is only a limited set of formats that really work well for video editing. Without those, rendering a project could have been inaccurate at times*. The user experience degrades when using container formats not designed for fast & accurate seeking. For example, seeking a video format where the keyframes are followed by one or more inter-frames (such as in VP8, H.264) is often slow.
To provide the best experience, we decided to give you the ability to seamlessly work with media formats suitable for video editing. This is accomplished with what we call “proxy” files. Other video editors advertise the use of downscaled proxy files which, besides the improved reliability, also provide better performance when the system is not powerful enough to handle high-quality video files in real-time. We do not cover the the downscaled proxy files case yet—that’s planned for post 1.0. For now, our proxies are high-quality proxies intended to be used instead of the original, when editing and also when rendering the project.
Oh, and since we’re so smart, we now also generate the audio waveforms and filmstrip thumbnails at the same time as we are processing proxies. Super efficient.
So what is the difference between an original and its “proxy”? The proxy is a slightly degraded but easier-to-work-with version of the original:
Okay, okay, we might be stretching the analogy a little bit, but you get the picture.
When are proxies needed for stability?
The whitelist below shows the formats we officially support—which can be used safely without proxying them. For the rest, we recommend using proxy files to guard against bugs in the decoders and demuxers. Depending on how popular other formats get, we will consider officially supporting them. The intention is to keep the list short to avoid spending too much time on decoders and demuxers.
At the moment, the whitelist is comprised of the following:
Containers: QuickTime, Ogg, Matroska, WebM
Video codecs: H.264, MJPEG, VP8, Theora, raw
Audio codecs: Opus, MP3, Vorbis, FLAC, raw
How does it work?
When you import an asset with an unsupported format, it is by default transcoded using a codec suitable for video editing. You can disable or force the proxy creation on a per-asset basis, when you import or later at any time. Finally, when rendering, you can specify whether to use the proxies or the original files. You are always in control.
As seen in the screenshots above, the state of an asset’s proxy is shown by an icon in the bottom-left corner of the asset thumbnail, in the Media Library. The icon can be one of:
none The original is used.
The proxy is being created by transcoding the asset.
The proxy is ready and is used automatically.
The proxy creation failed and the original is used.
The main disadvantage of using the intermediate proxy files for rendering is that some quality is lost when transcoding, but this is not noticeable in most cases. Other than that, transcoding the originals takes a bit of time and they take some disk space (around 10 times more for H.264 content, for example). In practice, we think that using proxy files is the best option. You decide what works for you but take into account what is considered safe and officially supported, and what is not.
It was very important to pick a codec that preserves the visual quality of the originals, so that final renders can be done out of the intermediate proxy files with no perceived quality loss. The other requirement was that seeking should be fast, meaning it had to be an intra-frames-only codec, where a frame is encoded and decoded independently of any other frame. Currently Pitivi uses MJPEG or ProRes for the proxy files, depending on which is available on the system.
For now, the audio is transcoded using Opus, even though it’s a lossy format, because the codec is very good. We’ll most probably switch to FLAC when it will be more supported in GStreamer because it loses no quality and has a pretty good compression ratio.
Scrubbing the timeline easily
Currently the user is able to select, move and edit clips by using the left mouse button. This is exactly how a user expects to interact with a GTK UI and it works very well. In addition to that, when clicking using the left mouse button, a seek is performed and the playhead is moved to that position. Some users were annoyed by this combined behavior so we added an option for disabling seek on left-click, but this was not enough.
Inspired by Blender’s different functionality assignation for left and right mouse buttons, we added the ability to seek by right-clicking. Now you can quickly and safely scrub on the timeline as well on the ruler using the right mouse button. Since Pitivi did not provide a context menu when right-clicking on the timeline, it was easy to make this possible. Try it out and let us know what you think. More details on the reasoning that led to this are visible in T3158.
The transformation box is back, thanks to Collabora and Lubosz Sarnecki! This is the third implementation of the transformation box, first one being hacky and second one being available only for users with GL video cards. Now it works great due to the almighty GtkSink, GStreamer’s latest GTK widget for displaying video.
The new transformation box fits great with Pitivi by making clips selectable from the viewer, so you can manage multiple overlapping clips quite easily. But the best part of this implementation may be its extensibility, an example being the module for the title clips.
Project settings: letting the computer figure things out
At least one of our contributors felt intimidated when he first saw the project settings dialog which used to show up when creating a new project. The project settings define how you preview the project while editing. You might want for example to choose a small video size if your computer is not very powerful, then at render time change that to get the best quality. These settings can be changed safely at any time.
The question was whether showing that dialog is really necessary at that point in time. Well, not really. Now, when you create a new project, some default values are set and then when you import the first video file, its properties are transposed to the project settings.
The logic is thus: either users don’t really care about the project settings and the “Project Settings” dialog showing up when creating a new project overwhelms them, or the user is an experienced videographer, in which case they’ll know when they need to change the project settings and where to find them. A very nice touch is that when the project settings are changed automatically, an infobar appears, so the user has the chance of opening the project settings and correct them if need be. More details in T3161.
New project settings and rendering UI
The widgets in the project settings dialog and the rendering dialog were split into multiple tabs, to fit on 1024×768 screens. Unfortunately this required clicking the tabs to see or edit the settings. Moving the eye is faster than clicking so we looked into this and came up with a solution to get rid of the tabs and keep the dialogs as small as possible.
The preset selection UI is now composed of a ComboBox plus a MenuButton. This way the preset selection UI takes extra vertical space instead of extra horizontal space. We find it works pretty good. More details about how it works and more screenshots in this doc.
Keyboard shortcuts window
Jakub Brindza joined us as a GSoC student. The first step of his project is already done, a brand new keyboard shortcuts window built dynamically, to be able to support custom shortcuts. Next will be allowing users to customize the keyboard shortcuts.
Other noteworthy changes
The user manual has been updated to reflect the changes since we switched to the headerbar, about one year and a half ago.
The undo/redo removal has been undone.
We switched from AppImage to Flatpak to distribute our own builds. A big advantage is that due to Flatpak’s sandboxing we can use it for the development environment while previously we needed a separate complex script to take care of that.
* There was a very high chance of ending up with artifacts at the start of a clip with an inpoint.
This is the first blog post of a series of updates about our latest development efforts in GStreamer / gst-editing-services / Pitivi.
This post’s focus will be on MPEG transport stream, a format now nearly twenty years old, originally developed and still widely used for and by the broadcasting industry. In the mid-2000s, some people decided it would be a great idea to use this format in camcorders, stuffed a rather useless timestamp in there for good measure and started to ship AVCHD camcorders like it was the greatest thing since sliced bread.
It was not. See, most modern video codecs such as h264 rely on the notion of keyframes: to compress video streams as much as possible, most frames are encoded as their difference with the previous frame, we call these frames delta units. Sparsely distributed in the encoded stream are keyframes. These frames can be decoded without any reference to past frames, and when trying to seek in such a stream, decoding has to start from a keyframe and progress through all the delta units up to the requested frame.
Video editing implies accurate seeking, for example if you only want to include the 10 last frames of a 2-hour clip, decoding the whole file to obtain these few frames would be a pointless waste of time.
Failing to start decoding from a keyframe when seeking creates artefacts, garbled frames : the decoder is missing information to decode the delta unit, tries to provide a frame nevertheless and fails in doing so, until the next keyframe has been reached. Containers that are readily usable for editing contain information about the location of keyframes, in one form or another. This is not the case of MPEG TS, of which AVCHD is a subset. Locating the keyframes thus becomes a rather involved process, as one needs to parse the video streams in order to do so.
Backtracking to the introduction of this post, good news everyone ! We just did that, and here is a before / after video to demonstrate our changes. We can now ensure full support of AVCHD, enjoy 😀
The next two posts will be respectively focused on our refactoring of our video / audio mixing stack, and our ongoing work on gnonlin, our non-linear editing engine.