Per-title optimization in CODECbench

CODECbench can easily be used to calculate ‘Per-Title’ optimization values when encoding content. The main idea behind Per-Title optimization is that one should choose bitrate values for encoding that depend on the rate distorsion curves for the content being encoded. Netflix has a good overview of how this works in their platform. Per-Title tries to answer these questions:

  • Given a desired video quality, what’s the optimal bitrate for this content?
  • Given a number of different resolutions for video available, at what moment does one need to switch from one resolution to the next?

Netflix and others approach this problem by drawing a convex hull on top the of the PSNR curves of the different resolutions, then drawing samples from this convex hull to obtain the resolution x bitrate pairs.

CODECbench does this slightly different by using a full scale metric like SSIM and establishing a minimum desired quality for such metric across the encoding of the title. By using a full metric CODECbench avoids having to derive a convex hull line that has to be suboptimally sampled. Then using the ‘Optimal Bitrate Resolution Switching’ report we can calculate our switch points.

Let’s look at an example for the coastguard sequence SSIM RD curve in 4 different resolutions..

In order to find out the bitrate switching points, we just have to define a ‘minimum’ level of quality desired for the sequence. For example values of 0.8, 0.9 and 0.95 would be reasonable for SSIM. We add a ‘minqual’ parameter to the SSIM report to obtain the switching points:

"minqual": [0.8, 0.9, 0.95] #minimum quality required. Scalar values are acceptable too

One the sequences above this produces for 0.9 and 0.95:

Bitrate switch for 1920×1080@0.9 at 1658351.3268310914 bps. 1280×720 qual at br is 0.9324741017417775
Bitrate switch for 1280×720@0.9 at 989156.8165966908 bps. 640×480 qual at br is 0.9428834337863248
Bitrate switch for 640×480@0.9 at 485318.3305768017 bps. 320×240 qual at br is 0.9349399124018616
Minimum bitrate for 320×240@0.9 at 280510.36393845885 bps.

Bitrate switch for 1920×1080@0.95 at 3857779.601799102 bps. 1280×720 qual at br is 0.9628820330960579
Bitrate switch for 1280×720@0.95 at 2493719.9488699837 bps. 640×480 qual at br is 0.9737948824384565
Bitrate switch for 640×480@0.95 at 1163040.4454250892 bps. 320×240 qual at br is 0.9714908213551666
Minimum bitrate for 320×240@0.95 at 659464.021887705 bps.

Notice how precisely at the point where the switch happens the previous resolution will always have a higher quality metric. This is expected, as the resolution we are switching from is being encoded with more bits than needed.

Go ahead and check this out with your own sequences and CODECbench

AV1 support added to codecbench

codecbench has been just updated to support AV1. AV1 support is enabled through the use of ‘aomenc’ and ‘aomdec’ which are the encoder and decoder utilities from the AOM project. MacOS binaries are provided in the current github release. If you want to use other platform you can drop in your ‘aom[enc|dec]’ binaries into the respective codecpack, located under [codecbench_root]/codecpacks/aom/[platform_dir].

Additionaly we are providing a sample configuration file to get you started. You can find the [codecbench_root]/sampleruns/av1.json config file which produces the following PSNR report:

The report is produced by the following file:

  • ‘cpu’ setting controls the ‘cpu_used’ setting inside aomenc. 0 is slowest, 8 is fastest
  • aom_codec only supports ‘av1’ at the moment. If aom libraries add more codecs you’ll be able to change them here.
  • all other parameters should remain compatible with this codecpack
{
    "sequence_dirs": ["../seq"],
    "codec_dirs" : ["../codecs"],
    "keeprecon" : false,
    "runs" : [
    {
        "seq" : ["foremancif"],
        "codec" : ["aom"],
        "bitrate_range" : [ 50,1500,100],
        "cpu" : [0,8],
        "aom_codec" : ["av1"],
        "ignore" : false,
        "clobber" : false
    },
    {
        "seq" : ["foremancif"],
        "codec" : ["x264","x265"],
        "bitrate_range" : [ 50,1500,100],
        "ignore" : false,
        "clobber" : false
    }
    ],
    "reports" : {
        "defaults" : {
            "res" : "800x600",
            "fontsize" : 8,
            "format" : "png"
        },
        "reports": [
        {
            "metric" : "ssim"
        },
        {
            "metric" : "psnr"
        }
        
    ]
} 
}


4K sequence set released


Remember “The Classics”? ForemanCIF, MobileCIF… they are back in 4K and in full force Elemental technologies has created updated versions of these video testing clips in 4k format. We have taken them up and they are now part of our main testing benchmark setup for some of our CABSscore . You can download originals here but we have prepared a CODECbench ready YUV sequence sets, with json metadata information attached to them.

Download 7Zipped file here. ( 11.32GB 7zip compressed, Creative Commons License )

Description of individual sequences from json metadata files follows as well as youtube clips of the video content. For original files and benchmarking, please download file above.


parkjoy4k

This is a very hard sequence to encode. There is an amazing level of detail in it while it’s panning globally. Additionally, there is a few people running in the opposite direction of the global panning that can be seen in the distance.

Things to look for

  • Correct rendering of peoples clothing and objects
  • Correct rendering without chroma aberrations of umbrellas
  • Water reflection should not have observable blockiness
  • Tree leaves everywhere should be clearly visible and without artifacts
  • Tree trunks panning at the beginning of the sequence should not leave trails behind
{
    "filename": "parkjoy4k.yuv",
    "format": "420P",
    "frame_count": 500,
    "framesize": 12441600,
    "height": "2160",
    "width": "3840",
    "nickname" : "parkjoy4k",
    "fpsnum" : 50,
    "fpsden" : 1
}


mobile4k

This sequence shows a man talking in front of the camera with heavy head motion and then pans onto a construction site.

Things to look for

  • Artifacts around the moving head of the construction worker
  • Has has rain drops on it and should be rendered and visible clearly at 4k
  • Smooth panning without artifact trailing
  • Clean introduction of all the new scene content (the construction site) without streaks
{
"filename": "mobile4k.yuv",
"format": "420P",
"fpsden": "1001",
"fpsnum": "24000",
"frame_count": 355,
"framesize": 12441600,
"height": "2160",
"nickname": "mobile4k",
"width": "3840"
}


foreman4k

This sequence presents global panning from left to right while many items are moving in the opposite direction inside the frame.

Things to look for

  • The sharpness of the numbers in the calendar against the white calendar
  • The artifacts shown when the atomium object shows up from the left of the screen at 4s
  • The consistency of the red color of the spinning ball in front of the train
  • The correct rendering of the static panning background
{
"filename": "foreman4k.yuv",
"format": "420P",
"fpsden": "1001",
"fpsnum": "24000",
"frame_count": 248,
"framesize": 12441600,
"height": "2160",
"nickname": "foreman4k",
"width": "3840"
}

cactus4k

This is a simple sequence that tests global zoom out of a camera.

Things to look for

  • Smooth zoom out without artifacts
  • Correct introduction of cactus thorns and comb spikes
{
"filename": "cactus4k.yuv",
"format": "420P",
"fpsden": "1001",
"fpsnum": "24000",
"frame_count": 334,
"framesize": 12441600,
"height": "2160",
"nickname": "cactus4k",
"width": "3840"
}

coastguard4k

This is a complex sequence due to the ‘noise-like’ nature of the water waves and ripples. A codec should be able to properly identify global motion and render the water without blockiness or artifacting.

Things to look for

  • No artifacts before global panning starts
  • Overall quality of rendering of water features (waves, ripples) over the scene
  • Chroma aberrations on the flag of the ship
{
"filename": "coastguard4k.yuv",
"format": "420P",
"fpsden": "1001",
"fpsnum": "24000",
"frame_count": 240,
"framesize": 12441600,
"height": "2160",
"nickname": "coastguard4k",
"width": "3840"
}

suzie4k

Easy video conference style scene with static background.

Things to look for

  • Scene should be able to be coded at low bitrates
  • No artifacts should be observed during head movement
{
"filename": "suzie4k.yuv",
"format": "420P",
"fpsden": "1001",
"fpsnum": "24000",
"frame_count": 355,
"framesize": 12441600,
"height": "2160",
"nickname": "suzie4k",
"width": "3840"
}