Author Topic: Reference guide for spotting encoding problems  (Read 34260 times)

Offline cyberbeing

  • Member
  • Posts: 62
Re: Reference guide for spotting encoding problems
« Reply #20 on: October 02, 2011, 11:53:23 pm »
if so it's already under "oversharpening/ringing". for me ringing and haloing is the same.

The two have unrelated causes and effects, so it's best to distinguish ringing from haloing.

Ringing usually refers to a type of blocking artifact which happens on hard edges from not enough bitrate or misuse of AQ.

Haloing is a side-effect from strong sharpening effects, which causes bright 'halos' around hard edges. When it's really bad, you can have multiple halos.


Could you also add a section for 'Dirty lines' (brightened, darkened, or discolored lines on the edges of a frame which contain parts of the image), since that has come up a couple times recently? Possibly something like 'Wrong aspect ratio', 'Overcropping/Cropped-source', and 'Ghosting/Double-image' as well.
« Last Edit: October 03, 2011, 06:07:38 pm by der richter »

Offline tormaid

  • Member
  • Posts: 12
    • Tormaid.com - HD Anime Torrents
Re: Reference guide for spotting encoding problems
« Reply #21 on: October 03, 2011, 12:36:55 am »
1080p is just too huge in dimensions. nearly no one would be able to few the whole site with 1080p images. they will lose the overview. 720p its just the right size and all the problems listed can be found on releases with 720p or smaller resolution.

did you imply adding haloing to the list? if so it's already under "oversharpening/ringing". for me ringing and haloing is the same. please correct me if i'm wrong. if you want me to add your example please upload the pictures separately on a image host.

this is not the right place to discuss your fondness of heavy debanding. ^^
I still think that blocking and other artifacts caused by bitrate starvation are less obvious at 720p, but just a suggestion.

The previous commenter explains the difference nicely, I'd only add that the description of such things should explain the balancing act between sharpening, aliasing, and haloing.

The last was simply in agreement with Desbreko's first comment.

« Last Edit: October 03, 2011, 12:38:59 am by tormaid »

Offline El Mendigo

  • Member
  • Posts: 76
Re: Reference guide for spotting encoding problems
« Reply #22 on: October 03, 2011, 01:39:48 am »
Quite a few BD rips I've seen also suffer from cross-conversion artifacts. It's probably easy to mistake for regular combing due to telecine or interlacing, but it's not the same thing and needs another approach for fixing.

source | fixed

The second girl from the left's Alice band shows the fix quite well.

EDIT: Oh, a description too. Hmm... How about something like; These kinds of artifacts, while similar to telecine and interlacing ones, are a result of a poorly done conversion between different video formats. In the example case from 720p to 1080i. Most often seen on Blu-rays. Distinguishable from telecine and interlacing artifacts by the fact that every frame appear combed even if there is absolutely no motion taking place.
« Last Edit: October 09, 2011, 04:58:51 pm by der richter »

Offline Desbreko

  • Member
  • Posts: 614
Re: Reference guide for spotting encoding problems
« Reply #23 on: October 03, 2011, 01:51:15 am »
he suggested "double-hard telecine" for the video section. so i would rather add your images to "poor deinterlacing" in the static problems section. that is how it looks to me.

Double-hard telecine is both an IVTC and deinterlacing problem and you really need to see it in motion to tell what's going on, yeah. Otherwise it will just look like any other badly deinterlaced frame.

It might be worth separating poor deinterlacing and bad field matching in the static section. While the two tend to go together, the former results in jagged lines from deinterlacing bad field matches (truly interlaced content is pretty rare in anime except in scene change effects), but the latter will result in residual combing like in jackoneill's example if the post-processing deinterlacing doesn't catch it.


Double-Hard Telecine

Example: bad, good

What is it?
Two different telecine pulldown patterns applied to one section of video. Since only one of the patterns can be field matched and decimated correctly, the combed frames from the other pattern must be interpolated with a deinterlacing filter. It's most commonly found during cross-fades and other scene change effects in hard-telecined sources.

What does it look like?
- Combing left on some but not all moving parts of the frame
- Jagged lines from poor deinterlacing on some but not all frames in the scene
« Last Edit: October 03, 2011, 05:44:35 pm by der richter »

Offline der richter

  • Former Staff
  • Member
  • Posts: 1235
  • Judgment
Re: Reference guide for spotting encoding problems
« Reply #24 on: October 03, 2011, 01:53:03 am »
Deinterlacing has nothing to do with this. There is an explanation towards the end: http://searchcio-midmarket.techtarget.com/definition/S-Video
i changed it to "Is a discoloration on lines." for now. sry for my lack of knowledge i really thought it is deinterlacing bound.

Quote
What is deinterlacing?
Is the process of converting a interlaced video into a progressive form. Interlaced video is a sequence of pictures where every other frame contains only the even lines of an image, the rest contains odd lines. A sequence of two images combined is one full image.
Pretty much wrong again. :(
k, i'm really certain i know what interlacing/deinterlacing is. it may be my lack of english why the description failed. i had a really hard time explaining it but i'm not really satisfied. the first part should be right "Is the process of converting a interlaced video into a progressive form."

here what i meant:
source   interlaced Upper Field    interlaced Lower Field   deinterlaced
111111
222222
333333
444444
555555
111111
000000
333333
000000
555555
000000
222222
000000
444444
000000
111111
222222
333333
444444
555555
for example 25 fps progressive video -> 50 fps interlace video -> 25 fps progressive video.

it's not hard to spot those problems but explaining it is another thing ^^. maybe you can help me with the right wording, i don't want to copy the wiki entry. it's too wordy and most people won't read it at all.

Quote
Also, dot crawl should appear in the static section, too, because even in stills, the checkerboard pattern can be quite obvious:
Badgood
(see the red text in the top left corner).

i thought i will add example images/comparisons to the video comparisons, so we don't need an extra entry for it.

The two have unrelated causes and effects, so it's best to distinguish ringing from haloing.

Ringing usually refers to a type of blocking artifact which happens on hard edges from not enough bitrate or misuse of AQ.

Haloing is a side-effect from strong sharpening effects, which causes bright 'halos' around hard edges. When it's really bad, you can have multiple halos.
i see, they have unrelated causes, but they are bit similar. lot of people referring to it as the same error (so do/did i). i will add separate entries.


Could you also add a section for 'Dirty lines' (brightened, darkened, or discolored lines on the edges of a frame which contain parts of the image), since that has come up a couple times recently? Possibly something like 'Wrong aspect ratio', 'Overcropping/Cropped-source', and 'Ghosting/Double-image' as well.
i initially thought "edge noise" would cover that, but what you mean with "Dirty lines" is a bit different. maybe name it "Dirty Borderlines" this may cover edge noise and the discolored/brightened/darkened lines. what do you think?

Wrong aspect ratio is a severe problem but it's hard to demonstrate with two pics. one is squeezed and one is less squeezed. you just need the hang to notice it without a source pic. but will see what we can do about it. maybe an explanation with an image from a round object. on one screen the object is more oval and on the other more like a circle. will add overcropping and ghosting.

@tormaid
could you elaborate "explain the balancing act between sharpening, aliasing, and haloing." we want to make a simple guide, this may be a bit to much tech talk for most of the people. just stating the problem and simply state what it cause.

@El Mendigo
it may need another approach of fixing but it looks just like deinterlacing or IVTC problem


all the comparison images are really nice and i appreciate the effort, but some explanations and descriptions would also be nice. ^^

[edit]
@Desbreko
what exactly is "bad field matching". i may be wrong but is it something like this or this?


i'm sorry. i may be a bit unknowledgeable, but that is exactly the reason why i (personally) started this. i hope you can bear with me.
« Last Edit: October 03, 2011, 02:09:07 am by der richter »

Offline Desbreko

  • Member
  • Posts: 614
Re: Reference guide for spotting encoding problems
« Reply #25 on: October 03, 2011, 02:10:13 am »
Bad field matching it just the IVTC filter picking the wrong match-up of two fields when reordering them to restore the original progressive frames. It results in a combed frame instead of the original frame. In jackoneill's example pictures, the soccer ball is combed in the bad shots instead of being solid like it should be in the good shot. That's an example of a bad field match.

Offline El Mendigo

  • Member
  • Posts: 76
Re: Reference guide for spotting encoding problems
« Reply #26 on: October 03, 2011, 02:20:23 am »
Moar:

Aliasing. Hourou Musuko is the god of aliasing.

source | fixed

Chroma-shifts. First one is sort of extreme (Look at her face, or at the red on her boob and sword). The second one is a lot harder to notice but there.

source | fixed
source (closeup) | fixed (closeup)

Chroma shifts usually appear as if the colors (i.e. not the blacks, grays and whites) of the video has moved a couple of pixels in any direction, causing it to look shifted. It is usually noticeable on deep reds and blues and around hard, black edges, such as the line art of animation. There, a white line or an overlap of a color and black may appear, depending on the actual direction of the shift.


EDIT: I added a poorly formulated description of the cross-conversion problem in that post. Added crappy description of chromatic shifts in this here post.
« Last Edit: October 03, 2011, 04:15:37 am by der richter »

Offline tormaid

  • Member
  • Posts: 12
    • Tormaid.com - HD Anime Torrents
Re: Reference guide for spotting encoding problems
« Reply #27 on: October 03, 2011, 03:15:35 am »
@der richter
I just mean that when sharpening you create both haloing and aliasing, most often neither of these are addressed properly. If to address these problems the encoder needs to consider a few things: firstly that if they use too much AA the edges get blurry, and if they use too much dehaloing the edges become too light and you loose most of the sharpening you were trying to achieve. You have to balance these things when applying sharpeners.

Offline cyberbeing

  • Member
  • Posts: 62
Re: Reference guide for spotting encoding problems
« Reply #28 on: October 03, 2011, 03:29:37 am »
Another good thing to add would be 'I-frame/keyframe pulsing' under the video section.
« Last Edit: October 03, 2011, 06:23:55 pm by der richter »

Offline Mirek

  • Member
  • Posts: 38
Re: Reference guide for spotting encoding problems
« Reply #29 on: October 03, 2011, 09:15:04 am »
Double-hard telecine:

bad|good

It's an example of source treated in the wrong way resulting in jerky pan.
« Last Edit: October 03, 2011, 06:48:12 pm by der richter »

Offline jokli

  • Member
  • Posts: 104
Re: Reference guide for spotting encoding problems
« Reply #30 on: October 03, 2011, 11:05:35 am »
If I remember correctly, some rips have problems with their color plane being shifted a bit causing color halos. Perhaps you can include that?

Offline doll_licca

  • Member
  • Posts: 39
  • Founder of Licca Fansubs
Re: Reference guide for spotting encoding problems
« Reply #31 on: October 03, 2011, 03:55:57 pm »
If I remember correctly, some rips have problems with their color plane being shifted a bit causing color halos. Perhaps you can include that?
I've actually got a really good (and rather extreme) example of that; I'll post it later this afternoon.

Offline der richter

  • Former Staff
  • Member
  • Posts: 1235
  • Judgment
Re: Reference guide for spotting encoding problems
« Reply #32 on: October 03, 2011, 07:33:18 pm »
Possibly something like 'Wrong aspect ratio', 'Overcropping/Cropped-source', and 'Ghosting/Double-image' as well.

i'm not sure where to put the ghosting part. i think it would be better to put it into the video section?

Quite a few BD rips I've seen also suffer from cross-conversion artifacts. It's probably easy to mistake for regular combing due to telecine or interlacing, but it's not the same thing and needs another approach for fixing.

source | fixed

The second girl from the left's Alice band shows the fix quite well.

EDIT: Oh, a description too. Hmm... How about something like; These kinds of artifacts, while similar to telecine and interlacing ones, are a result of a poorly done conversion between different video formats. In the example case from 720p to 1080i. Most often seen on Blu-rays. Distinguishable from telecine and interlacing artifacts by the fact that every frame appear combed even if there is absolutely no motion taking place.

i don't know where to put this exactly. it looks a bit like poor deinterlacing. maybe a video would be better to explain it?

Also, dot crawl should appear in the static section, too, because even in stills, the checkerboard pattern can be quite obvious:
Badgood
(see the red text in the top left corner).

i would rather take this with a video example if you don't mind making one. ^^

I just mean that when sharpening you create both haloing and aliasing, most often neither of these are addressed properly. If to address these problems the encoder needs to consider a few things: firstly that if they use too much AA the edges get blurry, and if they use too much dehaloing the edges become too light and you loose most of the sharpening you were trying to achieve. You have to balance these things when applying sharpeners.

i considered the first part and added it accordingly. the second part is to much on the encoding part. it would rather help encoding videos than help spotting errors. ^^ thx for the explanation.

Another good thing to add would be 'I-frame/keyframe pulsing' under the video section.

added this for now, even though i don't know what exactly this is.

@all
i added everything i could add. if you want please review it, there may be some errors. also what do you think about the video section? in my opinion it's sufficient (with the screenshot comparison) and the mentioned problems won't need multiple entries in the static and video section. hope i didn't forget anything.

Offline El Mendigo

  • Member
  • Posts: 76
Re: Reference guide for spotting encoding problems
« Reply #33 on: October 03, 2011, 08:19:00 pm »
i don't know where to put this exactly. it looks a bit like poor deinterlacing. maybe a video would be better to explain it?

I wouldn't mind seeing it under poor deinterlacing really. Probably no one except encoderfags would be able to tell the difference anyway. I'll edit in a link to a video into this post soonish.

But maybe the title of "Poor Deinterlacing" could be changed to "Combing" or something of the like, since that would also include faulty IVTC and other lookalike problems which are not actual deinterlacing errors.

EDIT: wrong :( | correct :D
« Last Edit: October 09, 2011, 05:09:52 pm by der richter »

Offline doll_licca

  • Member
  • Posts: 39
  • Founder of Licca Fansubs
Re: Reference guide for spotting encoding problems
« Reply #34 on: October 04, 2011, 01:02:18 am »
Okay, here's a really extreme example of chroma shifting from Licca-chan: Fushigi na Mahou no Ring  There's a lot of other issues with the bad video in question, but that was probably the most annoying one:

Basically, look at the moon, and look at Licca's dress and how there's a shadow of it a bit to the left.

This is a good version of that scene.


Basically, chroma shifts occur mostly in older footage that was transferred to LD or DVD using faulty equipment, due to the luma and the chroma being transmitted out of phase with each other during the transfer.
« Last Edit: October 09, 2011, 03:43:03 pm by der richter »

Offline cyberbeing

  • Member
  • Posts: 62
Re: Reference guide for spotting encoding problems
« Reply #35 on: October 04, 2011, 01:06:07 am »
i'm not sure where to put the ghosting part. i think it would be better to put it into the video section?
It could be done either way. For example, doll_licca's top image above shows noticeable ghosting on the right side of the characters.

Another good thing to add would be 'I-frame/keyframe pulsing' under the video section.

added this for now, even though i don't know what exactly this is.

Since the issue I'm thinking of doesn't fall under the strict definition of 'I-frame pulsing' it could probably use a better name. Possibly something like 'Rollercoaster Ratecontrol', 'Ratecontrol pulsing', or 'Quality pulsing' would be a more self explanatory title.

The basic idea is as follows:

Hard to compress source. (for example: anime with heavy grain)

Long scenes that surpass the max-keyint without a scene-change.

I-frames in such a scene causes an extremely sharp but temporary jump in quality while the surrounding P-frame & B-frames near-instantly turn to crap. Most easily seen with grainy sources, since it would usually mean quite obvious and sudden fluctuations of blurry/blocky and sharp within a long scene, when optimally you want a smooth transition of quality between frame-types.

It was mainly an issue with old builds of x264, before MBTree, RC-Lookahead, and WeightP/WeightB stuff was added. My most memorable experience of this problem was encoding the Kara no Kyoukai DVDs back in 2008.
« Last Edit: October 04, 2011, 01:09:22 am by cyberbeing »

Offline Desbreko

  • Member
  • Posts: 614
Re: Reference guide for spotting encoding problems
« Reply #36 on: October 04, 2011, 01:36:26 am »
I wouldn't put this in the guide, but you can see an extreme example of the quality pulsing thing in this old YouTube video of mine. Watch how the sand pattern on the floor gradually gets blurrier and more discolored, and then how it pops back to normal when the video hits a key frame.

Edit: While I'm at it, here's an example of another type of chroma shift:

bad, good

Here, the whole chroma channel hasn't just been shifted in one direction. The blue hues have been shifted down and right while the red/green hues (which appear as yellow) have been shifted up and left.
« Last Edit: October 09, 2011, 03:52:53 pm by der richter »

Offline junh1024

  • Member
  • Posts: 2
Re: Reference guide for spotting encoding problems
« Reply #37 on: October 04, 2011, 06:45:14 am »
Possible Colour Problems:

Not fixing limited <-> full range luma issue and/or changing brightness/contrast such that it falls outside the legal limited range (need a histogram to check)

Modifying Brightness/Contrast/Saturation with an algorithm that doesn't dither, resulting in spikes/gaps/discontinuities in histograms


Colour correction lack thereof, or badly done (colour is a big issue on DBZ releases)

Incorrect colour matrixing

Possible Audio  Problems

Normalizing anime series, which risks inconsistent volume levels across episodes. A few incidents of clipping per episode is OK.

Boosting volume levels (this is not a lossless operation, even if the source & destination is lossless format), or other manipulations, unless it is to fix a specific audio problem and is not overdone. Bad: http://i.imgur.com/FSzDz.jpg Good: http://i.imgur.com/sCs7t.jpg

Upsampling to higher sampling rate or bitdepth than what was on the source (or changing 24<->16bits, downsampling to 44k unless for good reason)
Bad examples: http://i.imgur.com/ikgac.jpg http://i.imgur.com/YJPta.jpg http://i.imgur.com/DCT2p.jpg
Good: http://i.imgur.com/2ZfeV.jpg

Decoding lossless audio with a lossy decoder and reencoding to lossless (Quite easy to spot with AC3, could be harder to spot with DTS).

Inferior audio quality or encoder (Nero, Aften AC3, Reference OGG, Surcode DTS, etc) as opposed to superior (qAAC or any other that uses the Quicktime engine, Mainconcept AAC (found in Mainconcept Reference) Dolby Digital Professional, DTS Master Audio Suite )

Not including a Stereo track where the Surround track is an upmix of the stereo track.

Generally for series, stereo is native, surround is suspect, for movies, surround should be fine. E.g, Clannad both seasons, surround was an obvious upmix but most groups did not include 2.0 track. For Rurouni-Kenshin Seisouhen both 2.0 and 5.1 tracks were suspect. See case study.

Short explanation: http://uguu-subs.org/2010/06/28/delays-and-technical-tidbits/ "Ive also started including the 2 channel mix in FLAC..."
Long with case study: https://docs.google.com/document/d/1H22TMCryBzYys4oD-vj4qBT-ujSXNhJPm8_T-a5cx5c/edit?hl=en_US

Offline Desbreko

  • Member
  • Posts: 614
Re: Reference guide for spotting encoding problems
« Reply #38 on: October 08, 2011, 10:13:06 pm »
While this is a deinterlacing issue, I'd give it its own section since it's handled differently than normal deinterlacing and is best seen in motion:

Interlaced Fades

Example: bad, good

What is it?
Interlacing during fade-ins/fade-outs or cross-fades between scenes. It appears as a faint combing pattern on every frame during a fade.

Where to find it?
- Scene change fades in hard-telecined sources
- Most obvious in areas fading between light and dark colors
« Last Edit: October 09, 2011, 04:27:40 pm by der richter »

Offline der richter

  • Former Staff
  • Member
  • Posts: 1235
  • Judgment
Re: Reference guide for spotting encoding problems
« Reply #39 on: October 09, 2011, 05:45:59 pm »
But maybe the title of "Poor Deinterlacing" could be changed to "Combing" or something of the like, since that would also include faulty IVTC and other lookalike problems which are not actual deinterlacing errors.
i think we will consider this when the guide is getting too cluttered. for now first priority is gathering infos and adding them.

also i would rather name it "poor combing" because "combing" isn't a problem.

@junh1024
the first brightness/contrast comparison is nice. but i can't add it like this. it would be nice to have all those images separated and in unaltered form. the screens without the yellow text and the diagrams as separated images.

the audio part is really interesting, and i may add it at a later time. but for now it has a low priority. also it would be nice to have those diagrams in a cleaner form. without the jagged red highlights you added (i will add them later and cleaner). some explanations would be nice too.

i can't guarantee the inclusion of it, because it's a rather hard thing to spot without a separate program which is showing those diagrams. most people won't bother to do it. after all the target of this guide are people with little to no knowledge about encoding. but that is open for discussion and i really like it. ^^

@Desbreko
it was really hard for me to spot it on this short fade. hope i highlighted it correct.

@all
added the remaining things. a little change i added a "Unsorted" category for things i'm not sure about and/or just for rechecking purpose. i will add everything new there first, so you can look over it before i add it to the right category. i will sort/add them every weekend so there should be enough time to check everything.

i also will add a link to the proper AniDB page of the anime for every example. just have to upload it later. ^^

[edit] site updated
« Last Edit: October 09, 2011, 08:35:21 pm by der richter »