[css-view-transitions-1] Handling mix-blend-mode on elements captured for a transition

[khushalsagar]

Summarizing the discussion here.

Let's say a view transition is capturing the root and element foo, and foo is semi-transparent with a non-normal mix-blend-mode. And let's say the author defines a very basic animation, e.g. sliding the old root out to the right and element foo out to the left to reveal the new document underneath.

The mode specifies how an element blends into its target stacking context. There is no conceptual way to bake the blending mode into the element's image itself so capture the image ignores this mode. But what this means is that as soon as the transition starts, the blending of foo against its background changes, resulting in a visible change in the color of foo.

One way this could be handled is to have the mix-blend-mode on one the ::view-transition pseudos which will display this image. But that has the following issues:

1. The mix-blend-mode can be copied over to the replaced pseudo which is displaying the image. But by design, the mix-blend-mode on the ::view-transition-old/::view-transition-new pseudos is set to plus-lighter for the perfect cross-fade. The cross-fade shouldn't be noticeable if the images are exactly the same.
2. The mix-blend-mode can be copied over to the ::view-transition-group for foo. It would then be conceptually similar to the element's viewport position, properties which don't make sense for the element in isolation but define how the element is laid out and composited into the Document embedding it. Except as opposed to transform, there is no way to animate it. So the mix-blend-mode will immediately flip from the old to new value instead of interpolating.

---

Note that option 2 above only works if the element's closest ancestor that is also being captured into an image is the root. For example, if you have a tree like this:

root
|_B mix-blend-mode: multiply;
   |_C mix-blend-mode: multiply;

The blending order in the DOM is : Blend(root, Blend(B, C). If B and C are being captured for a transition, then the pseudo tree is:

::view-transition-group(root)
|_::view-transition-image-pair(root)
    |_::view-transition-old(root)
    |_::view-transition-new(root)
::view-transition-group(B)
|_::view-transition-image-pair(B)
    |_::view-transition-old(B)
    |_::view-transition-new(B)
::view-transition-group(C)
|_::view-transition-image-pair(C)
    |_::view-transition-old(C)
    |_::view-transition-new(C)

root, B and C become siblings so the blending order is Blend(Blend(root, B), C) which is not the same as what's in the DOM.

We have the same problem with a bunch of other properties that depend on the DOM's hierarchy. For example, as soon as the transition starts you'll stop being clipped by an ancestor which is participating in a transition. Nested transition groups is the principled fix for it. Authors would opt into this mode in which case group pseudo for C will be nested inside group pseudo for B which itself will be nested inside group pseudo for root.

[fantasai]

Option 2 seems to make sense to me? If the author picked two different mix-blend-modes for the same element, that's kindof their problem. :) And when we have nesting, it seems like Option 2 will work reasonably well with it for the more deeply nested compositing cases.

[khushalsagar]

If the author picked two different mix-blend-modes for the same element, that's kindof their problem.

+1.

Option 2 seems to make sense to me?

For mix-blend-mode yeah. Trying to think through other CSS properties which could be conceptually similar.

Early on, we were debating whether effects like filter, opacity or clip-path should be lifted onto the ::view-transition-group instead of baking them into the captured image. But there were use-cases where that doesn't make sense. For example, you have a widget's content being replaced with another with a slide-and-fade animation. It was common to see customization where the image does a full slide but the fade animation finishes half way through the slide, such that you see the widget contents completely midway during the slide.

Now if the morphing widgets have different opacity values, we have 2 options:

1. opacity is baked into the image. Such that any fades during the transition are on top of all the effects on the element.
2. opacity from the element is copied over to the ::view-transition-group and animated like transform.

Both are valid use-cases. The spec currently does 1 but we plan to have a mode to support 2.

I'm convinced that mix-blend-mode is different from other effects since it can't be "baked" into the image. So it must be copied onto the group. But not sure if there are other properties like mix-blend-mode.

[css-meeting-bot]

The CSS Working Group just discussed [css-view-transitions-1] Handling mix-blend-mode on elements captured for a transition, and agreed to the following:

• RESOLVED: Add mix-blend-mode to list of properties copied from the element to the group pseudo

The full IRC log of that discussion

<TabAtkins> khush: when we're doing any kind of animation, we try to bake as many things as possible into the snapshot itself
<TabAtkins> khush: but some just can't be done, like mix-blend-mode
<TabAtkins> khush: Right now it just gets dropped on the floor
<TabAtkins> khush: so if you want the pseudo-dom to reflect the real dom's rednering, the best way to handle m-b-m is to copy it over to group pseudo
<TabAtkins> khush: And more generally, for other properties that can't be baked into the snapshot, we propose to copy them over to the group pseudo
<TabAtkins> noamr: if it's copied to the group, what happens if the two states have different m-b-m
<TabAtkins> khush: it doesnt interpolate so it'll flip immediately to the new value
<TabAtkins> emilio: Do we have a list of which properties are meant to be baked into the iamge than the group?
<TabAtkins> noamr: it's in the spec
<TabAtkins> khush: opacity/filter/etc are baked into the snapshot, and the list of copied properties are very epxlicit
<TabAtkins> khush: There's a "capture these proeprties" step in the algo
<TabAtkins> khush: So if there are more properties to be copied we'll bring those in another issue
<TabAtkins> emilio: it feels a little weird to special-case some props vs others, but yeah
<TabAtkins> astearns: Do we run the risk of ahving this list continue to be expanded?
<TabAtkins> SebastianZ: maybe we have to put these special properties in some defined group
<TabAtkins> SebastianZ: I saw some more properties that are on the snapshot, like clip-path. ARe mask in that group?
<TabAtkins> khush: All of those are baked into the snapshot itself
<khush> https://drafts.csswg.org/css-view-transitions-1/#captured-elements
<TabAtkins> khush: The set of properties that are copied as properties is in the spec here...
<TabAtkins> vmpstr: basically what we copy are things that interact with other content, not just the subtree
<TabAtkins> vmpstr: So opacity/clip-path only affect the element, but mix-blend-mode interacts with something else and we can't capture that statically
<TabAtkins> vmpstr: I was thinking backdrop-filter might be one of those props, but i'm not sure how it works
<fantasai> TabAtkins: backdrop-filter lives in a similar space to mix-blend-mode
<TabAtkins> vmpstr: i just dont know if the topology is important for the tree
<TabAtkins> vmpstr: The element bieng affected is in a pseudo-element that is a pseudo-child of the root...
<astearns> q?
<TabAtkins> khush: It does matter
<TabAtkins> khush: There's a feature extension of VT that will let you not lift the VT pseudo up to the root
<TabAtkins> khush: So that'll be necessary to get backdrop-filter to work correctly
<TabAtkins> astearns: I think some prose explaining why this list exists would be good in the spec
<TabAtkins> khush: good idea, i'll add a note with motivation
<TabAtkins> emilio: are the snapshots we take semi-transparent? or do we composite with the background
<fantasai> s/in the spec/in the spec, that these are not effects that can be captured in the snapshot/
<TabAtkins> khush: The base can be semi-transparent
<fantasai> ACTION: khush to add a note about the motivation for the copied properties
<TabAtkins> khush: it's composited just as part of being rendered in the pseudo
<TabAtkins> vmpstr: the v-t-name causes a stackign context
<fantasai> TabAtkins: I think the conclusion is that while backdrop-filter is in the right spot for this, it's not usable with VT atm
<TabAtkins> emilio: So that means mix-blend-mode works okay so that's fine
<TabAtkins> emilio: Stil seems weird to special-case some properties, but better than dropping it on the floor
<TabAtkins> astearns: so proposed res is to add mix-blend-mode to the list of properties
<TabAtkins> SebastianZ: and opacity/clip-path/mask?
<fantasai> [various other properties mentioned are baked into the snapshot]
<TabAtkins> astearns: They're already cpatured in the snapshot
<TabAtkins> RESOLVED: Add mix-blend-mode to list of properties copied from the element to the group pseudo
<TabAtkins> astearns: The list is a little weird but we do need to have this behavior captured
<TabAtkins> astearns: and as people use VT and notice things being dropped form the snapshot, people should have an idea of where to fix it, so the note will be good