When drawing transparent surfaces, rgl tries to sort objects
from back to front to get better rendering of transparency.
However, it doesn’t sort each pixel separately, so some pixels
end up drawn in the incorrect order. This note
describes the consequences of that error, and
suggests remedies.
We’ll assume that the standard glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) blending is used. That is,
when drawing with transparency \(\alpha\), the new colour is
mixed at proportion \(\alpha\) with the colour previously
drawn (which gets weight \(1-\alpha\).)
This note is concerned with what happens when two transparent objects are drawn at the same location. We suppose the further one has transparency \(\alpha_1\), and the closer one has transparency \(\alpha_2\). If they are drawn in the correct order (further first), the three colours (background, further object, closer object) should end up mixed in proportions \(C = [(1-\alpha_1)(1-\alpha_2), \alpha_1(1-\alpha_2), \alpha_2]\) respectively.
If the objects are drawn in the wrong order, the actual proportions of each colour will be \(N = [(1-\alpha_1)(1-\alpha_2),\alpha_1, \alpha_2(1-\alpha_1)]\) if no masking occurs.
rgl currently defaults to depth masking using glDepthMask(GL_TRUE). This means that depths are saved
when the objects are drawn, and if an attempt is made to
draw the further object after the closer one (i.e. as here),
the further object will be culled, and the proportions will be
\(M = [(1-\alpha_2), 0, \alpha_2]\).
The question is: which is better, glDepthMask(GL_TRUE) or
glDepthMask(GL_FALSE)? One way to measure this is to
measure the distance between \(C\) and the incorrect proportions.
(This is unlikely to match perceptual distance, which will
depend on the colours as well, but we need something. Some
qualitative comments below.)
So we have
$$|C-N|^2 = 2\alpha_1^2\alpha_2^2,$$
and
$$|C-M|^2 = 2\alpha_1^2(1-\alpha_2)^2.$$
Thus the error is larger with \(N\) when \(\alpha_2 > 1 / 2\), and larger with \(M\) when \(\alpha_2 < 1 / 2\). The value of \(\alpha_1\) doesn’t affect the preference, though small values of \(\alpha_1\) will be associated with smaller errors.
Depending on the colours of the background and the two objects, this recommendation could be modified. For example, if the two objects are the same colour (or very close), it doesn’t really matter how the 2nd and 3rd proportions are divided up, and \(N\) will be best because it gets the background proportion exactly right.
Typically in rgl we don’t know which object will be closer
and which one will be further, so we can’t base our choice on
a single \(\alpha_i\). The recommendation would be to use
all small levels of alpha and disable masking, or use all
large values of alpha and retain masking.
The classic example of an impossible to sort scene involves three triangles arranged cyclicly so each one is behind one and in front of one of the others (based on https://paroj.github.io/gltut/Positioning/Tut05%20Overlap%20and%20Depth%20Buffering.html).
theta <- 2*pi*c(0:2, 4:6, 8:10)/12
x <- cos(theta)
y <- sin(theta)
z <- rep(c(0,0,1), 3)
xyz <- cbind(x, y, z)
xyz <- xyz[c(1,2,6, 4,5,9, 7,8,3),]
open3d()
## null
## 72
par3d(userMatrix = M)
triangles3d(xyz, col = rep(c("red", "green", "blue"), each = 3))
plot of chunk unnamed-chunk-1
To see the effect of the calculations above, consider the following four displays.
open3d()
## null
## 74
par3d(userMatrix = M)
layout3d(matrix(1:9, ncol = 3, byrow=TRUE),
widths = c(1,2,2), heights = c(1, 3,3),
sharedMouse = TRUE)
text3d(0,0,0, " ")
next3d()
text3d(0,0,0, "depth_mask = TRUE")
next3d()
text3d(0,0,0, "depth_mask = FALSE")
next3d()
text3d(0,0,0, "alpha = 0.7")
next3d()
triangles3d(xyz, col = rep(c("red", "green", "blue"), each = 3), alpha = 0.7, depth_mask = TRUE)
next3d()
triangles3d(xyz, col = rep(c("red", "green", "blue"), each = 3), alpha = 0.7, depth_mask = FALSE)
next3d()
text3d(0,0,0, "alpha = 0.3")
next3d()
triangles3d(xyz, col = rep(c("red", "green", "blue"), each = 3), alpha = 0.3, depth_mask = TRUE)
next3d()
triangles3d(xyz, col = rep(c("red", "green", "blue"), each = 3), alpha = 0.3, depth_mask = FALSE)
plot of chunk unnamed-chunk-2
As you rotate the figures, you can see imperfections in rendering. On the right, the last drawn appears to be on top, while on the left, the first drawn appears more opaque than it should.
In the figure below, the three triangles each have different transparency, and each use the recommended setting:
open3d()
## null
## 76
par3d(userMatrix = M)
triangles3d(xyz[1:3,], col = "red", alpha = 0.3, depth_mask = FALSE)
triangles3d(xyz[4:6,], col = "green", alpha = 0.7, depth_mask = TRUE)
triangles3d(xyz[7:9,], col = "blue", depth_mask = TRUE)
plot of chunk unnamed-chunk-3
In this figure, all three triangles are the same colour, only lighting affects the display:
open3d()
## null
## 78
par3d(userMatrix = M)
layout3d(matrix(1:9, ncol = 3, byrow=TRUE),
widths = c(1,2,2), heights = c(1, 3,3),
sharedMouse = TRUE)
text3d(0,0,0, " ")
next3d()
text3d(0,0,0, "depth_mask = TRUE")
next3d()
text3d(0,0,0, "depth_mask = FALSE")
next3d()
text3d(0,0,0, "alpha = 0.7")
next3d()
triangles3d(xyz, col = "red", alpha = 0.7, depth_mask = TRUE)
next3d()
triangles3d(xyz, col = "red", alpha = 0.7, depth_mask = FALSE)
next3d()
text3d(0,0,0, "alpha = 0.3")
next3d()
triangles3d(xyz, col = "red", alpha = 0.3, depth_mask = TRUE)
next3d()
triangles3d(xyz, col = "red", alpha = 0.3, depth_mask = FALSE)
plot of chunk unnamed-chunk-4
Here depth_mask = FALSE seems to be the right choice in both cases.