Following is based on “Beauty of Moving Color” and “Musical Notation for Computers”, which one can find on the same site, next to this presentation.
As in case with sound, we rely on perception of harmony, only this time harmony of colors. Detection of spatial harmony relies on perpetual scanning of the scene by the Eye and (mental) mixing of observed colors. Detection of temporal harmony relies on memory.
In Music/color, we use special kind of dynamic color distribution on the screen - color-image. Colors of different “color-images” are mixed (color-vectors are added-up) at every pixel of the screen.
We construct color-images around a chosen grid on the screen, with equal horizontal steps and equal vertical steps. Each color-image is tied to a node of the grid – it is it’s “anchor”. The color-image has high intensity color at the “anchor”. From such an “anchor”, color fades in all directions – this is it’s “skirt”.
While the “anchor” of a color-image provides main impression of its color, “skirts” of color-images support dazzling interplay of colors.
All three color components of the “skirt” should fade at the same degree, when an Eye moves away from the “anchor”.
We want fading to be uniform in all directions from the “anchor”. Hence, we should define a Fading Function with the argument, which is the vector from the “anchor” to the pixel, where we should compute degree of fading.
Components of this vector:
x – the distance between projections of the pixel and the “anchor” on the bottom part of the screen,
y - the distance between projections of the pixel and the “anchor” on the left side of the screen.
The value of the this function
should be positive and it should gradually diminish, as either x or y grow. When x=y=0 it should be equal to 1.
We have a special requirement for the Fading Function. We want color fading to look similar on screens of different sizes and different ratios of screen’s height to screen’s width. Hence parameters of the screen:
H - screen’s height
W - screen’s width
should be parameters of our Fading Function.
d = x/W + y/H
FF(x,y) = F(d) = F(x/W + y/H)
F(z), z>0, is one dimensional function decreasing monotonously from the value 1 to the value 0, as z growth.
On the way, we got additional benefit: the shape of color-image reflects the shape of the screen and this looks proportional.
We impose restrictions on the function F(z):
a) when z is small, i.e. the point on the screen is close to “anchor”, speed of change of F(z) should be small – at “anchor” color should be strong,
b) when z grows, i.e. the point on the screen is far from the “anchor” and we are moving away from it, F(z) should diminish fast - colors of other color-images should be seen.
We should maintain a limited fixed set of different functions F(z), to create different types of color-images with different fading from “anchor” to the border. Only a fading pattern from this set could be used in creation of color-images. This restriction supports instant recognition of color-images.
In Music/color we distinguish following temporal patterns:
· color-image, which is not changing in time for awhile; it is similar to a musical tone, and we call it color-tone;
· a set of color-images, which colors are harmonious and not changing in time for awhile, is similar to a musical accord, and we call it color-accord;
· changing in time color in the process of rump-up from black to a color-tone or fade-out from a color-tone to black;
· coordinated changing in time color in the process of rump-up from black to a color-accord or fade-out from a color-accord to black;
· changing in time color from one color-tone, to another color-tone with its anchor in the same place on the screen;
· coordinated changing in time color from one color-accord, to another color-accord.
When color-images form an accord, colors at their “anchors” form a harmonious set, i.e. added up (mixed) they produce white color. Hence, we need to provide unobstructed observation of color at “anchors” of color-images of an accord. For that we modify functions FF(x,y) for each color-image of the accord, that they yield (diminish in value) in vicinity of an “anchor” of every other member of the accord.
When we combine sound and color, we generate a movie file, which guides the coordinated harmonious show of sound and color produced with sound equipment and color screen. It is guided by our understanding of the rhythm, phrases, and engagement of our senses fully, but without overwhelming them.
There is experience of making shows, were different senses are engaged in balance. It should be easy to apply it to this new type of show.
Since we need to display colors on existing color screens, which have only three sub-pixels per pixel, we need to specify colors with vectors of three components (ideally, we need four sub-pixels). We need also a 3x3 transformation matrix, which allows transformation between our color-vector and one, which is traditionally used to represent colors on the screen and for which there is mapping to hardware. This 3x3 matrix provides needed calibration of the screen that harmonious combinations of colors, which we compute so carefully, are seen as harmonious on the screen.
Adjusting this 3x3 matrix to a given screen is like tuning musical instruments before performance.
We could use more precise presentation of color with 4 dimensional vector and 4x3 matrix to transform our color presentation into one fed to the screen, but that would only increase amount of computations without providing improvement to the show. We will wait until color screens have four sub-pixels, and then we will switch to more precise system.
Alexander Liss 12/30/2019