Why shouldn’t the Eye enjoy the pleasure of Music the way the Ear has it?
We like watching changing colors:
· the sky at sunset,
· a field of flowers under the wind,
· dancers in colorful clothes on the stage,
· flowing colorful dresses on the streets.
We should be able to create similar beauty of the Moving Color (Music/color) on a screen and we should be able to combine it with the beauty of Moving Sound (Music/sound).
A Composer could create a Music/color part and a Music/sound part, with the help of computers, and produce a movie – Music of color and sound.
Alternatively, a Composer/color and a Composer/sound could collaborate with help of computers and create the Music/color part and the Music/sound part and merge them into Music of color and sound.
The basis of Music is Harmony. Practically everyone experience strong pleasant feelings from Harmony and this is a basis of perception of beauty of the Music - Music/sound and Music/color.
Harmony of sound could be described in terms of its decomposition into a “sum” of sounds each one with only one frequency of vibration. When these frequencies satisfy some well-known relationships based on whole numbers only, we perceive it as harmonious. One could only guess why it is, maybe because this is how a solid “object” vibrates. It could be a piece of metal or it could be a skillfully controlled orchestra. Such sound gives hope that there is something unifying behind it.
We perceive a part of colorful surface as harmonious, when we add up all light emitted/reflected from it and we get white light. We are satisfied, even when we just take some representative sampling of the surface and get white color as a sum of these samples.
As with sound, it is hard to say why it is. Maybe this gives us hope that entire scenery is illuminated by the white light, as light of the sun, and we are not getting confused by the tint of illuminating light in our analysis of the surface.
Note that the mechanism of harmony determination is much more forgiving in case of colors than in case of sounds. Analyzed colors could change somewhat during the analysis, the sum of light collected from the sampling does not need to be precisely white, it could be almost white, and still combination of colors could be judged as harmonious.
From the other hand, the color cannot change too fast during harmony determination – time is needed to detect harmony.
Movie, which Composers produce, should be displayed on a rectangle of the screen – a screen of a computer, a screen of a theater, or a screen used for advertisement.
Boundary of the screen causes some points (inside the boundary, on it, or outside it) to be different from others, to be in harmony with the screen. Painters and cameramen know this effect.
These points are determined as nodes of one of special grids.
These grids are constructed as follows. Top and bottom of the screen are divided onto N equal parts and vertical lines are drawn through division points. Similarly, left and right sides of the screen are divided into M equal parts and horizontal lines are drawn through division points. This grid is naturally extended beyond the bounds of the screen.
The nodes of the grid, where lines intersect, are special places, to which various “images” should be “anchored” to be perceived as placed harmoniously.
However, for such spatial Harmony to be felt by a spectator, only relatively small numbers N and M could be used. This limitation is similar to Harmony of sounds.
There could be variety of “color-images” and ways to “anchor” them to these special spots, and this should provide a lot of room for application of Composers’ ingenuity.
Eventually, Composers will experiment with different types of screen – not flat, not rectangular, etc., and then other forms of spatial Harmony will be used.
We learnt from the experience of making beautiful colorful clothes, that the Eye (entire system processing visual information) traverses the Line and checks if combination of colors in both sides of it is harmonious. This happens with a static line and with a line, which moves (slowly). Again, painters and cameramen know this well and use it.
Only a few types of sound (“sound-images”) are suitable to be a part of Music/sound. Composers/sound still experiment with introduction of new “sound-images” to create new effects, and this is not an easy task.
We should expect similar situation with “color-images”. Only a few types of distribution of color on the screen should be suitable for Music/color, but we should expect consistent expansion of the set of such “color-images” by efforts of Composers.
It is important for a “color-image” to be “anchored” on described above nodes, selected for a piece of Music.
Hence, the “color-image” has to have a special place “anchor” somewhere in it, which:
· has easily identified characteristics,
· is large enough for an Eye to find it quickly.
It could be a unique spot of the “color-image”, where intensity of the light is the highest.
On the screen, “color images” overlap, the way sounds of music overlap. It is important for “color-images” to “work” like “sound images”. “Color-images” should have this “dual nature” – they should be perceived as parts of a whole and in the same time each “color-image” should be possible to identify as a separate entity.
Hence, it is important that different “color-images” on the screen do not interfere much with perception of other “color-images” on the screen.
This could be achieved with quick diminishing of the intensity of the light as one scans the “color-image” from the “anchor” outward. This part of the “color-image” is the “skirt” of the “color-image”. The diminishing could be gradual, as one moves the gaze from the “anchor”, or could have some minor variation of intensity of the light or even variations of the color.
Hence, these are two basic parts of the “color-image”: “anchor” – a small bright area of the “color-image” placed on the node of the chosen grid, and a “skirt” - an area, where brightness diminishes, as the gaze moves away from the “anchor”.
Different variants of “skirts” are possible.
For example, in some “skirts” brightness of the light could diminish fast to nothing, as the gaze moves outward from “anchor”. Some could be monochromes, while other could have variation of the color.
It would be nice to place additional restriction on “skirts” with variation of color: colors of the skirts on any circle with center in the “anchor” should be harmonious (add up to white).
In a “regular skirt”, intensity of the light diminishes to nothing at the border of the skirt (points farthest from the “anchor”). “Regular skirts” do not produce “hard borders”, they produce a smooth picture.
Obviously, “anchors” on the boundary of the screen produce special type of “color-images”.
The other group of special “color-images” has anchors outside the screen, their “skirts” should hint where these anchors are. The Eye should be able to determine, where the anchor is, as it determines, where the Sun is through analysis of the colors of sunset, after the Sun sets.
It is expected that color of a “color-image”, including the “leading color” – the color in “anchor”, changes in time.
For this change to be perceived as harmonious, it should be slow enough, to allow the Eye to check the scene for harmony, and the set of colors in all “anchors” should be harmonious in all times during performance of one semantic part of the Music.
This should not be an absolute rule – Composers should be free to use not harmonious combinations, but that should be done on purpose, not accidently.
It is natural to attempt combining of “color-images” on the screen just by adding-up light at each point of the screen, similar to how it is done with sounds of music - sounds are just added up.
Unfortunately this could lead to unpleasant spikes of light in places of the screen, where we do not want to have them. Hence, we need to deploy special technique of combining of “color-images”.
First of all, we should think of each “color-image” as occupying entire screen, the way each sound occupies entire space. Because of the presence of a “color-image”, in some places of the screen amount of light emitted by the screen is large than in others.
We will define combining of “color-images” as adding up light, caused by different “color-images” to be emitted by the screen. This is similar to the way we work with “sound-images”.
However, we will add some correction to that straight-forward “color-images” combining. We will slightly modify “color-images”, when we combine them. To avoid confusion, we will use only “color-images”, which are easily recognizable after such modification. Following describes this minor modification.
We do not want a “skirt” of one image to visibly affect the color of “anchor” of another “color-image” and even colors of the skirt near the “anchor” of another “color-image”. The reason for that is obvious – we want to preserve spatial harmony.
We also want to avoid another artifact. The nature of the “skirt” is diminishing light intensity, when one moves the gaze from the “anchor” outward, and we do not want this property to change, when we add-app lights of “skirts” of two different “color-images”. “Skirts” should be recognizable as “skirts” (subordinate to anchors), even, when they are combined.
One of techniques, which could be deployed to “protect” recognition of the “anchor” and “anchor”-“skirt” relationship of a given “color-image” after “color-images” are combined, is as follows.
In each point of the screen, we combine a few “color-images” by adding-up light corresponding to each of them. However, we add-up the light (color-vector) with weight coefficients. The value of these coefficients depends on the distance to the “anchor point” of the corresponding “color-image”. When that distance is small, the coefficient should be close to 1.0, but the larger the distance the smaller should be coefficient. The sum of these coefficients should be equal to 1.0.
This way, we assure easy recognition of “anchors” and their corresponding “skirts”. “Color-images” will be recognized as separate entities even after mixing of light corresponding to them.
The Line and especially a Moving Line adds additional level of complexity. It could dominate, and then colors would be checked for harmony only along it and across it. It could create two separate areas of perception, etc. This should be good fun for Composers to work with, especially, when a few Lines are deployed.
It is hard to expect development of special instruments producing “color-images”, hence from the very beginning this art-form would require development special computer Languages, which could be used to describe “color-images” and “sound-images” and their dynamic.
This description would be passed to the compiler, which would generate some, hidden from the user, “software objects” and rules. These internal elements of software would be used by the compiler to generate entire presentation of color and sound. It will create a digitized movie with moving picture and with sound.
That movie would be possible to copy and distribute.
Alexander Liss 10/28/2019