contact
MOMus
Museum of Modern Art
Costakis Collection
21 st Kolokotroni Str.
Moni Lazariston
56430, Stavroupoli
Greece
| GRAPHICAL SOUND | ||
Graphical (drawn) Sound is a technology for synthesizing sound from light that was invented in Soviet Russia in 1929 as a consequence of the newly invented sound-on-film technology which made possible access to the sound as a trace in a form that could be studied and manipulated. It also opened up the way for a systematic analysis of these traces such that they could be used to produce any sound at will.
By 1936 there were several main, relatively comparable trends of Graphical Sound in Russia: At exactly the same time very similar efforts were being undertaken in Germany by Rudolf Pfenninger in Munich and, somewhat later, by the animator and filmmaker Oskar Fischinger in Berlin. Among the researchers working with Graphical Sound after World War II were the famous filmmaker Norman McLaren (Canada) and the composer and inventor Daphne Oram (UK). Ornamental Sound The ornamental sound technique, developed in 1929-1930 by Arseny Avraamov, was similar to German animator and filmmaker Oscar Fischinger's sounding ornaments first presented in 1932. In 1930, however, Avraamov was the first to demonstrate experimental sound pieces – based In December 1930 Mikhail Tsekhanovsky wrote in his article About the Drawn Sound Film: "with the invention of new drawn sound techniques (developed by Arseny Avraamov in Moscow, Sholpo and [Georgy] Rimsky-Korsakov in Leningrad) we are achieving a real possibility of gaining a new level of perfection: both sound and the visual canvas will be developing completely in parallel from the first to the last frame […] Thus the drawn sound film is a new artistic trend in which for the first time in our history music and art meet each other." [2] In autumn 1930 Avraamov founded the Multzvuk group at Mosfilm Productions Company in Moscow. To produce his first drawn ornamental sound tracks he had on staff a special draughtsman, cameraman Nikolai Zhelynsky, animator Nikolai Voinov and acoustician Boris Yankovsky who was responsible for the translation of musical scores into Avraamov's microtonal Welttonsystem as well as Samoilov's Ober-Unter-Tone Harmony system. The final scores were coded in Yankovsky's 72- step ultrachromatic scale with the dynamics and speed variations indicated by the number of frames. Yankovsky was also involved in the production of From 1930-34 more than 2000 meters of sound track were produced by Avraamov's Multzvuk group, including the experimental films Ornamental Animation, Marusia Otravilas, Chinese Tune, Organ Cords, Untertonikum, Prelude, Piruet, Staccato Studies, Dancing Etude and Flute Study. In autumn 1931 the Multzvuk group moved to NIKFI (Scientific Research Institute for Cinema and Photography) and was renamed Syntonfilm Laboratory. In December 1932 NIKFI stopped supporting Syntonfilm and the group moved to Mezhrabpomfilm where in 1934 it was closed as it was unable to justify itself economically. The whole archive was kept for many years at Avraamov's apartment, where in 1936-37, during Avraamov's trip to the Caucasus, it was burned by his own sons, making rockets and smoke screens with the old nitro-film tapes, which were highly flammable.
Because of the cross-disciplinary nature of the new technique, people involved in it had to be skilled not only in music, but in acoustics, mathematics, sound-on-film technology and engineering. As a result even skilled journalists often could not understand the physical meaning of the phenomena under consideration or specific technological ideas. Having no developed terminology, many mistakes and unexpected “puzzles” appeared in their writings. Moreover, there were several known research groups – competitors in Russia and Germany working in parallel. It led to a very specific problem – encryption of the information. For instance, in the well-known photograph Oscar Fischinger holds ‘fake' rolls made by his Studio for publicity purposes as he did not want his competitors to learn his actual techniques. He never used rolls as large as this – they were fakes. [3] Yankovsky had a very special way of making notes on his ideas. It is impossible to understand the construction of his tools from reading one description without referring to several other manuscripts that offer important keys for understanding it. Paper Sound As of 1931 Voinov was involved in the activities of the IVVOS group (Ivanov, Voinov, Sazonov). This group produced a number of animated cartoons with synthetic sound tracks, including Barynia (1931), Rachmaninov's Prelude (1931), The Dance of the Crow (1933), About a Pig's Snout and a Soviet Kitchen Garden (1933),The Thief (1934) and Zones, Safety Lines (1934). In early 1936 Voinov was dismissed from the Moscow Film Factory and his laboratory was closed. For the rest of his life he worked successfully as a cameraman at Souzmultfilm Studio. In Voinov's official biography his most experimental work from 1931-36 is not even mentioned. The Variophone
The first version of the instrument was built with assistance from the composer Georgy Rimsky-Korsakov (grandson of the famous composer By the early 1931 at Lenfilm Studio, with assistance from G. Rimsky-Korsakov, Sholpo made a soundtrack for the short propaganda film The year 1905 in Bourgeois Satire (director N.I. Galkin, composer V.M. Deshevov). In the summer of 1932 Sholpo and Rimsky-Korsakov produced the synthesized soundtrack for the new color cartoon The Symphony of Peace by E.J. Johansson and G.V. Bankovsky. Many soundtracks for movies and cartoons were produced using the Variophone. Among the most accomplished pieces recorded with the Variophone in 1933-34 were The Carburettor Suite by G. Rimsky-Korsakov, Waltz by N. Timofeev, Flight of the Valkyries by Richard Wagner, and Franz Liszt's 6th Rhapsody.
Syntones and Audio Computing
The method developed by Boris Yankovsky was based on pure audio computing techniques and possessed properties, very common for digital technologies, such as discretization and In 1933 Yankovsky was invited to Mosfilm Studios to organize the Laboratory for Synthetic Sound Recording, where in 1934-35 he recorded a sizeable collection of samples of instruments from the Symphony Orchestra of the Bolshoi Theatre. By 1936 the collection of 110 synthesized templates was created. In 1935 Yankovsky joined the Autonomous Research Section (ANTES) at the Union of Composers in Moscow, founded by commissar Boris Krasin, Avraamov and Ogolevets. It was the last significant manifestation of creativity with its roots in the forward-looking In 1939 Yankovsky met Evgeny Murzin – a young inventor, fascinated by the idea of a universal tool for sound synthesis. The same year Boris Yankovsky and Evgeny Sholpo decided to unite their efforts and the new Laboratory for Graphical Sound at the Institute of the Theatre and Film was established. Yankovsky moved to Leningrad. He expected to complete the final version of his vibroexponator in 1940 but finally was prevented by World War II and never returned to the graphical sound. The ANS Synthesizer
In 1967 the studio of electronic music was established in Moscow, with the ANS synthesizer at its core. Among the composers working with the ANS were Alfred Shnitke, Sofia Gubajdulina, Edison Denisov, Eduard Artemyev, Alexander Nemtin and Stanislav Krejchi. The instrument was used for scoring many films, in particular the early films of Andrei Tarkovsky. Although there were several short articles related to Graphical Sound published in German, French and English [4-6] most publications about research and developments in the USSR were only in Russian. At the same time most important documents were never published at all and were circulating only in manuscript form, similar to ‘Samizdat' (self-published forbidden literature).
REFERENCES [1] Avraamov, Ars. “Sinteticheskaya muzika” Sovetskaya Muzika , 1939, No.8, pp. 67-75 [2] Tsekhanovsky M. 1930. “O Zvukovoi Risovannoi Filme.” Kino I Zhizn, Moscow. 1930, no. 34-35, p.14 [3] According to information received from the Fischinger Trust, the Center for Visual Music [4] Von A. Leo. “Die Grosse Erfindung – ‘Graphomusik’”, Moskauer Rundschau, 15.03.1931 [5] Le “Variophone”, Le Journal de Moscou, 01.06.1935 [6] Sîlåv, V. “Absolute Music by Designed Sound.” American Cinematographer. April, 1936, pp. 146-148, 154-155 [7] Yankovsky, B. “Analiz i sintez tembra” (Analysis and Synthesis of Timbre) March, 1935, Moscow. [16] Smirnov A., Pchelkina L., Russian Pioneers of Sound Art in the 1920s. Catalogue of the exhibition 'Red Cavalry: Creation and Power in Soviet Russia between 1917 and 1945'. La Casa Encendida, Madrid, 2011. [18] Smirnov A., The book “SOUND in Z. Forgotten experiments in sound art and electronic music in early 20th century Russia”. Sound & Music, London / Verlag de Buchhandlung Walther Konig, Cologne, 2013. [19] Smirnov A., 'Fruhe Versuche der Sprach- und Gesangssynthese im postrevolutionaren Russland der 1920er und 30er Jahre.' Collection "Resonanz-Raume: Die Stimme und die Medien". Ed. Oksana Bulgakowa, Bertz+Fischer, Berlin, 2012. (German). Acknowledgments I would like to thank Lev Bolotsky and Marina Sholpo for unique documents which they rescued from destruction and kept for so many years. Special thanks to Nikolai Izvolov for the help and advise, Matthew Price and Rob Mullender for their help in preparation of texts. Andrey Smirnov, Moscow, 2011
|
||
The first practical sound-on-film systems were created almost simultaneously in the USSR, USA and Germany. In Soviet Russia Pavel Tager initiated developments in 1926 in Moscow. Just a few months later in 1927, Alexander Shorin started his research in Leningrad. Tager's system, the Tagephon, was based on intensive variable density optical recording on film while in Shorin's Kinap system the method of transversal variable area optical recording on film was realized. 
Among the first Soviet sound movies ever created was "The 5-year Plan" (Piatiletka. The Plan of Great Works) by Abram Room. The group working on this film in 1929 at Shorin’s Central Laboratory of Wire Communication in Leningrad included the painter, book illustrator and animator Mikhail Tsekhanovsky, the chief of the composer’s brigade Arseny Avraamov and the inventor Evgeny Sholpo. When in October of that year the first roll of film was developed, it was Tsekhanovsky who voiced the idea: “What if we take some Egyptian or ancient Greek ornaments as a sound track? Perhaps we will hear some unknown archaic music?” [1] He was referring to the shapes and outlines of vases and how these could be used like wave forms to generate sound. It was at this precise moment that graphical sound techniques were invented. The next day they were already furiously at work on experiments in what they referred to variously as ornamental, drawn, paper, graphical, artificial or synthetic sound. The laboratories that were soon created became the first-ever prototypes of the future centres for computer music. While most inventors of electronic musical instruments were developing tools for performers, the majority of methods and instruments based on Graphical Sound techniques were created for composers. Similar to modern computer music techniques, the composer could produce the final synthesized soundtrack without need for any performers or intermediates. 
on geometric profiles and ornaments – produced purely through drawing methods. This was achieved by means of shooting still images of drawn sound waves on an animation stand. 
acoustic experimental studies, developing methods for the synthesis of sounds with glissando, timbre crossfades, timbre variations and polyphony by means of multiple shooting on the same optical soundtrack (alternative to multi-track recording which was not available yet). 
In 1931 Nikolai Voinov left Multzvuk group and started his own research as a developer of paper sound techniques. These were based on the synthesis of sound waves by means of paper cutouts with the carefully calculated sizes and shapes produced by his newly invented tool, the Nivotone. Voinov's method offers a surprisingly efficient level of control over the dynamics of sound. 
The Variophone was invented by Evgeny Sholpo in 1930 at Alexander Shorin's Central Laboratory of Wire Communication in Leningrad. In May 1930 Sholpo applied for a patent on a “method and device for the production of a periodic sound track on film.” It was a continuation of research that Sholpo had been conducting since the 1910s when he was working on performer-less music. 
Nikolai Rimsky-Korsakov) in 1931 at Lenfilm Studios. It was capable of producing artificial soundtracks by means of Automated Paper Sound techniques. Sholpo’s method made access easier to varieties of timbres. Unlike Avraamov, who shot still images of sounds on an animation stand, Sholpo used cardboard disks with circular images of combs with suitably shaped cogs rotating synchronously with a moving filmstrip. The advantages of the Variophone were in its flexible and continuous pitch control and vibrato. 
Although the very first version of the Variophone was made with wooden parts fixed by wires and tuned with ropes, it already incorporated one of the most crucial and necessary devices – a mechanism for the precise and continuous changing of the speed of rotation of the optical disk with the sound wave pattern i.e. a means of controlling the pitch with the possibility of synthesizing continuous glissandos. Also from the very beginning the composer had full freedom to work with polyrhythmic combinations and almost unlimited tempos in any passages. 
The Variophone was continuously under development and by 1936 the arsenal of musical and acoustical means of the second version was enriched highly with possibilities of free glissando with a speed of up to four octaves per second, flexible and exact control over dynamics, and options for deep vibrato for pitch, volume and timbre. The Variophone could produce polyphonic soundtracks with up to twelve parallel voices. Even compared with the subsequent and more advanced third and fourth versions, it produced the most impressive quality and complexity of sound. In 1941 during the blockade of Leningrad, together with composer Igor Boldirev, Sholpo synthesized one of his most experimental pieces – the soundtrack for the cartoon The Vultures. Although aesthetically these works are similar to Walter Carlos' Switched-on Bach (1968) and sound like eight-bit music, the main difference is in their timing. In 1918 Sholpo developed special tools - the Melograph and Autopianograph - to register the temporal characteristics of live musical performance. Much electronic music has a rigid tempo, like a metronome; Sholpo was able to simulate more subtle variations in tempo such as rubato, rallentando and accelerando, based on his careful analyses of live piano performances by the best pianists. 
In 1931-32 Boris Yankovsky (1904-1973) was on the staff of the Multzvuk group. In 1932, however, disappointed with its Ornamental Sound approach, he left the group. Unlike most of his colleagues he understood that the waveform does not represent the tone colour uniformly and that only the spectrum of sound developed in time with all the nuances of its temporal transitions can give a complete picture. Of all the early graphical sound pioneers, Yankovsky alone pursued the approach of spectral analysis, decomposition and re-synthesis. His concept was based on the belief that it is possible to develop a universal library of sounds similar to Mendeleev's table of chemical elements. His curves were spectral templates, semiotic entities that could be combined to produce sound hybrids. As an option he developed several sound processing techniques including pitch shifting and time stretching based on the separation of spectral content and formants, resembling recent computer music techniques of cross synthesis and the phase vocoder. To realize these ideas he invented a special instrument, the Vibroexponator – the most paradigm-shifting proposition of the mid-1930s. 
In 1935 in one of his manuscripts Yankovsky wrote: ‘It is important now to conquer and increase the smoothness of tone colours, flowing rainbows of spectral colours in sound, instead of monotonous colouring of stationary sounding fixed geometric figures [wave shapes], although the nature of these phenomena is not yet clear. The premises leading to the expansion of these phenomena – life inside the sound spectrum – give us the nature of the musical instruments themselves, but “nature is the best mentor” (Leonardo da Vinci) […] The new technology is moving towards the trends of musical renovation, helping us to define new ways for the Art of Music. This new technology is able to help liberate us from the cacophony of the well-tempered scale and related noises. Its name is Electro-Acoustics and it is the basis for Electro-Music and Graphical Sound’. [7]
Yankovsky's approach had much in common with that of Rudolf Pfenninger. Both were primarily focused on acoustics. As Thomas Levin put it: ‘ Fischinger's curves are not derived from sound, they generate it, whereas Pfenninger's curves are in the last analysis derived from the sounds that they analytically recreate… Pfenninger's curves are decidedly not ornaments but are rather, as numerous critics have rightly noted, “templates or print-types”, that is, semiotic entities that can be combined to produce sounds in a linguistic – which is to say, thoroughly technical and rule-governed – manner. Unlike Fischinger's curves, which were continuous, Pfenninger's were discrete units'. [8] 
quantization of audio signals and related spectral data, manipulation with ready-made parts, and operations with selections from databases of the basic primitives (templates).
To perform complex mathematical calculations of waveforms as well as other important parameters of sound and automated musical performance such as rhythm, there were special ‘employee-computers’ on the staff in the laboratories of Boris Yankovsky and Evgeny Sholpo. These were mathematicians whose specific task it was to make calculations.
1920s. In 1936 the infamous Pravda article Confusion instead of music was published, initiating a war by the totalitarian state against the freedom of artistic expression. After the death of Krasin in 1936 ANTES was closed and the Ministry of Culture stopped funding Yankovsky's laboratory. It was passed to the NIMI institute at Moscow Conservatory. Little changed, but by the end of 1937 he finally got his syntones to make sounds. 
In 1957 the young inventor Evgeny Murzin (1914-1970) finished developing and patented a photo-electronic musical instrument called the ANS Synthesizer. Its name was derived from the initials of influential composer Alexander Nikolayevich Scriabin. It was remarkably close to the concept of Evgeny Sholpo's Mechanical Orchestra. 
The instrument was based on a set of sine wave oscillators, produced by pure optical means as multiple (144) graphical soundtracks printed on four glass disks, adjusted on fixed frequencies, forming a discrete pitch scale, covering the whole audible range with intervals between successive pitches imperceptible to the human ear. Control over the system and the process of sound synthesis was carried out by means of a special graphical score with a diagram representing the spectrum of a sound by means of drawn transparent strips, having appropriate shapse and slopes, making it possible to operate the whole set of sine wave tones synchronously and independently, controlling the sound on a spectral level, directly manipulating the overtones, erasing the difference between the pitch harmony structures and a spectral tissue of a sound. A similar principle of the graphical score was used later in the legendary UPIC computer system, developed by Yanis Xenakis in 1977 in the Centre d'Etudes de Mathematiques et Automatiques Musicales in Paris. 
The instrument was based on the scale of 72 steps per octave proposed by Boris Yankovsky, who was involved in the development of the ANS in 1939-40. The synthesizer incorporates a set of rotating optical disks with photo-printed round optical sound tracks. While in the variophone one rotating disk produced a single sound, in the ANS each optical disk contained 144 independent sound tracks. Four disks, used in the first version of the instrument, could produce simultaneously 576 sine waves with frequencies covering the whole audible range. ANS was a real-time instrument, producing the audible result directly during work. 