뒤로가기back

Dissonance, Mahler, and Beyond - Part 2

2024.11.21 ・ by Ste Park

Hello! I’m Ste, and I’m researching voice AI at Gaudio Lab.

 

In the previous post, I talked about how Mahler’s dissonance was the language he used to express his emotions.
This time, I’d like to explore the traces and significance that this language left in the history of music.
Shall we dive deeper into Mahler’s music now? 🎶

 

 

2 Mahler

 

One of the composers who tackled this challenge was Mahler. In his symphonies, Mahler elevated dissonance to an artistic pinnacle. He once said, “A symphony must be like the world. It must embrace everything.” His music encapsulates the complexity of human emotions, the order and chaos of the universe, and the entirety of life and death. To Mahler, dissonance wasn’t merely a clash of notes; it was an essential tool that freely crossed the boundaries between harmony and tension, simultaneously expressing contradictory emotions.

 

In his symphonies, Mahler fused the diversity of the world into a cohesive whole, leading audiences toward new philosophical reflections. For instance, in Symphony No. 1, he sings of the vitality of spring while reflecting on his painful childhood, juxtaposing consonance and dissonance, harmony and chaos. In Symphony No. 2, he explores the journey from death to resurrection, expressing the weight of life and the possibilities beyond it.

 

Symphony No. 3 delves into the layers of existence, showing the harmony of nature, humanity, and love. The Adagietto of Symphony No. 5 is outwardly beautiful but imbued with the sorrow and anxiety of love, demonstrating how music can embody love, loss, pain, and joy simultaneously.

 

Mahler’s final symphony, Symphony No. 10, represents the zenith of his artistic exploration and showcases the climax of dissonance. Although unfinished, Mahler used this work to embrace pain and despair, transforming human wounds and suffering into an artistic universe through dissonance.

 

 

2.1 A Love Letter in the Key of A: The Fourth Movement of Symphony No. 5

 

The fourth movement of Mahler’s Symphony No. 5 is famously dedicated to his wife, Alma Mahler. Its distinctively beautiful melody is slow yet far from monotonous, moving listeners with its profound emotions. It has gained renewed attention recently, being featured in director Park Chan-wook’s film Decision to Leave, dramatically portraying the protagonist’s existential struggles between life and death.

 

Set in F Major, the movement is slow and tranquil, and the persistent use of non-chord tones evokes a reminiscent of Schumann’s piano piece Träumerei. Both share the same key, similar opening note structures, and the prominent use of a high A in their climactic sections. In Träumerei, this high A is sustained in the climax, initially harmonized with an A Major chord and later with a G Major 9th chord. The latter intensifies emotional tension as the A becomes the ninth, evoking poignant sentiments.

 

 

말러 교향곡 5번 4악장(좌)와 10번 1악장(우)에서 발췌

Figure 4: Excerpt from Mahler’s Symphony No. 5, 4th Movement (left) and Symphony No. 10, 1st Movement (right)

 

 

Mahler adopts a similar structure. The same high A note is harmonized with an F Major chord in the first section, while in the second section, as shown in Figure 4, it is embellished with a B♭mM7 chord, a Bø7 chord, and then resolves to an F Major chord. While transitioning through two chords to reach F Major, two non-chord tones are employed: G♯ moving toward A and C♯ resolving to D, played by the second violin.

 

 

2.2 “My Wife Has a Boyfriend”

 

Though Mahler and Alma were married, their love did not last. The reasons for their estrangement, as is often the case, remain known only to them—and sometimes not even to themselves. Alma, who had aspired to be a composer in her youth, abandoned her ambitions after marrying Mahler. Mahler once criticized her compositions, saying, “Her music is steeped in nauseating dilettantism, and her mind wanders lazily between fantasies of submission and domination.” [1]

 

However, Alma’s compositions, such as the first song in her 5 Lieder collection, Die Stille Stadt, reveal significant talent. The opening melody descends D-C-B♭-A-G, echoed in the piano accompaniment, which modulates the B♭. Her harmonic abilities—such as seamlessly transitioning from diminished seventh chords to different tonalities—demonstrate a solid musical education and an exceptional grasp of the floating harmony system prevalent in the late Romantic era.


Despite her evident talent, Mahler discouraged her from composing and harshly criticized her work. Why he did so remains unclear. During a period of separation, Alma had an affair with architect Walter Gropius. Gropius deliberately sent a letter intended for Alma to Mahler, revealing their relationship. Distraught, Mahler sought therapy from none other than Sigmund Freud. Though the details of their discussions remain confidential, it appears Freud’s counsel was effective to some extent. On his way home, Mahler wrote the following poem: [1]

 

 

... “I love you!”—these words are the source of my strength,
The melody of life I’ve wrested from pain.
“Oh, love me!”—these words are the wisdom I know,
The root note grounding my soul’s melody.
...

 

 

Though Mahler often spoke harshly of his wife, his deep affection for her was undeniable. Despite receiving Alma’s promise to end her relationship with Gropius, their marriage never truly recovered. It was during this time that Mahler composed his final symphony, Symphony No. 10, pouring his inner torment into its notes.


The right-hand side of Figure 4 illustrates the intense dissonance in the symphony’s first movement. Mahler constructs an A∅7 chord anchored on the high A used in the climax of Symphony No. 5, juxtaposed against a G♯˚7 chord and further destabilized by a C♯ in the bass. These dissonant non-chord tones—G♯ and C♯—once enriched the expression of love in his earlier symphony but now create a wailing dissonance, mirroring the irreparable rift between Mahler and Alma. Unlike Mozart, who resolved dissonance consonance, Mahler leaves it unresolved, generating overwhelming beats akin to the tension in their estranged relationship.

 

 

3 And Beyond

 

At the turn of the 20th century, composers like Mahler began to incorporate dissonance and new harmonies more broadly into their works. Russian composer Stravinsky experimented with percussive dissonance in The Rite of Spring, while Hungarian composer Bartók fused folk melodies with unconventional harmonic elements in works like Music for Strings, Percussion, and Celesta. These approaches contrasted sharply with the lush tonal landscapes of Russian composer Rachmaninoff and the modal harmonies of French composers Debussy and Ravel.

 

Mahler’s exploration of dissonance influenced the next generation of Austrian and German composers. Arnold Schoenberg, for instance, developed the twelve-tone technique, which treated all twelve notes equally, laying the groundwork for serialism and influencing composers like Boulez and Stockhausen.

 

 

3.1 Penderecki’s Threnody to the Victims of Hiroshima

 

By the mid-20th century, clusters of dissonance had become a musical material in their own right, as seen in Penderecki’s Threnody to the Victims of Hiroshima. This piece, performed by a large string ensemble consisting of 24 violins, 10 violas, 10 cellos, and 8 double basses, opens with each instrument producing its highest possible note, played fortissimo. This piercing soundscape immediately immerses the listener in the raw, visceral emotions evoked by the aftermath of an atomic explosion. Penderecki employed not only traditional techniques like arco and pizzicato but also a wide range of modern techniques, including harmonics, col legno, and percussive effects such as tapping the instrument. These pushed traditional boundaries to deliver an unprecedented auditory experience. The result is an overwhelming cacophony that elicits a mix of terror and sorrow, perfectly encapsulating the devastating effects of the Hiroshima bombing.

 
펜데레츠키의 ’히로시마 희생자를 위한 애가’(좌), 리게티의 ’Atmospheres’(우) 에서 발췌
Figure 5: Excerpt from Penderecki’s Threnody to the Victims of Hiroshima (left) and Ligeti’s Atmospheres (right)

 

 

3.2. Ligeti and Atmospheres

 

Hungarian composer Ligeti took a different approach, using dissonant tone clusters to create atmospheric effects in works like Atmospheres and Lux Aeterna. These pieces evoke the microscopic movements of molecules or photons, suggesting a detached beauty akin to background noise. Director Stanley Kubrick famously used these works in 2001: A Space Odyssey, pairing them with scenes of space’s stark, indifferent beauty.

 

Ligeti employed micropolyphony to achieve these effects, layering clusters of notes to create evolving textures. In Atmospheres, for example, a single sound unfolds over several minutes, shifting imperceptibly as it builds tension. Unlike Penderecki’s violent dissonance, Ligeti’s approach emphasizes subtlety, drawing the listener into a meditative and almost otherworldly state.

 

 

스탠리 큐브릭의 영화 ”2001: 스페이스 오디세이” 중에서
 

Figure 6: From Stanley Kubrick’s film 2001: A Space Odyssey

 

 

As I write this, a couple is arguing in the café behind me. The woman, having discovered the man’s infidelity, caused a commotion at his workplace. I don’t know who’s more at fault, but their quarrel feels like the tone clusters—simultaneously tragic and comedic. As the Korean pansori verse says:

 

 

“Oh, such is life—futile indeed.
Do you not see the peach blossoms in the Eastern Garden?
They bloom for but a moment.
Mistress of the brothel, why do you laugh?”

 

 

References

[1] Jens Malte Fischer. Gustav Mahler. Yale University Press, 2011.

[2] Hermann LF Helmholtz. On the Sensations of Tone as a Physiological Basis for the Theory of Music. Cambridge University Press, 2009.
[3] Reinier Plompand Willem JM Levelt. “Tonal consonance and critical bandwidth”. In: Journal of the Acoustical Society of America38(1965), pp. 548–560.

pre-image
Dissonance, Mahler, and Beyond - Part 1

Hello! I’m Ste, and I’m researching voice AI at Gaudio Lab.   Lately, I’ve been diving into the music of the classical composer Gustav Mahler here at Gaudio Lab!Mahler is known for his music, which is full of strong dissonances and intricate structures.   To express the emotions of love and pain he felt throughout his life, he used dissonance as a key element in his compositions. Let’s explore these dissonances from an acoustical, musical, and historical perspective!     1 Dissonance   Music is an art that uses sound as its material. Just like how meat or seafood is essential to make a flavorful soup, harmonious chords are the base of music. But, just like those basic ingredients alone can’t make a rich soup, the addition of dissonance, with its strong personality, is what gives music its depth and variety. A piece of music made only from harmonious tones would sound monotonous, like a simple broth made only from meat. Dissonance, on the other hand, can provide stimulation and create tension, making the music more compelling—like adding seasoning to a dish.     1.1 Complex Integer Ratios   How does dissonance occur? It has to do with the ratio of frequencies between two notes. Sound is created by vibrating objects, and when two notes vibrate at a simple integer ratio, we hear them as consonant. But when the ratio between their frequencies is more complex, we hear them as dissonant. Here’s a table showing the ratios and consonance/dissonance of different intervals based on the note A:   Table 1: Intervals, Integer Ratios, and Consonance/Dissonance Based on A   For example, the perfect octave (2:1 ratio) and the perfect fifth (3:2 ratio) sound harmonious to the ear. These ratios form the foundation of basic harmony. As the frequency ratio becomes more complex, dissonance emerges. For example, intervals like the minor second (16:15) and the augmented fourth (45:32) sound dissonant. These dissonances are useful in music to express tension or emotional shifts. But what happens when the ratio is extremely complex, like 441:440, where the two notes’ frequencies differ only slightly?     1.2 Beating   Another way to understand dissonance is through a phenomenon called "beating." This occurs when two frequencies that are very close to each other vibrate together, causing their amplitudes to modulate periodically. If this modulation happens quickly, it can create an unpleasant tension—almost like a drum being struck rapidly. This effect is called "beating." According to the physicist Helmholtz, the tension caused by beating is most noticeable in the 30-40Hz range. [2] The principle behind beating is simple, and can be derived from the addition formula of trigonometric functions.   If we combine two sine waves with frequencies f1 and f2, the resulting wave can be expressed as:   Using the trigonometric addition formula, we get:   The cosine term represents the center frequency, vibrating at (f1+f2)/2, while the sine term represents the beating frequency, vibrating at |f1−f2|/2. This is fascinating: when we combine two waves with different frequencies, a new center frequency appears, and amplitude modulation happens, causing the sound to either amplify or dampen. I remember when I was younger, I thought that when I played C and E on the piano, I would hear the note D in between, but I was confused when this didn’t happen. It’s amazing to think that this phenomenon happens when the sound sources are the same and frequency differences are so small!   Figure 1: Combining the A tone of 440Hz with 441Hz, Bb, and C#     Figure 1 shows the result of combining two different frequencies in a graph. (a) and (b) show the combination of 440Hz and 441Hz. Despite the complex ratio, this combination sounds consonant, with a center frequency of (440+441)/2 = 440.5Hz and a beating frequency of (441−440)/2 = 0.5Hz. This results in a slow modulation, producing a "waa-waa" effect without creating the percussion-like tension we associate with dissonance.   (c) and (d) show a dissonant minor second, with a beating frequency of 29.33Hz. This rapid beating creates tension, like a drum being struck about 30 times per second, which makes the sound dissonant. (e) and (f) show consonant tones of A and C#, with a beating frequency of 110Hz. This frequency is too fast for us to perceive as rhythm, so we hear the notes as independent and consonant.   Figure 2: Dissonance within an Octave     [2] Helmholtz suggested this analysis of dissonance through beating, and [3] Plomp and Levelt, after experimenting with human subjects, created a graph of how different ratios within an octave affect the perception of dissonance. If we reproduce this graph based on intervals, we get Figure 2. From the graph, we can see that intervals like the perfect unison, major/minor thirds, perfect fourth, perfect fifth, and major/minor sixth have low dissonance compared to neighboring intervals. The dissonance increases between the minor second and perfect unison due to the tension created by beating. However, as the interval nears a perfect unison, the dissonance quickly decreases because the beating frequency slows down, no longer creating that percussive tension.     1.3 Non-Chord Tones   As mentioned earlier, dissonance is essential to music because it adds variety and depth. In music, the relationship between intervals becomes more structured, forming a hierarchy of harmonic tones that make up the system of harmony. For example, the C-E-G chord forms the harmony of the C major chord. The G note, a perfect fifth above C, is consonant, and the E note, also relatively consonant, fits in between. Once this harmony is established, all other notes, like D, F, A, and B, are considered non-chord tones, as they create dissonances with C, E, or G.   One of the key features of classical music, such as Mozart’s, is the clear hierarchy between chord tones and non-chord tones. Non-chord tones create tension with the other chord tones, and thistension is resolved when non-chord tones move to a chord tone. This process is known as "resolution" in musical terms. Within a clear harmonic structure, this tension-resolution pattern moves the music forward, like fitting pieces of a puzzle together. It’s similar to how in language, nouns are modified by adjectives and verbs, and adjectives and verbs are further modified by adverbs. Music without non-chord tones, consisting only of harmony, might sound innocent and naive, like a child speaking only in nouns, without adjectives or verbs.   Figure 3: Mozart’s String Quartet No. 19 "Dissonance" – Red lines show dissonance, green lines show resolution     Figure 3 shows how Mozart creates dissonance in his string quartet and elegantly resolves it. The passage begins with the cello playing C notes in succession. After two beats, the viola introduces an Ab note, creating a sense of tension due to the use of a first inversion chord. Normally, music starts with a root-position chord, and we expect the perfect fifth to be present. Soon, the second violin adds an Eb, forming an Ab chord and slightly alleviating the tension.   However, this is quickly undone by the A♮ played by the first violin on the next beat. Even though the viola moves to a G note to soften the dissonance, the G note continues to create discomfort, as it forms a major second interval with the A♮. Additionally, the Eb from the second violin, which was aligned with the Ab major chord, now forms an augmented fourth when paired with the A♮, creating the dissonance known as the "devil's interval." Eventually, the viola’s A resolves to F#, and the second violin’s Eb resolves to D, completing the resolution.   In this work, Mozart uses counterpoint, inherited from 16th-century church music, to carefully manage intervals, skillfully moving between consonance and dissonance. However, he stays within the limits of classical harmony and counterpoint, experimenting only within the boundaries of traditional tonality, leaving the development of functional harmony to his successors.     Dissonance, Mahler, and Beyond – Part 2 to follow     References [1] Jens Malte Fischer. Gustav Mahler. Yale University Press, 2011. [2] Hermann LF Helmholtz. On the Sensations of Tone as a Physiological Basis for the Theory of Music. Cambridge University Press, 2009. [3] Reinier Plompand Willem JM Levelt. “Tonal consonance and critical bandwidth”. In: Journal of the Acoustical Society of America38(1965), pp. 548–560.

2024.11.06
after-image
Taking Your Content Global: How to Solve Music Copyright Issues

Case 1. The Netflix documentary <Dear Jinri>, which delves into the untold story of the celebrity Sulli, encountered a critical issue during its production. The film aimed to incorporate a self-recorded video of Sulli, left on her phone, as a key element in the end credits. However, the video, much like a personal diary, also captured the background music La Vie en Rose by Edith Piaf. Due to unresolved copyright issues for the song, the filmmakers were unable to use this deeply emotional scene.   Case 2. A popular South Korean variety show was exported to Taiwan, where it became a major success. However, complications arose when the music used in the show couldn't be cleared for copyright in Taiwan, forcing the production company to pay substantial royalties. This unexpected cost ended up surpassing the revenue earned from the show's export, creating a net financial loss.   Case 3. A well-known vlogger encountered problems while trying to upload a video of their live experience at a soccer match. The stadium’s background music included copyrighted songs, which triggered YouTube’s Content ID system for copyright infringement. As a result, the video could not be uploaded as planned.   These cases highlight real-life inquiries received by Gaudio Lab, a company dedicated to solving diverse audio challenges.    Whether for individual creators like YouTubers or professional broadcasters, producing video content often requires dealing with unexpected situations where music must be removed or replaced. And these examples are just the beginning.   Let’s explore how Gaudio Lab resolved these music copyright challenges.     (Photo = Still from Dear Jinri)       Why Replace Music?   The most common reason is to address music copyright issues.   Broadcast networks typically pay copyright fees for music used during the initial airing of a program. To elaborate, most networks pay a fixed fee to music copyright management organizations, which grants them unlimited rights to use songs managed by the organization—but only for broadcasts on their own channels.   However, when the same content is distributed to platforms like Netflix or FAST (Free Ad-Supported Streaming TV) channels, additional music licensing must be secured in each country where the content is streamed. This can incur substantial costs. Even if the content is already complete and ready to sell, licensing fees can easily exceed the revenue potential.   To navigate such copyright hurdles, creators have historically relied on the following options: (1) Abandon exporting the content. (2) Remove the affected portions entirely (impossible for variety shows with music throughout). (3) Replace the music with copyright-free alternatives (a process called "video re-editing").     For option (3), the process has been entirely manual: editors use basic audio tools to isolate the music, search for similar alternatives in limited databases, and seamlessly reinsert the replacements into the original video. This painstakingly labor-intensive process often takes two to three weeks to edit a single 60-minute video.   Individual creators like YouTubers face similar obstacles. When dealing with videos containing copyrighted music, they often (1) abandon uploading the video, (2) edit out the affected portions entirely, (3) (recently) use music separation technology to remove the music while retaining other audio elements.   Music copyright is so crucial for platforms like YouTube that their Content ID management system detects copyrighted music in all uploads and suggests these same three options.       GSP-MR: A Revolutionary AI Solution for Copyright Issues         Gaudio Lab’s latest AI-powered video audio editing solution, Gaudio Studio Pro - Music Replacement (GSP-MR), revolutionizes how these challenges are addressed.      How does it differ from traditional manual methods?     First, users upload their video to GSP-MR, where the AI separates the audio tracks into dialogue, music, and effects (commonly referred to as DME). Gaudio Lab’s audio separation technology, GSEP, is renowned for its exceptional quality. It has been widely recognized as the best-in-class audio separation technology. At CES 2024, Gaudio Lab even won a CES Innovation Award for its product, Just Voice, which enables real-time voice isolation.   Next, the separated music track is analyzed by AI, which identifies individual pieces of music and divides them into segments. The system then uses a music recommendation engine to search its extensive music database for similar tracks. This database contains tens of thousands of songs across various genres, all cleared for global use. Unlike low-quality AI-generated music, these tracks are created and uploaded by professional artists worldwide, ensuring top-notch quality.   Finally, the AI seamlessly re-mixes the replaced music track with the original dialogue and effects tracks to produce the final edited video.       With GSP-MR, creators no longer need to (1) abandon their content, (2) cut out affected portions (3) or spend weeks and substantial resources on manual re-editing. Instead, they simply upload their video to GSP-MR, wait briefly, and… Boom! They receive a video where the original musical intent is preserved, but all copyright issues are resolved. (Blind tests conducted with professionals found that many couldn’t distinguish the edited version from the original!)   The clients mentioned earlier have already adopted GSP-MR as a core solution. Exporting content internationally involves more than resolving music copyright issues—it often requires broader content localization efforts. Clients naturally request additional services like dubbing, subtitles, cue sheet generation, and video editing.   “I want to handle everything within GSP-MR!”   As demand grows, Gaudio Lab’s research and product teams get busy adding new features, enhancing both convenience and performance. The name Music Replacement (MR) no longer fully captures the product’s capabilities, leading the product owner to contemplate a new name. 😅   Most of GSP-MR’s features are powered by AI. In our next post, we’ll dive deeper into the technical details of each process and showcase the convenient tools in the GSP-MR Editor. Stay tuned!  

2025.01.24