Dave Johnson Gaming Journal: January 2015

Music and sound in video games

By Dave Johnson

Music and sound in games is a vital ingredient to creating emotions, pushing along the narrative and immersing the player. Similarly sound effects can emphasize or accentuate gameplay elements or create moments of emotional impact. Below I have listed three moments of music and two sound effects that I believe effectively illustrate this principle.

Super Castlevania IV - "Dance Of The Holy Man"

This first piece of music I consider one of the best video game openings I have ever heard. Although it is taken from a very old 1994 SNES game, I think the amount of care, imagination and emotion in this opening track is hard to beat. The SNES always had an advanced set of sound chips for it's day, and I think this is the technology used at its very best.
Set in a region of Transylvania many centuries ago, this historical time period is reflected in the use of stringed, woodwind and organ instruments. The hero, Simon, must set out on a quest to kill the resurrected Count Dracula. His journey takes him across the Transylvanian landscape and into the Count's castle. Along the way Simon must fight many classic monsters from myth and the movies such as Medusa, skeletons, golems, mermen, ghosts and even Frankenstein's monster.
The opening bars of the track begin with a booming gothic organ, reflecting the game's horror theme. The melody suggests trepidation, danger, heroism and hope and successfully helps the player to identify with Simon's determined nature. As the music builds, the emotion in the player builds, empowering both them and their onscreen character. There are quiet passages that suggest misty landscapes and eerie settings, again alluding to the landscape of gothic novels.

Bioshock Infinite - "God Only Knows"

The fact that this actual 1966 song by 'The Beach Boys' turns up in a game set in 1912 is a massive clue to the player that something very weird is going on. As the player progresses throughout 'Bioshock Infinite', various anachronistic versions of popular songs turn up throughout - rather than being just clever jokes by the developers, they are actually brilliant clues to the game's plot. Without spoiling anything, there is a very distinct reason that songs are turning up in 1912 many years before they were actually been written. Music as part of a game's plot is rare, and even rarely handled in such a careful, intriguing manner. To those who haven't played the game, I strongly recommend completing this game to discover the full story behind the music.

In terms of how I, the player, feel about this it's difficult to put all my emotion into words. Being a musician and music fan I immediately recognized it, followed by my asking the question 'why on earth is it here?'. By that I mean in terms of the game's plot - by the time I reached this point in the game I knew that more or less anything was going to be possible and I was prepared for some surprises. The melody and the arrangement took me back to another age, a different time and way of living. A different set of attitudes and social values. This was probably what the developers intended as social values play a huge part in the game's plot later on. Interestingly, pretty soon after 'God Only Knows' things go downhill fast for the protagonist Booker DeWitt. The gentle tones of 'The Bee Sharps' bely the violence that shortly follows and create a false feeling of security for the player. In this sense, music is used to toy with the player, putting them on edge and teaching them not to trust anything they see.

The Secret Of Monkey Island - "Main Theme"

This pirate themed game is full of mystery, voodoo magic, swashbuckling and surprisingly, comedy. The player cannot die as it is a graphic adventure where the emphasis is on puzzle solving, rather than combat. It is the kind of game that rewards the player's patience and logic rather than punishes them for making mistakes. This friendly approach is reflected in the game's music. The flute used in the main melody creates an inviting atmosphere as well as evokes feelings of mystery and magic. The game's main theme, like 'Bioshock' and 'Castlevania IV' also uses a certain kind of instrument (in this case woodwind) relevant to the time period to give a sense of history. Used throughout, the melodies and arrangement are soothing and relaxing, reflecting the game's pace and giving a good indication to the player of the kind of environment they will be experiencing.

Alan Wake- "The Poet and The Muse"

The way this song makes its entrance into the game is probably as impressive as the song itself. Following a clue from two ex-musician characters in the game who advise him on how to proceed, Alan arrives at a deserted shack in the middle of some woods. Here he finds a copy of an old LP, recorded some decades before. Putting it on, Alan hears this song and realizes the lyrics are his clue on how to make progress. They fill in some details of characters mentioned in the games back story and hint at characters yet to come.

I think it is a genius moment and a brilliant song. It is wistful, melancholy, mystical and suits the feel of the game entirely. The music is a literal narrative and game play cue to the player as to what to do next, effectively being an in game hint. I can honestly say I've never seen a story in a game moved along in quite such a way before. As a piece of music it works, and as a narrative device it also works fitting in with its environment perfectly.

Max Payne 3

This example is quite clever and one I found recently whilst playing Max Payne 3 on the Xbox 360. It is a use of sound that directly relates to the development of Max's character as the game goes on. In short, Max starts the game a washed up alcoholic and drug addict who's main aim in life is just to survive it. As the game goes on, he becomes more in touch with his purpose in life and learns to make sense of the world around him again. Several pianos that are placed strategically throughout the game can be played by Max when he walks up to them. The first few produce a discordant noise with a barely recognizable tune under the din. The later ones produce a clearer more recognizable set of notes that eventually reveals itself to be the original Max Payne theme tune. This is a clever, symbolic use of music to illustrate Max's road to recovery. From the players point of view, it helps to create a sense of reassurance that Max, though not exactly ok, is on the road to recovery and that they have helped guide him to that point.

Red Dead Redemption

This piece of music plays about 1/3 of the way through the game when you finally cross the border and head down into Mexico. The hero John Marston is a loner and an outlaw, someone with a brutal past and a highly uncertain future. At this point in the game it is just you, your horse and a huge unexplored landscape in front of you. The story lies waiting to be discovered, their are new characters to find, enemies to fight - it is a massive turning point in the game. When this tune kicks in it underlines the excitement and suspense of what you will discover. It also speaks of Marston's character and his lonely personality. To the player it helps them empathize with both Marston and the desperate situation he finds himself in, along with truly immersing them in a western.

Sound effects in games
The 'chime' noise in the legend of Zelda has become an important audio cue in the history of the series. It is a sound with a clear objective - let the player know that they are making successful progress or have discovered a secret. It is a soothing sound with the player quickly learning that to trigger it is to progress in the game. Therefore its psychological effect to the player is one of an audio ally and friend. :

Similarly, the sound of failure in Metal Gear Solid is shocking, suspenseful and keeps the player on their toes. In a game that involves stealth and sneaking, the sound means the player has been spotted and must take action over what to do next. Due to its sharp, piercing nature the psychological effect of this sound is one of a warning. To hear it is to fail to a small degree as the player must hide from or fight enemy soldiers. Therefore the sound is an audio cue to the player to up their game and become more alert if they want to survive.

The power up sound effect from the Super Mario series is one that empowers the player. It is probably one of the most iconic sounds in gaming, actually matching the process your character Mario undergoes. I would describe it as a 'growing' sound suggesting your character becoming more powerful from a smaller state.

Why is music and sound FX so important within games development?

Because it helps to immerse the player and help them to feel emotions whilst playing the game. Music can define settings, characters and plot events within a game, along with providing audio information.

What is waveform, (wavelength, amplitude, frequency); pitch; Hertz (Hz); decibel level (dB); sound generator (loudspeaker)?
Pitch: Pitch is a perceptual property that allows for sounds to be ordered on a frequency related scale. Pitches are compared as higher or lower from each other and are associated from musical melodies.

Hertz: Hertz are the unit of frequency in the international system of units. They are named after Heinrich Rudolf Hertz who was the first to provide conclusive proof of the existence of electromagnetic waves.

Decibel level:The decibel (abbreviated dB) is the unit used to measure the intensity of a sound.

Sound generator: A sound generator is a vibrating object that produces sound such as a speaker, or musical instrument.

What is Foley?Foley is the pre-production of everyday sound effects that are made for games, films, video or any other form of media. The sound effects that are created can be anything from the closing of a door to footsteps or even telephone rings. Without Foley, there would just be music and audio dialogue which would give the impression of the world being unrealistic. Foley helps to flesh out the world and make it more authentic and convincing.

What is Timbre?Timbre is the distinguishing feature that makes one musical sound different from another. For example, if a piano and a guitar were playing the same musical note at the exact same pitch and loudness, timbre will be the difference between the two musical notes as they sound distinctly different from each other despite being the same note played in the exact same way.

-----------------------------------------------------------------------------------------------------------------------

Task: List at least 3 games or films that use sounds as information.

Indiana Jones and The Last Crusade

The grail knight that Indy meets towards the end of the film has his own musical theme. This theme crops up throughout the film whenever he is mentioned or the knights medieval origins are referred to. As the grail plays such a massive part in the healing of the relationship between Indy and his father. Henry Jones Snr, it becomes symbolic of the power of restoration. In the 4th film, Henry is mentioned briefly and we see his photograph on Indy's desk. The grail knights theme is again briefly heard. The music itself becomes a metaphor for hope and restoration, being played in key scenes between Indy and his father.

Jaws

The distinctive Jaws the tune marks the points in the film when there is about to be an incident involving or attack by the shark. The music deliberately resembles the shark's motives, behavior and personality. If we consider that when seen in the film the shark always appears very briefly and it's fin is usually seen first. The theme mimics this behavior with the first strains very indistinct as it takes us a while to recognize whose theme this is. The first few bars actually sound like they are rising from the depths of the piece, much like the titular shark emerging from the abyss of the ocean. Once established, the score picks up the pace with a menacing rhythm and discordant stabs of notes warning us that tragedy is about to strike. Whenever we hear the theme, we immediately become tense as we know what is coming. The strains of the strings mimic our own strained nerves and illustrate horror in an audio way very effectively.
In many ways, the music becomes like a voice for the titular shark - menacing, aggressive and powerful, waiting to strike as it builds up momentum.

Aliens

The aliens from the 1979 movie 'Alien' and 1986's 'Aliens' are amongst some of the most vicious, threatening creatures ever committed to screen. They are distinctive in their sounds as well as their looks. The sounds they make give viewers the sense of them being overgrown insects with a pack mentality. They slash, crawl, bite and pierce anything that gets in their way. The Foley applied to these creatures makes them seem merciless, primeval and completely terrifying due to the use of screams, hisses and slithering. The use of these effects in the context of the films environments is particularly effective - often the films features sparse, silent expanses of architecture making the bursts of Alien noise that break the quiet even more jarring and unsettling. Silence and Foley are both used to expert dramatic effect to highlight the entrance of these creatures and shock the audience. They become speech for the creatures, giving them a personality and air of intelligence along with a highly destructive nature.

Staying with the Aliens universe the M41A MK2 marine pulse rifle is also worth a mention. It's distinctive sound stands out in the history of cinema as a futuristic weapon that is also highly believable. It's airy rasp implies high technology, deadly efficiency and a brutal nature (in many ways representing many of the humans in the films too.) The rifles sounds as tribal and angry as the Aliens too, standing out in the battle scenes as a voice that shouts above the combat. The sound sparks both emotion and suspense in the viewer due to the large gaps of noise between rounds fired.

----------------------------------------------------------------------------------

P1 - Exercise

List a range of music/FX sources

http://www.freesound.org/

- A collaborative database of Creative Commons Licensed sounds

http://www.openmusicarchive.org/

- A distributed source of digital and out-of-copyright sound recordings
http://musopen.org/
- A provider of recordings, sheet music, and textbooks to the public for free, without copyright restrictions
http://www.stockmusic.net/
- Royalty free music that can be downloaded and used for projects
http://www.incompetech.com/
- Royalty free music that can be downloaded and used for projects
http://www.pacdv.com/sounds/index.html
- Free sound effects for download

http://www.stonewashed.net/sfx.html
- Free sound effects and music for download

You could also hire musicians or record music and sound fx yourself by using software such as Audacity or Adobe Audition.

Using at least 3 of your favorite games explain the purpose of game music

Super Castlevania IV

Mood:
Dark and gothic, a quest through a medieval landscape and environments to slay Dracula. The music is designed to fit the region and time period with strings, woodwind, orchestras, various sorts of percussion and church / pipe organs. The atmosphere created is suitably dark and gloomy, which becomes more triumphant and upbeat as you near your final goal.

Action:
Depending on the level, the music alters and creates different emotions. An early level has a subterranean waterfall inside an echoing cave. The music here is suitably soft and haunting, a noticeable contrast to the next level which features a journey through a forest of monsters. Here the soundtrack is more upbeat, though dark and menacing as if describing the creatures waiting in the dark. There are occasional lighter passages of music that suggest hope and heroism, but these seem to be overcome by the darker pieces of music.

Towards the end of the journey as you near Dracula's lair, the music becomes more heroic, driving the player on and encouraging him/ her to complete the game. It is a clever device in that it seems to audibly state the fact that the time for evil is almost at and end the hero's victory is close by.

Suspense:
The use of music in the boss fights creates drama and excitement, building to crescendo as the battle goes on.

Intro sequence:
The use of a strange discordant and menacing tone against a picture of a gravestone sets the scene for Dracula's return. Strains of a broken flute and string composition filter in. There is no melody to speak of, just a set of sounds overlaid which suggest an oppressive and evil atmosphere. It heavily suggests that something is about to return and wreak havoc, setting the events of the game into motion.
Lightning strikes the gravestone, shattering it and changing the soundtrack to a mystical and eerie tune that plays over an onscreen narration. It gives the player the feeling of villains, heroes and an epic quest .

Closing sequence:
The closing sequence is the final battle with Dracula where you defend yourself against an onslaught of attacks. It is an ominous, sparse piece of music, but is particularly interesting in the sense that when you kill him the ambient track is broken by cascades from an organ. A shaft of sunlight disintegrates Dracula and the mentioned organ kicks in, followed by several long drawn out chords. It is almost as if light has triumphed over dark and this is reflected in the musical theme.

Credit sequence:
Killing Dracula triggers the credits sequence which is deliberately scored and written to tell the player that they have a achieved a great victory. The melody is upbeat, harmonious and triumphant suggesting that all is well and the player has finished an epic quest.

Plot advancement:There is no plot to speak of in the game other than the player's objective, being a series of side scrolling platform levels. The biggest clue to a narrative however lies in the music on the map screen which changes from dark themes suggesting fear and anxiety to slow, winding optimistic themes as you ascend the tower of Dracula's castle. Similarly the first level of the game speaks of determination and survival which gives way to themes of fear and darkness as you realize the odds are stacked against you. In a way the music does reflect an emotional journey and suggest a change in the characters personality.

Much of the music in the game seems to be inspired by the monsters that inhabit particular levels. Walking skeletons inspire bone-like percussion sounds, whilst water based creatures reside against swirling, deep sounding jazz inspirations.
Interactive adaptive music:
The game does not really rely on interactive elements. The extent of these in Castlevania IV's case are themes for the player dying and winning.

The Legend of Zelda : A Link To The Past

Mood:
In this action adventure you play Link, a boy who must save Hyrule from a dark presence and rescue princess Zelda. Along the way the player will traverse every part of a vast landscape and meet a huge gallery of characters. Despite using some tried and tested plot devices and the simple ideas of good vs. evil, the game is quite light hearted. This is reflected in the game music, offering the player a range of heroic tunes and atmospheric tracks that are specific to the regions on the map. The Lost Woods for example has a haunting, eerie woodland melody, whilst Hyrule Castle has a real sense of royalty in its music. These musical ideas work in many ways like a compass, helping give the player a sense of where they geographically are in the land.

Action:
The game is largely concerned with fighting enemies and collecting objects to solve puzzles. There is a huge amount of exploration in the game and a lot to find and collect. There are musical themes to let the player know when they are on the right track, have made progress or have found a secret. Similarly there are also themes to suggest failure and defeat which are more discordant and melancholy in their tone.

Suspense:
Zelda does have moments of suspense such as creeping through dank caves or eluding capture by castle guards. These pieces are usually audibly a lot quieter and low key to emphasize silence and quiet. This could communicate to the player that they need to keep quiet, or that they have entered a vast, cavernous area.

Intro sequence:
As the game features monsters, myths and magic quite heavily the intro sequence to the game is heavily dramatic. The score for this sequence begins with an ascending series of chords from a string section, suggesting an epic tale is about to unfold. As this sequence continues, the music matches static images on the screen - the images depict wizards, heroes and battles, each having a suitable audio ambience to match the events being narrated in the story. Wizards gathering to form a binding spell has an undertone of mystery to it, whilst an image of a battle has a heroic yet wistful theme, suggesting loss.

Closing sequence:
The closing sequence is a boss fight with the villain Ganon as he transforms into several guises. The music is fast paced and bombastic, emphasizing drama and urgency. This is the final battle in the game where the success of the player will end the quest. The music reminds the player of this constantly and helps to keep up the energy and charge adrenaline for the sequence.

Credit sequence:The music in the final credits sequence is gentle and serene, helping to give the player the sense that Hyrule has been returned to normal at last. It also uses the same sense of geography set out earlier in the game as the camera visits various places in the game world. We see how the characters lives are returning to normal and that evil has been banished. As the credits roll we hear updated versions of a number of themes, this time arranged to sound happier, more positive and peaceful. The overall impression is that the player has restored balance to the world, creating a lasting connection in the memory with the game they have just completed.

Plot advancement:
The plot features a number of developments, at one point having the player travel to a dark mirror image world where everything is a reflection of the light world. As an example of how important music is in the game, the background music becomes somber and urgent to reflect this plot update. The plot of the game is a sequence of light and dark sequences where the player discovers, interacts with and influences events and characters. The games musical themes mirror plot ideas that revolve around optimism and pessimism. These can range from tunes suggesting heroism and hope to ones of despair.

Legal considerations of using music and sound effects

Copyright law?

Copyright gives the creator of any media or product exclusive control over how their work is represented, reproduced, preparation of derivative work, used and distributed. Music, books, video and software can all be covered by copyright law. Exclusivity means only the creator of such
work, rather than anybody who comes into possession of it.

Limitations imposed by copyright law
As an example, when you buy music or sound fx, copyright law would forbid the user from:

Making a copy and giving it to a friend

Making a copy and then reselling it for your own profit

Using the music or sound that is installed on a network (unless the licence approves this practice)

Unlawfully renting the music or sound fx without the consent or knowledge of the copyright owner