Linguistics in Performance
Description
In this prototype we are exploring the possibilities of Natural Language Processing in the context of live performance. As the performer speaks the system analyzes the spoken words and with the help of the Oxford American Writer’s Thesaurus (OAWT). Each semantically significant lexical unit initiates its own semantic cluster (or gets attached to some existing one in case the cluster has been already generated) which is iteratively expanded over the corresponding entries in OAWT.
As the story is being unfold by the performer the environment is shifting from a state to another according to the censoring data that occur from the system analysis. In the documentation we see the sounds and the videos that shift from an industrial environment to a more forest-like environment accompanying the shift in the improvisatory story of the performer.
As the story is being unfold by the performer the environment is shifting from a state to another according to the censoring data that occur from the system analysis. In the documentation we see the sounds and the videos that shift from an industrial environment to a more forest-like environment accompanying the shift in the improvisatory story of the performer.
Live Animated Speech
Description
During this experiment we are exploring the possibilities of transcribed text from spoken utterances. The spoken words of the performer already burry a communicative value as they already have attached to them a semantic component that has been evolved and transform through out the history of language. The objective of this prototype is to see what happens when light, imagery and texture is added in the text and how it is perceived by the performer.
The pieces of text when encountered as audio-visual forms animated by quasi- intelligent dynamics in digital media become widely perceived as animate entities. People tent to regard animated glyphs as things to be tamed or played with rather than a functional and abstract system of communicative symbols.
In this example, the text is being projected on the sand and the performer as moves around in the space the words are moving as if they were particles of sands that are being misplaced by the performers footsteps. Sometimes they move away fast and sometimes they change slowly. The amount of the text is being manually manipulated creating a relation of co-performance with the performer, the live speech and the media artist in the space.
The pieces of text when encountered as audio-visual forms animated by quasi- intelligent dynamics in digital media become widely perceived as animate entities. People tent to regard animated glyphs as things to be tamed or played with rather than a functional and abstract system of communicative symbols.
In this example, the text is being projected on the sand and the performer as moves around in the space the words are moving as if they were particles of sands that are being misplaced by the performers footsteps. Sometimes they move away fast and sometimes they change slowly. The amount of the text is being manually manipulated creating a relation of co-performance with the performer, the live speech and the media artist in the space.
Spatializing Voice
Description
This is a sensory and immersive installation, that explore concepts of embodiment and presence in responsive spaces. The experiment is built on theories of space and body, where space is regarded as an extension for sight and an entity that is created by the motion of the body.
The installation is looking into sound as an essential element in examining embodiment, presence, and agency in responsive spaces,by examining the affordances and the semiotic values of everyday objects, and deconstructing the coupling between objects and their expected sonic properties. This is done by creating a disconnect between the action that is performed on these objects, and the expected sonic reaction. This disconnect can be a dislocation in space, in time, and in the social and representational codes with which the object is loaded, and sometimes defined.
This concept examines verbal communication as a potential way for interacting with spaces. Furthermore it is examining concepts such as: how does a conscious space demonstrate personality in interpreting speech and dialogue, and if predefined behavioural patterns can influence our perception of space,presence and agency. This experiment works on a semiotic and social level by examining and deconstructing the representational codes of objects and entities, and it work on a psychological and cognitive level by looking into ways to construct meaning without resorting to representational media as a feedback system.
The installation is looking into sound as an essential element in examining embodiment, presence, and agency in responsive spaces,by examining the affordances and the semiotic values of everyday objects, and deconstructing the coupling between objects and their expected sonic properties. This is done by creating a disconnect between the action that is performed on these objects, and the expected sonic reaction. This disconnect can be a dislocation in space, in time, and in the social and representational codes with which the object is loaded, and sometimes defined.
This concept examines verbal communication as a potential way for interacting with spaces. Furthermore it is examining concepts such as: how does a conscious space demonstrate personality in interpreting speech and dialogue, and if predefined behavioural patterns can influence our perception of space,presence and agency. This experiment works on a semiotic and social level by examining and deconstructing the representational codes of objects and entities, and it work on a psychological and cognitive level by looking into ways to construct meaning without resorting to representational media as a feedback system.
Prosodic Feature Extraction
Description
Research into prosodic characteristics of speech has focused almost exclusively on two aspects of its contribution to communicative events. The first concerns the processing and disambiguation of the structure of spoken language and the second concerns the role of prosody in indicating the emotional or attitudinal state of the speaker. However, we extend beyond this use of prosodic features by searching for ”sonic patterns” of speech that can be treated as texture rather than semantic representations. Thus, we propose to shift how we regard speech features, not to see how they can be used to represent words but instead to modulate time-based media that condition a live-action event.
Semantic representation may seem more canonical from the perspective of natural language processing. However a speaker can say an utterances that all map to the token ”hello” but in countlessly different ways. In this experiment, we are investigating how we can overcome the obstacle of simple representation and use the prosody of the speech to enhance the experience.
The sound is being produced according to the movements of the performer. The visual transformations are driven by the prosodic features of the performers voice and the sound of the room, however the initial position of the visual effects derives from the position of the body in the space.
Semantic representation may seem more canonical from the perspective of natural language processing. However a speaker can say an utterances that all map to the token ”hello” but in countlessly different ways. In this experiment, we are investigating how we can overcome the obstacle of simple representation and use the prosody of the speech to enhance the experience.
The sound is being produced according to the movements of the performer. The visual transformations are driven by the prosodic features of the performers voice and the sound of the room, however the initial position of the visual effects derives from the position of the body in the space.