Session 16 MIT 4.370 Interrogative Design Workshop

um hi everybody welcome welcome to the Center for transpal studies I'm loresa Harris the associate director um thanks for coming I am um very pleased to introduce Pia lyen who needs very introduction here um having not only taught here um but she actually received a master of science and visual studies here art at the visual art program in 1999 um after coming to the States from Finland onbre um he has practice stands video performance drawing and something that I like to call social engineering but tonight she'll walk us through three projects in which she uses her body to reenact in a way to learn captured images of of different of different types um in all her work there's this this boundless interest in and and faith in human capacity for learning engagement and change and this can be read not only in the end product but in every stage of the process um her engagement with three MIT undergraduates Caitlyn wheer Jen FIA and J Lee um as part of the undergraduate research opportunity program has shown me personally um what learning by doing can actually mean and of course this is a classic MIT type learning way way of learning um and wanted to mention that her time at the center has been very significantly supported by the MIT Council for the Arts um and I say this not just to thank him but to recognize that they recognized how much and on how many levels she could enrich MIT um now just some statistics pia has shown her work performed and been in Residence around the world in New York City at PS1 at the sculpture Center at the lman cultural Council in Mexico City and Hell City Ino and vus um next week she'll be performing in Andrew CS Gallery right that's a month sorry next month next Saturday it's going to be at the very center of Art and politics okay next week at the ver Center for art and politics that's in New York and then next month she'll be performing as as part of the as part of perform perform as part of this major new performance planning on perform 05 that's also in New York um her videos are in the collection of the Museum of Modern Art she has lectured widely on her work in this such as Columbia Ry Yale and NYU uh The Institute of architecture Fones in Paris um after she presents I hope you could stay a little bit to talk um people are very excited to see you thank [Applause] you thank you for a wonderful introduction um and thank you everybody for coming and I'm thrilled to see so many people here um I'm I've spent since summer I spent the summer and the fall here at MIT as a fellow and um started working on this new project that has really sort of evolved in ways that I wasn't anticipating necessarily and um this um really kind of in order for me to present that work for you I really feel I need to show first two other projects that I've done um previously that really led up to the assumptions and the ideas that I had when I started this project and then I'll show what's G been going on and then sort of we can maybe all think about what where the project might be going at this stage and uh what kind of new openings and Vistas are there having said this in other words uh this is definitely very much a work in progress so um I want to be quite dis discussion about it I want to um engage all of you as much as possible in whatever you want to ask comment or bring forth criticism Etc so I will welcome any kind of um utterance from the audience um and um I don't know if I should take responsibility of this term social engineering and just shortly briefly explain it a little bit um social engineering is definitely something that Lissa um worked out to together with me when I was working with um projects like the sauna um the hybrid sauna and the public SAA the hybrid SAA here at MIT and the public SAA at PS1 and uh what came about from that type of work where um I make an intervention into public sphere with something that can be seen as architectural but even more than just architectural um some sort of a social interaction Machinery or system so like for instance a Sona but um Sona is a social practice and by inserting it in unexpected places and by um manipulating the architecture that um gives the confines of that social interaction I made an attempt to um bring forth through social practice questions about nudity and public space and um gender politics so that's in a nutshell what social engineering can could be envisioned as and I I think that it's um not um totally irrelevant to mention about this in this moment because um what this continued what the what this work that I'm going to be showing you has to do with has to do with exactly this sort of um intention and and attention to social interaction that I've always had in my work and um I personal feel that at this point the work has started to evolve in ways that um it starts to sort of also um prompt questions about well what's social interaction here and um what where is that um initial tenant of my practice going at this moment so uh I want to start by talking about my New York Times project it's there any questions at this point or any comments while we start the DVD yeah you can move up closer there's uh there's space here and there's some chairs here as [Music] well okay I got to do it this [Music] way there's one seat here and one over there if somebody wants to be seated okay good so I'm starting with the New York Times project um and with the video called laicon that was um the first version the first stage of the New York Times project um after 911 in New York where I live um I experienced something that I I think everybody in New York was was agreeing with that there was an immense Collective spontaneous Collective reaction from people to to go out every night every evening people went out during the day as well and met up in the park there was a certain kind of sense of of collective togetherness mourning and and sort of helping each other to deal with the the the constant I mean the two first weeks there's this constant anxiety no matter what the public media was trying to um say that there was resilience and resolve and da da da there was a certain Collective reaction that I I I've never experienced before and um it was expressed in the flowers in the candles in uh statues public statues being covered in peace marks and etc etc the police would not react would not take things things down I think they were up for maybe a month or so and after that slowly the police started to clean up the city and go back to normal so that was a um Collective ageing process I have um not seen uh you know the width of it and depth of it I've not seen before and and uh what really shocked me personally at that time was how the media really captured hijacked and transmuted that reality into uh something that they could use uh to well they weren't actively using it for any kind of propaganda or anything but it became one of the many elements very very sort of easily used for arguing for action of the government or in the city that um I would say that most of the people who were being photographs weren't agreeing with mainly of course the sort of latest outcome is is the Iraq war that um these kind of images just become so powerful as arguments even especially because they're irrational especially because they're based on uh Perce as authentic and spontaneous emotional reactions they become as if they were neutral and personal they can then before because of that they become so vulnerable to be used in various public ways in various political ways um they're like floating free floating markers that you can just then sort of fix on some message and say this is what we got to do you know let's go bomb Afghanistan so this was my personal reaction to what happened there and I felt that because of this okay it's time for me to to do a piece that um tries to analyze and re understand what does the media do when it does what it does and so for one year there was actually a year after 911 for that one year 2002 to 2003 I collected images off the pages of the New York Times and solely of people grieving and I traced all of these images I think I traced maybe 600 of them um and then I reenacted these images in front of the video camera in my in the sort of solitude of my studio so what you're seeing in this video here is um me meticulously carefully reconstructing each of those images basically the gesture of a grieving person and what I try to do is repeat mechanically with my body I don't don't try to interpret I don't try to invest any emotion into it I just want to look at the gesture as it was a language uh almost like trying to um bring forth the alphabet of grieving as it is being presented by the New York Times and in this video I feel it's very important that you can see the construction of the gesture and you can see that moment of striking the pose and uh potentially also be struck by the effective uh effect of the uh of that image that has um been reconstructed um having the I mean there's this video goes on and on for an hour there's um 400 of the gestures that I have reenacted um it was important to me to wear gray and have all the props that I used be of the same gray um the idea was that you know people were showing pictures of their lost loved ones they were holding up Flags they could be any kind of flag they because these are from all over the world and from all over all these different situations children men women uh I had to come up with some way of an ostensibly at least neutral um physical um apparatus in a way where then you can really just focus on the gesture itself rather than oh this must be from Afghanistan oh this must be an American from 911 because there would be the colors or the particulars of of the context so uh it's kind of like the 18% gray of the neutral gray of the photographer it could also be seen as the uh the gray of the the newspaper s of um undistinguishable but then when I had finished the video I printed out stills from the video and based on those Stills I then draw drew again the gestures so I have these instruction books of um 100 gestures each of these various uh images and um I'm going to actually show that to you now um that was the Second Step stage of the project and for me the drawings were the sort of end result in the beginning I thought they would be the end result where I really was hoping to get exactly the alphabet of the gestures 400 gestures as signs like these are the ones that are used in New York Times and they can be arranged and put together in various combinations but what happened when I showed this piece to a lot of people was that because of their very laconic quality um I'm sorry I have to find the right spot here because of the laconic quality of the drawings especially when they weren there was no information about the context there was no caption there was no uh spe particular sort of information people started to invest their emotion into these drawings what I tried to take away and discard in order to analyze people were putting back in so it became um another a different kind of process of emotional quality that I had not foreseen and in a way sort of reinstalling some sort of um some sort of a psychological iCal process that definitely is not happening when you're reading through the newspaper when you're just browsing through the newspaper you don't pay attention to the the the images you're combining it with the the various information that come out but this way something else happened um you have to start thinking why is this woman in this particular way is it a woman actually is it a man uh who did she lose and uh why is she crying where in the world is this what situation is this depicting um and I like that actually I I felt that you know I was just going to end up with cold uh emotionless images that could be um that then could be used as this sort of clear-cut information about well this is how the media uses it or whatever but instead I ended up with something that was totally blow blowing off of my hands because people felt things when they watched this when they looked at this so um that got me into thinking about what is emotion and what does it mean that people have to hold here when when somebody's lost their son and they're standing next to the body of of their son they they the mother has to hold her heart what does that mean where where is it coming from um how much is there this biological reality to it that actually your heart feels like it's breaking how much is it that it's a learned cultural thing where you've been socialized into um expressing your emotion in a with a physical gesture and actually getting into a stage where the physical gesture then produces that emotion how much is it this way how much much is it um um um can one say that there actually is any biological reality to emotion what kind of neurological processes take place in the body why do we physically feel something when we grief so I was left with these questions and I don't have any answers maybe maybe somebody else has um and um then it got me into thinking more about also um well grief is kind of clear it's such a strong uh sensation uh that the gestures become very obvious and we can really talk about physical pain um but what about other kinds of situations where we uh have some sort of an effective relationship to our environment that makes us act in a certain way in response to that environment in expectance of having some kind of retro um um reciprocal sort of interaction going on um I was interested in people like gofman and later uh writers as well about how um we perform ourselves on an everyday basis we um we we we do what we do we do the gestures we use the voice in these particular ways because we've learned to expect our audience to be able to respond to to uh these gestures and and voices in the way in a certain way so who I am really becomes an A A an equation of how I'm being reflected by you and this has become for me something that then can be faceted into various situations that prompt us to perform ourselves as ourselves in different ways a certain kind of splitting of our identity that comes from various contexts that change the way we um actually literally feel about ourselves and identify ourselves but it is about this interaction so um grief and especially public trauma public uh Global trauma situation where you have to at the same time grief on a personal level but at the same time you're being photographed by global media what does that mean to your performance to your identity that was the first one that I was asking and then I started to think about well what are the other Frameworks um for instance um corporations um where as an employee a person has to um very well be aware of their uh hierarchical position their ethnic and racial position um their their sort of expectancies of how they are supposed to move up or down on the ladder inside that cooperation and also you're sort of situated within a framework of um a certain kind of economic Society capitalist corporate Society so you you are embeded uh within a certain framework of beliefs and social interactions so um how do you then Express Yourself um what is your identity in a situation like a corporation and that's why I did the the next piece which was um uh a residency artisting residence in the uh in the in a capitalist Corporation in downtown Manhattan and I asked the employees of this I literally had an office inside their office so it was a working office and I went every day as everybody else I walked into the office and I stayed there all day for a month and I actually had to really carefully also think about what I was wearing and how I would prod you know present myself because I had to not only I actually had to sign a an agreement form where they said that I commit to wearing appropriate corporate tire at all times I think they might have seen my sauna piece and were afraid that I might come naked there or something someday so but I signed it and and I actually really had to spend quite a while thinking about what should I wear because I don't I don't think I should wear what they wear it's not me infiltrating I Am the Artist in this environment I should play that card of being something else but I can't be too crazy I have to be approachable I can't scare them off so I ended up having these nice loose Ellen Fisher pants and a T-shirt and sort of a little more colorful but not too bright and not too crazy it was I think it was very successful so I'm sitting in my office with a window that shows into the the rest of the space and um I sent out emails to everybody that anytime you want to walk in come to my office and we'll do an interview and I would interview each of the employees when if they came for quite a while I would really interview them and try to find out what they um what they are you know like what's their education what's their position in the in the company uh what's their values how do they feel about life and how do they feel about their work all these different questions and then eventually we would step into this situation where I had the camera film myself and they they would then direct me as I would reenact the gestures that they perceive themselves to be doing on an everyday basis in the office a lot of times we would have to simulate situations I would be like okay so you're walking down the hallway and you're seeing your boss and you know you screwed up something excuse me the word uh something you know the previous day so how do you talk to your boss at that moment or you know um you're bored at a client you're talking on the phone how do you talk how what what do you do and um here's then some of the results and again I use the same technique of um reenacting in front of the camera and then uh also making Stills drawings but uh what I'm showing now is just the video of some of the reenactments I again there's hundreds of the gestures but this is just a sort of distilled version of the whole thing the place was called the global Consulting Group how appropriate takes a little bit of acrobatics of your mouth I have to say I would then show this in my office in a monitor and then I would also have the drawings on the walls and it was quite interesting to have the employees in the office come in to my office and recognize themselves in the drawings which is me you know these are of me and the other thing was for instance uh um how people were I mean I at least having spent so much time on this can really say what's the social stratification of the gestures what is a typical receptionist gesture what is a typical um kind of a mid-range uh um actioners gestures what is a CEO gesture for instance when I showed the drawings of this particular gesture now showing I asked uh uh the employees um who they think that this might be and they said oh it must be one of the CEOs and uh the truth is that a CEO would never use a gesture like that it would be way too patronizing absolutely ever never uh that was the receptionist talking to the maintenance guy that you know you piece of something something go and clean up that bathroom so um that I found interesting in that sense that I think that most people still link authority to mother and father to this kind of um very sort of uh patronizing uh ways of body language and that they are easily still sort of um uh seduced or or uh taken in by uh for instance the CEO is never using those kind of uh bodily uh Expressions a CEO would be much much more sort of um uh expressing certain interest you know and and time for you know attention but at the same time a certain kind of um I'm giving you five minutes of my time that's a lot of time for me you know so you better respect it and and but never showing any kind of direct um authoritarian um gesture that's how I uh realized at least while I was in global Consulting Group however it might be different in some other company you never know and also definitely different in different parts of the world um do you have any questions at this point or anything like that NOP MH do you any direct observation of people or was it all their interpreted view of themselves yeah this is a good question because when I did the New York time space I used images that I got off the pages of New York Times which means that I was not not the person who went there and took those snapshots of people in their very intimate moments of grief these were images that had already been published and they they definitely those people must have been aware of the fact that they were being photographed um and so when I went into this Co Corporation I really felt like I don't want to be a spy I don't want to be um snatching their MO private moments or anything I want what I'm looking at is identification what I'm looking at is how do you see yourself in this Corporation so therefore I thought that it was totally fine to exactly um for the work that I'm doing to have them tell me through interviews what it is that they do I have to Grant it though that a lot of times I would have them some of somewhere very very sort of easy outspoken in their verb in their bodily language so there was no problem but a lot of them would just be well I don't know and I you I don't you know and so we would talk and I would actually look at them and then I would say well you know you do a lot this so what is this you know so how so I would help them and then find a little more of what their gestures were there were very interesting moments for instance um the the computer consultant guy of the company he would never he was so unmoving but then he had this one thing he would go like this like I thought he was going to snap his fingers he kept on pressing them down the table and I asked him why why he was doing that and he said that well when you are taught how to teach people in a company about how they should be using their computer you should never go over and show you should never lean over you should never take their seat you you only have to you know hold your hands behind your back and tell them what they need to do so he had developed this sort of I think he had an urge to go over to the screen and no no press this button and so he he had developed this way of suppressing that urge by pressing those fingers towards the desk um so you know you come across a lot of various things um but no I would not want to go and um just sort of um well two things I don't want to spy on people I don't want to find their most intimate moments when they're picking their nose and then make a gesture of that um but I also don't think that what I'm doing I mean what I'm doing has a little bit to do with anthropology but I don't think I have to follow any sort of system of you know system of collecting information that then can be sort of pro used as evidence for stating certain claims I am um I'm still just making art and um leaving it open for the process to tell me what what I what I find out and how I'm going to make that into something Vis visible something for you to look at okay both both cases You' been talking about structur very very you're also Liv creative scope I to say um you're focusing and you're also making a decision the minute you open the New York say is this an expression of G yeah that's that's very good question you're already you already you already said the typology of g j yeah and so maybe you didn't see a gesture of maybe on the sports pages when somebody missed the uh you got the last strike yeah I I had a long time to think about that yeah oh yeah yeah yeah absolutely yeah no I and I I spent a lot of time thinking about that what it because first decision was to just find images of grief and then as soon as I started doing that I started to realize well how do I Define that first of all New York Times has filtered their own set of gestures that they approve and then there's going to be a second filter and that is me no matter what I choose to do it's going to be a filter so I had to kind of in the process think about that exactly how is it that I'm you know collecting these I included images of mothers and fathers of teenage African-Americans who who had been shot by the police because I perceived that that was um um well yeah now I'm jumping into concl I I realized at some point that most of the images in New York Times if they showed grief was of um of political content it was either war or terrorism a few times you would have natural catastrophes and I made made that decision then that I wouldn't include the natural catastrophes into my selection and of course now I've totally changed my mind after new orans because I started to think that okay I'm going to choose the the images that are of political content uh so I I did leave you can see that in on the SP P Pages you would have some at some point and they could also have political content but on the whole I decided to to sort of narrow it down a little bit this way because there was so much um but then exactly um I open it up in situations like for instance the police abuse cases I included the images because I I perceive that to be police I mean State terrorism so um but to your question about narrowing um the project down I do think that I often times start with some tenants that um almost like a mind game like look if I look at this this way and I just sort of narrow it down to looking at this particular strain of this particular thing and then I see where it takes me and even if I narrate down very very carefully I will always end up having these problems um basically that it's impossible to follow the line of thinking or the the categories that I've set up for myself and that sort of forces me to rethink the whole thing and and I see that as a way to really kind of go someplace in with the thinking and definitely it is in in um in dialogue with the work the the what actually comes out of the work itself which is something I want to talk about when I start talking about the MIT project oh just a quick question how certain these images has affected I I was even more animated before I I think I've I've can I reenact myself as the 14year old um that might be one of the biggest challenges of my life I I could say that um and I should maybe do it um I think that um what was really important for me was that um that question of emotion affect authenticity of emotion um physical gestures are perceived to be more sort of inadvertent and spontaneous almost unconscious whilst whilst verbal expression would be something that we knowingly do and I think that um gestures are huh I think that gestures are just equally language and equally um a sort of um a repertoire that we've learned however um um I'm still struggling with the that relationship and I'm I'm not sure where it's going to go I I want to have a dream panel one day where I'm going to have a neurobiologist with me and then a psychologist and then an artist and whatnot to really kind of hash that out what is biological about emotion what is cultural about emotion but I do definitely think that [Music] um uh with this process of the New York Times piece and the corpcom at the same time What a what was really affecting me was that I got deeper and deeper into this questioning that I think is very very important to my whole life to my whole personal life um this um problem of self- representation identity and um a sense of wholeness of emotion which I sely severely and sincerely doubt so did that help any there was a one day Symposium in New York City Saturday it was part of the separate part we called um York Academy time NY and and the reason I mention because it's to your bringing Crossing disciplines and with art um and the Vitality of Art and usually artists do already incorporate several disciplines um unlike Specialists here in MIT very expert at narrow scope in general sence Al I don't to say MIT is one of the more sort of crossover kind of cross disciplinary places I've ever been to talking most russan Arts versus science Tey website because they will be putting up what they call synops U briefings of one page or something plus Little Cook of each of the panels so the artist and they were cross R so people from MIT working with now movement in Optical camera movement biolog you to something you can go and and then share with sort of similar attemp because those those can be and can be and so we can maybe talk a little after discussion and and I would like to get the website that would be very interesting um speaking of the biological origins of emotion um there have been a lot of studies of that I don't know if you're familiar if you use that you know as part of your inspiration but when you were doing these projects uh did you feel any of that did you feel that you know putting yourself in you know the the position of grief did you feel oh or oh yeah that's a that's another thing I we won't have time I was going to show you some of the live performances that I've done in in public spaces in New York in Vienna in Mexico City where I perform these gestures live and and um a lot of times I have people come up to me after the performance like oh my God you must be exhausted so much emotion and I'm just like that was you it wasn't me um I just focus on performing it I focus on reenacting it repeating the gesture and my body remembers already very well I when when I did it once I have 10 hours of me just really meticulously reconstructing every single thing about those gestures it got registered into my body it's quite amazing I had no idea it was that effective but now when I see that same image it's very quick for me to put out the same thing again um and I don't think about it that much um so all that emotional that comes from you seeing the gesture but of course yes you could also say there's that old saying of you know first you been first you bend down on your knees and then you pray meaning that you don't get religious sentiments before you do the gesture and so one could imagine that if I was like this I could actually feel it and I I'm sure if I wanted to I would I could evoke it you know I could bring it forth but I I don't because I have to perform um but also H your you know the point of the decision to do that thing with 91 people their victim well well the reenactment of all of these gestures is not 9911 um it's all people all kinds of people from all over the world over the course of one year of the New York Times so it's not specifically about but the decision to do this was really initially I just wanted to shed off all that effective thing that we have when we look at an image of context of um flags of lar pictures of loved one just all that paraphernalia that comes with producing emotion producing um empathy and just look at the politics of what that is that language like for instance when um over that whole year there were maybe five six uh incidents in Middle East um some bombing in Israel retaliation in some areas in Palestine and you ended up having an article of course reporting on that and you every single time you would have a closeup of an Israeli mother grieving over a over a coffin um you know of over some lost blood one and then you would have a picture of Palestinian funeral of maybe some pre-team boy but what you would see was just a mass a black mass of Palestinian angry men throwing stones um now the Palestinians turn their funerals into riots because they use that as a platform for political message equally this Israeli mother is used as a platform for a political message um and the the article might actually sort of write about everything in very very objective way and really kind of talk about how the Palestinian boy was innocently killed and whatnot but what what is going to be the immediate effect of that article is these two images that are placed next to one another where you have a closeup of something that really evokes empathy and then you have angry mob so it's very clear how um this is a way of using images in an effective way so what I did was I would take I would reenact the mother and I would reenact a throwing of the stone and then I would make a drawing of them they would be on the same level they would have the same language they would not be specific to context they would just show that these are two different sort of um uh signs in that alphabet and it's not that I'm saying that by this I could say that from now on we're going to think about the the the the news reporting in a different way it's just about pointing at this um the different ways that these images are being circulating there and how how they are being sort of embedded or changed or transmuted I don't know if that was really very empty you know good explanation but something along those lines I don't know be but I think it's a 15 well no mob us has been an experimental Black Box platform that you no no longer have a permanent space but what one of the artists doing what what you're talking about if you're here for art ranges for yearly party they also have perance AR and one is of pictures from newspapers specifically narrowing a subject to bomb cars okay and inserting in uh a different image in so you think you it looks very the Montage is quite marful when you seeing it up close and then you're realizing question [Music] assumtion also the Atlas Group and W Rod who's done this uh whole fantastic work on on uh duping as in terms of news reporting um yep I know we have to move on uh you no so you remember I talked about the um the um of of this idea of different frames different contexts creating prompting us to um perform an a partial us a partial identity of ourselves um in different ways so that you know in a way we we are there we are who we are we really are in that moment but we are also performing at the same time a a c so that we can play in that effective Network and um so what I wanted to then start looking at um here at MIT having done The Grieving part having done the coroporation you could imagine maybe doing religion I started to think well what about scientists what about researchers what about uh people who work in in Academia and specifically people who uh worked with who work with finding out about things they don't know about and that being a relationship to something an object be it abstract uh a concept or a real physical object and I I felt that you know we we we already know that there's this idea of of course this IR rational thinking and scientific thinking where you build an argument by by basing it on experiments and finding out facts but at the same time um everybody who's working with finding out something of things that they don't know they they also have an effective relationship to that object um they they also function as psychological entities not just rational thinkers but they also engage in this whole thing with other levels of their um mind rather than just IR rational so again gestures being this inadvertent and unawares kind of language as we think I thought that this would be interesting to look at how through gestures a researcher might Express something of this relationship to the object of study that they're studying [Music] um that might reveal or might not something of the the things that don't come across in that dissertation or in that paper that they're writing so I wanted to be here at MIT and go into Laboratories and sit down and work with uh scientists researchers and just see how they do that interaction um and this is now the part that's very much a work in progress so that this is where I'm going to be threading on thin eyes and um I uh I want to show you just a a few examples of uh what I did in the beginning over the summer I went to various groups of people and interviewed them pretty much in the same idea as I was doing with the ccom that I'll just interview them and we'll see how they talk about it and I'll I'll see where I get um but it has changed anyway so here are some steals by um and this is Mitch yakim who's a a designer um from um School of Architecture and I definitely think that in terms of animated speech I I've met my match here he is describing a a car that he has worked on is Mitch here he said it's going to come but maybe it's um and I really kind of you know they're hilarious right but I also really like the fact that here we are he's talking about a car that does not exist he only imagines it and he uses his body he uses his body to really become it that's fantastic that's really you know an unknown that he imagines and this is the same car being now reviewed um and um what happens here is also interesting that it's almost like a wave of gesture that I give this to you and then Bill Mitchell picks it up and then you know another person picks it up and it go it goes on and it repeats itself and it becomes a collective idea you know uh and here also what I found interesting was that I'm just quickly going to pause here um this the tall lean guy here is from the General Motors so he's the main sponsor of of uh this project and um he's of course an Oldtimer he knows everything about serial production of cars and designing cars and all that so he does appear more sort of laidback and cautious these are the few moments when I saw him make any gestures but this is very much the way he would be most of the time kind of taking in this position of I'm just observing and you know there's definitely a sense of power position in this and and I also like this picture very much when they're all peeping in like that and this is also then um how [Music] um research is one thing but also engineering and design in invols also envisioning something that does not yet exist and so you are building and working with things that you're you're imagining at the same time as you're you know part of it you know you know how to weld or you know that this is this and this metal but you are doing something that has not yet been there so you are partially knowing something partially working with something unknown and I really like the the sense of um the the intensity of attention and sensitivity of of these people when they were working on the car it was quite intense to look at them so um having followed this process over the summer with the Gary car um and some other works as well I I sort of started to um wonder where I'm going with this and uh I I actually haven't worked on the gy car after that um I I totally could reenact this and make drawings but it didn't seem like the most important thing to do at this moment and um I also became more and more interested in the sort of relationship of this object and this idea of this um that um uh ion hacking it was actually Caroline Jones I interviewed Caroline Jones about um ideas of uh working with scientists and how scientist relate to the objects of knowledge and she said that iron hacking has written this book where he's basically saying that um uh scientists relate to their object of knowledge through two very of distant ways like rarely do they relate to the object directly so it would be either by representation models temp sort of showing a model of something or by changing the conditions in which this object is and then measuring it with instruments so it's always by instruments and different sort of um uh things in between the object and the researcher itself that the researcher is interacting with the object on a general h uh now I went to see Rodney Brooks at Cale computer science and artificial intelligence lab and talked to him about the work that I'm doing and wondered if I could work with any of his researchers and that's when I really kind of came into this conclusion that well if we think about what iron hacking has said what is interesting about these humano robots if anything is exactly this that at the same time as it is the instrument it is the object itself and those researchers do interact directly with this object so I kind of went into seale and into the humanoid robotics group and that's where I'm paused with my work and looked at it much more carefully so first introducing Merz and legion's Legion Aranda's um project Merz oh now I have to put some volume up um this is actually Merz is sort of a followup up of kismet by Cynthia Brazil so Merz is um a face a head that can uh make facial expressions based on affect and uh it's um interesting and something that I might not want to venture into uh is the Des description of How It's been programmed but as far as I understand it's fairly sort of simple um operations of uh um like um merch speaks to you the one that's trying to learn work part but it doesn't necessarily know what it's saying it reacts to the your tone of voice your facial expression and then in a way mirrors back now mirrors back to you meaning like if you speak angrily to it it will get you know angry maybe but the way it seems to be working is that it's impossible to tell why it's reacting what way in what instance in any case so there's a lot of um different functions and different variables that interact at the same time so that Its Behavior really becomes confounding and definitely like an infant so you're like is it constipated why is it making that you know you really kind of um have to start relating to it um as an unknown and um another thing is is that um uh this is a situation where um Legion went to talk to a group of people that came to see it but merch is left alone so this is now and merch is not aware of the camera Merz does recognize if you try to look at look it into the into its eyes it will eventually see look at you into your eyes so you feel like you have eye contact with it so you really kind of get you get this sense of having an interaction with somebody more than just a machine now but it does not recognize camera as I so um I have the camera filming Merz while nobody else is looking at [Music] Ms so this this is the teenage group interacting with it [Music] each you want your yeah [Music] I for some reason merch really liked this one particular [Music] [Music] girl so this teenage girl and where they [Music] connected don't get mad and get glad [Music] um okay I'm just going to let it roll for a while now so Domo is another of these humano robotic robots and um domma has the same capacity of of being able to see you look you in the eye but this is particularly now specialized in recognizing objects moving around and then picking them up and learning about them as it holds it them in his its hands and this is one of the very very early efforts I think that this is actually yeah do they they learn in the sense are they important question yes um do they take information in and then recute it so the next time they can reuse the same information it feels so insecure talking about this things because I know they're all sitting over there but um my answer would be yes I mean the idea is that they they do accumulate that information and can reprocess it and use it in the future will Rea H well I don't know if there's any gesture recognition going on no but it's more this the merch can merch can learn words if you wave a yellow thing in front of it and say yellow merch will learn yellow and then when it sees a yellow object it will say yellow but if I approach push my hand into it face it'll probably PR yeah I mean this is and would repeat that gesture again make the same gesture so it's not it's not actually it doesn't learn in the sense that you and I would you sort of extended sense of learning the sense it would it would uh store information that could be reused in a similar situation they they do in a very specialized specialized case their bodies M so yeah it makes a difference it does make a difference um and um it's certainly has made a difference for me because I've gotten more and more interested in in what this means for my interaction with them or anybody's interaction with these characters these uh robots um and um so my first idea was that okay I'm I'm filming this I want to film the researchers as they interact with them now I noticed that okay I can also have these these visitors how do they re interact with these um robots and then I realized that but I also want to reenact the robot because because um um first of all when we make this kind of humanoid entities we are doing it because we want to mirror ourselves we want a mirror for ourselves and U there's different levels of when the mirrorin becomes too perfected so that it becomes scary and uh thus you get the Frankenstein kind of syndrome or the the what H um and you know so the there's definitely this constant balancing between uh human to human all to human or machine and thus other or two human does other too and um so I really felt like I I want to start understanding more exactly the robot itself as the mirror as this um what we desire kind of thing um and so having done some reenactments of the robot and then making these drawings I realized that uh I I I I went through classical training of of human you know life model studies I I I train years of human body how to draw a human body and whenever I draw myself after reenacting or even another body I can do it pretty quickly I can get from to simple keys I can I can sort of figure out what's going on but with the robot I realized that I I I'm totally at a loss I don't know this thing I don't know how to draw it and so um that's the next step I realized that I have to learn this object and so I started making life model studies for the first time in 20 years but this I'm off the robot and drawing for me really is a way to analyze what is there it's not just looking it's really looking through it's about understanding what's behind what I'm seeing as well as what's in front of it and so um it in a way it's a very sort of bodily way of knowing to me and that's why I've been doing this um after having done these drawings I felt more confident in going back to the drafting board with the re ments and um this is definitely something I have to continue with I I still don't know it well enough um and also when you make these drawings I I really feel like I can start understanding what what works effectively in this what is machine and what is what is that in those things that make you look at them and want to talk to them and so um and here Al I think this these are very nice deals from a research video that eron made with Jeff that um how this sort of touching the robot is this attempt is this question um actually before I showed that um that's the other thing that's very interesting with Domo is that it's not just a machine that you know if it puts its hands out hand out if something comes in the way it will just go straight through that's what a machine does right this one actually has a way of sensing that there's something in the way and will respond you know it's not like it's just going to machinic go through so when you do touch Domo there's there is a dialogue and so here's me for the first time interacting with Domo Gra and I just really think it's amazing how Domo keeps looking around as I'm going about trying to understand it it's like what are you doing pardon Domo yeah I don't know what Domo means do can can you say AR uh it's just a it's a Japanese word as a reference to song something and then you want to thank you yeah yeah yeah yeah somebody and talk about something and then get some information or whatever you know he doesn't like or he doesn't like it we can just say oh you know and then we can just glare smoing out yeah smoing out that's that's good yes so you mentioned that you're using uh drawing as a way of uh uh getting to understand how robot works and so on and so forth there was a question earlier about whether you reenacting some gestions triggers emotions in you uh would you actually um following that line of reasoning think that uh um if you were to you know gain some ground in terms of uh understanding through drawing and other means um how a robot act you could also reverse engineer the codes that generates this behavior for the robot do you think you would be able to do that and how would you go about it well that would be very very daring of me to say but um with the drawing um like I said it's not necessarily learning to understand how it functions how it works it's more about learning what it is uh you know relating to it and I actually think that um in this process of making these robots it is useful for the maker to know what it is you know like what what is it that it's doing on another level than just um the function what is it you know how and I I know that this is something that um surely is being discussed and is being you know considered um this that what does it look like and like it's not um coincidence that mer looks like a baby face it's but it's also not a coincidence that it doesn't have a a furry tail you know like expiration is not in understanding what makes that thing do what it does but it's sort of a self- exploration you're trying to find means to access your own you know the things of like I looked it in the eyes it's sort of a projection of sorts is oh yeah yeah yeah and I'm really kind of wanting to uh I think on the bigger hole that is very much around what I'm working that how do we project and why do we project and what does it do and how how can I intervene in that projection because I I think that um it's always something good to be looking at a little critically um because um if you don't think critically about what you project uh you are going to be manipulable yeah I see something very interesting in comparing what you're doing now to the the the earlier project you're empty you empty out those to other emotion and now you're looking at something performing you're projecting into it yeah it see it seems to seems to you're s switching positions a little bit in terms I mean just thinking in that in that might what you're doing I I think that I'm oscillating between the two positions constantly and um and definitely yes I mean um I'm I'm trying to look at what happens in that relationship and what are the different projections not just me but others as well um well let me just show this because this relates to what you were saying so this is now me um you saw that the video where Domo is catching the ball this is me now reenacting those gestures I chose specifically this overall of course because I um the robot is sort of displacing and replacing the the mechanical uh work you know the car mechanic worker so um the robot is taking on the the dress of that it's a little bit of a internal pun or an internal joke but I also think that it works nicely as a some sort of a costume you have to have some sort of costume anyway so when I was making this I wasn't quite sure how this was going to pan out but as you can see exactly the drawings that are on this side of the the walls are now the drawings that came out of this video and then putting them in a mirror relationship to Domo as it is doing the same gesture and of course here I have to fill in a lot of blanks like physically I can't do what Domo was doing with his three fingers um I have a mouth and a nose Domo doesn't but I think that doesn't really matter and then the next one was um and this is me now reenacting merch merch's facial expressions as he is interact it is interacting with Legion in the very first video that you saw I noticed that I think that there was something wrong with Mer's neck at that time and it was constantly sort of tilting back so for me it was pretty hard to reproduce that tilt constantly but I was doing my best I really like Mer's eyes when they were like this this sort of very very um intense eyebrows and this is lingin in U and I'm reenacting the same exactly the same still of I mean this is just one of the many Stills but this is in direct relationship to one of the gestures of Merz as Legion is is looking at Merz and Merz is looking at Legion and here this is the teenage girl that um it was such a messy video so it was maybe hard to notice but there was that teenage girl that really connected to mer and in the very beginning of the video actually Legion says that Merz must think that you're the mother and I picked out all the gestures that the teen well many of the Gest that this particular girl was doing in relation to Merz this is when when she's actually touching Mer's neck um and here definitely coming full circle from The New York Times piece because here this particular moment I'm trying to really reproduce this Look of Love That this teenage girl had and I mean she was just she was fascinated it was quite amazing um and here are now some of the drawings she she kind of touched the neck not sure sure whether she was allowed to she could have sneaked over to touch it and looked at him with this very very meaningful gaze it was very strange moment and here is now mer and Legion which is me reenacting both of them and then placing them on the same page so this is actually the same as the video and so these are the same stills from the same moment but because I'm reenacting them separately putting them together it was kind of interesting to notice that I couldn't of course I couldn't fit them they don't look like they're from the same video anymore and they actually look like they they to me they're like as if they were each other's thought bubbles so that's it um this is how far I've gotten this far I'm just curious process is going from thinking about the scientists and their object and the interaction with the object and then going to the object itself and trying to re the object itself and then the way that SLI into one another curious how how you're trying to sort through all that yeah um I just really very very recently did this reenactment of Merz and Legion and put them on the same page and it came out as a shock for myself what what it looks like because it looks very strange um but for me I I feel like it is um potentially really expressing I mean I felt that the way I was in a way becoming it's not that I was becoming merched but I felt that um I was doing what merch would be if merch was human potentially and I was putting myself in this position situation that um um when you then look at it next to Legion it's like Legion is actually very emotive very much relating and you know and then there's this sort of this character who just sort of um responds the way it wants to you know it's like it's much more unwilling to reciprocate and I don't think that that that feeling would have come just from looking at the video itself but actually from going through this process of trying to um be both of them potentially that's that's I mean that's a kind of a realization that I have um I wonder if you're also asking about this quality of embodiment what it means or well also just thinking about the the the gesture that it took to make the robot try to catch fall and the decision to focus on the robot response versus the the gesture of trying to get the robot to Heaven a gesture of you have to repeat that I I couldn't quite it to the extent that the you're focusing on for example the robot CCH fall then there's a whole series of gestures to try to get that reaction in the first place um yeah yeah yeah yeah to produce it to have the robot produce it and like SLI over Focus the robot Rob study together that that's and this is where the whole code question comes in as well because that is not necessarily like yes there is that this person who's waving the ball definitely and yes I haven't done it yet I you know I have yet to come to that situation where I have one camera filming the other and you know from other Ang def but I have definitely realized that I have to do it I have to get two cameras and I have to really kind of put together this whole situation um but also there's there's this whole other gesture that's going on which is the coding which is the the designing of the thinking and um and I actually do want to work with that as well and see how that comes but I don't know code I don't know how to work with that so um I have luckily this Europe Caitlyn Miller who hopefully will help me figure that out we'll see but um this is I I wasn't even going to talk about this part because it's it's another very very big alley you know like highway to to go into that um how do we produce this but even with that I it's still about the same thing it is about us wanting to have this mirror so it is us uh reflecting our own gesture in a way there also a control that you want to control the opposite by certain responses you responses in sense control the object Rob I this just a negative not even only controlling the object I'm trying to control the researches the whole situation the S trying to something from from the you know re might react in a certain way stimulate yes I mean and and um you know like well that your question goes to so many it's mirror gets to be okay I want this to move in that direction the mirror becomes even more complete that you mean that the robot completes is more complete mirror I don't know how to react that's the question if they react erratically well that's the thing often then then of course it's not the but if they react to certain things in similar ways all the time then you learn you yourself learn how to control this motions I mean I suppose you you have you have a whole set of things that you do you certainly learn how to act very particular ways make it do what you want to do naive if person comes to interact with it they can't get it to generate behavior that absolutely yeah yeah and and that's like you know you you meet a strange dog or you meet a strange you know like a a a child and you know you can assume certain interactions will fly but others won't um in that sense it definitely works um as trying to relate to something that is also partially unknown but um I think that it's a very very tricky question because nevertheless they are being designed you know they're being made so and that's the first gesture is that deciding of what kind of programming what kind of you know what kind of eyelids do we put on this thing you know so there's a lot of that kind of control as well but what is really hilarious is that there are moments when it's surprising you know like it's it's not behaving in an expected way or anything like that but in any case that can happen with almost any kind of situation um okay yeah um well you talked about are you sort of touched on it briefly but your choice of bre um you know you said it was sort of a joke but actually seems pretty relevant oh yeah you know in terms of I you're interacting with with the robot on the one-onone but then there's like a larger interaction maybe why why I wanted to belittle this whole costume Choice thing is because I feel like any costume that I choose is still so rudimentary in time if that's the right word it's like any costume is referring to something that is that has sort of these societal meanings and I always you can't get rid of that and um it just you know like the researcher has a costume that is referring to um uh medical uh act Auctioneer you know like somebody who's in a hospital but is the administrator uh or the the the car mechanic is again referring to this sort of obsolete sort of activity of Mach masculine machine treatment um there seems to be very little of any dresses that would not have connotations that automatically will bring in questions of power and society as we know it into it so that's why I'm kind of a little joking about it because it feels like I can't get away from that so I have to play it I have to play the game right well it's just that I mean it's sort of real you know robots do sometimes take place of him and then it's been an issue you know it's kind of then what does it mean for the technology that they're making out what how do you mean what is what is it that these ders are going to do um like how will they function you know in relation to society uh you know like um that is a relevant question question and obviously it's in the back of it's kind of part of the platform from where I also started to do what I'm doing but um I I feel like I want to ask more this question of of of mirroring our narcissistic selves because even I I feel that that is even further uh Elemental if you want to use that word like that is playing a huge part in all of that that we're doing in both the the being scared of it as well as wanting to promote it um we there's there are so many different things that are going on at the same time military interests and economical interest um biom medicine and um all these questions that have huge impact on everybody's lives but um and they definitely walk along next to all of the stuff that I'm doing but I feel like at this moment it really uh it's sort of we're still just sort of working in that s same sort of ontological question you know like we we we urge to do these things because we want to um see it happen and and it's sort of um I think it's the the most sort of strangest ridiculous thing that we're stuck on it and uh but it's still working it's still that's the one that's uh in the back of it to to a great extent um is that answering your question yeah I'm wondering if I can kind of reverse the clo a little bit and ask what sense you get of what your subjects might learn from you learning from them whether corcom work yeah you're getting any sense that your subjects are actually learning from you imitating and learning from them um I'm not sure um if I may just a little bit I I think that with um with the copc com it was more clear that um people had some first of all they really like this opportunity to um they they saw me as a different Channel as a as an open door to um Express themselves and talk about themselves in ways that they usually can't do in a in a corporation so to them it was very sort of in instant kind of um coffee break kind of thing and and that was fine but it was not what I needed right so it wasn't it was just a kind of a byproduct of what I was doing that extend to more day today like did they reconsider what they were doing yes that's what I'm thinking that I I obviously I left after that one month but um I did get some sense of this that when they walked into the room and they were able to recognize themselves in my drawings and then they were able to start looking at all the other gestures something interesting might have happened in terms of their um thinking of their own position and their self representation and identification in the corporation potentially I can't prove it I can't say that this is right but I I think that there was an there was an opportunity there um The New York Times piece is a very different question I think that there the the PO most powerful thing was that people were able to process emotion um looking at those images I was a lot of times afraid as still can be but not as badly anymore because I've been doing it for two years now but in the beginning when I was performing in public I was afraid that people might attack me because um I was afraid that somebody might suddenly recognize their own grieving selves in the newspaper being repeated by me I was afraid that they would think that I'm mocking their grief um which is not what I'm doing but I can kind of see that somebody might react that way so I don't know what that's takes anybody um with the U with the robots I I think that maybe some valuable thing can come out of this um exactly this sort of um investigation into you know like putting in a third mirror into this mirroring of each other and and really kind of try and see what is it that we're looking for in these objects you know as as as not as some kind of entity rather than oh it works this way oh it does this and this and this um is this okay the same question I guess the one of the scientists could answer that that again the same exact question one of the scientist one here on the spot uh you know I'd say that one thing is it certainly gave me a better sense of how uh the human body and our actions related to the robot there a lot of design choices that are made that not arbitrary but you don't give a lot of thought to them while you're doing and you're being very functional about it and then when you see a person actually trying to recreate that movement you realize well if I designed it differently maybe be more better fit well thank you when you said U the third miror I think really helpful um strategy to take It's Not Unusual for even even just to help be help with a third person or the third uh can be part of Technology um for years now they have classes in you refer to corporations and and there's class class workation skills I know I and even the fact of having a camera allowed a techology allowed a thing out there this video going of themselves was allowing them to be open to seeing and learning from something out there not the little girl telling this successful guy you know anything it was it was helping dissolve that wall that with extend your your tur you know it's not not a b put the right thing on that it's it's part of our wiring and part of our social Society but when we have this third thing out there like video or for anything especially for controversial uh and learning issues it really can can help pay attention should I should reenact some of those features when they are trying to show you how to emphatically talk to somebody convincingly I mean it's definitely something that um has come up before and um I I feel like I I I would like to think that my work is antithetical to that you know not trying to convince anybody that way but but what to the point what you just said about trying to um control by by using a language and language of gestures um but on the side is the mirror in which in social sciences is noticed as an attempt to communicate to find that common language because we cannot read each other's thoughts and intentions and that's why email turns into blaming so often different interaction with the robot have communication because communication was a robot is still on I would say rudimentary level so the communication is not of imparting something so much it is getting the robot to react the robot is reacting rather than then thinking through what this communication understands communication but point I really ear think work so well um you know I try to get to think about V's Abby Bar's iconology and the way he was dealing with with uh with images and a central a couple Central formula in that one of them is called um light shot dis mentions which is The Treasure of of human passions um and this is and how they're communicated over over the generations repeated and this is when you're deal with a robot I think it's very difficult to to insert this sort of and he sees that as a very communicative structure over Generations over centuries um and the other thing is AOS form and there you might get something with a robot could repe that form forming in any way I know Andrea has been telling me to look at this for the longest time IDE idea actually looking at character of these of the communication in the gesture and how it's repeated in various ways and what what it can be well I think that that that's um maybe one of the things is that um uh to some degree we are familiar with our own repertoire of gestures and the history of that but what is kind of interesting exactly is this this is a robot that first of all we're trying to make it do what we do but then it fails and it does things that are unknown to us and that's when it becomes interesting because it is doing something else to us than exactly a complete mirror and um I think that that's why I'm I might be exactly interested in it um I can look at War Borg's gestures and recognize them I wouldn't find it interesting to repeat them because they're already they're already indexed by him you know what I mean so in that sense I don't I don't necessarily know what process I could try and see what process comes out of that but if you if you understand this is also why I did the New York Times piece the way I did I took them from something that is seen as live you know as something as part of life rather than an an an index and um yeah quality communication I in sense communication robot the question question is about understanding exactly where the where what the communication is where where it's limited not yeah you understand I think yeah I think I do um there's this I mean with those drawings Legion told me that whenever she's interacting with um merch she feels like she has to go down to the basement with it because she finds herself talking to it like a mother talks to an infant she's she's very she becomes very animated and emphatic and there's a lot of emotion coming out because you are trying exactly to manipulate it to behave the way she wants it and she feels vulnerable and she feels that it's an intimate moment and I think that that is exactly one of those things that I think is very interesting because because Mertz is not responding as you would expect it because it is a machine yeah yeah well as it should according to the codes that we wrote into it but because of this sort of failure and difficulty it does bring out even more of the human trying to enact its own humanness in this situation if that's making any sense and that's what I feel about this those drawings over there where where there's a disjuncture in that res that mirroring and it brings out something new something different than I was thinking or that would be sort of expected just in terms of looking at a particular kind of gesture if that's helping but I mean I I totally you know agree with what you're saying and I want to look into it a little more you can talk more about it definitely are we done anybody else I think thank you very much thank [Applause] you [Applause] oh you didn't get to back the question