Jonathan Harris: The web as art

So I'm going to talk today about collecting stories in some unconventional ways. This is a picture of me from a very awkward stage in my life. You might enjoy the awkwardly tight, cut-off pajama bottoms with balloons. Anyway, it was a time when I was mainly interested in collecting imaginary stories. So this is a picture of me holding one of the first watercolor paintings I ever made. And recently I've been much more interested in collecting stories from reality -- so, real stories. And specifically, I'm interested in collecting my own stories, stories from the Internet, and then recently, stories from life, which is kind of a new area of work that I've been doing recently. So I'll be talking about each of those today. So, first of all, my own stories. These are two of my sketchbooks. I have many of these books, and I've been keeping them for about the last eight or nine years. They accompany me wherever I go in my life, and I fill them with all sorts of things, records of my lived experience: so watercolor paintings, drawings of what I see, dead flowers, dead insects, pasted ticket stubs, rusting coins, business cards, writings. And in these books, you can find these short, little glimpses of moments and experiences and people that I meet. And, you know, after keeping these books for a number of years, I started to become very interested in collecting not only my own personal artifacts, but also the artifacts of other people. So, I started collecting found objects. This is a photograph I found lying in a gutter in New York City about 10 years ago. On the front, you can see the tattered black-and-white photo of a woman's face, and on the back it says, "To Judy, the girl with the Bill Bailey voice. Have fun in whatever you do." And I really loved this idea of the partial glimpse into somebody's life. As opposed to knowing the whole story, just knowing a little bit of the story, and then letting your own mind fill in the rest. And that idea of a partial glimpse is something that will come back in a lot of the work I'll be showing later today. So, around this time I was studying computer science at Princeton University, and I noticed that it was suddenly possible to collect these sorts of personal artifacts, not just from street corners, but also from the Internet. And that suddenly, people, en masse, were leaving scores and scores of digital footprints online that told stories of their private lives. Blog posts, photographs, thoughts, feelings, opinions, all of these things were being expressed by people online, and leaving behind trails. So, I started to write computer programs that study very, very large sets of these online footprints. One such project is about a year and a half old. It's called "We Feel Fine." This is a project that scans the world's newly posted blog entries every two or three minutes, searching for occurrences of the phrases "I feel" and "I am feeling." And when it finds one of those phrases, it grabs the full sentence up to the period and also tries to identify demographic information about the author. So, their gender, their age, their geographic location and what the weather conditions were like when they wrote that sentence. It collects about 20,000 such sentences a day and it's been running for about a year and a half, having collected over 10 and a half million feelings now. This is, then, how they're presented. These dots here represent some of the English-speaking world's feelings from the last few hours, each dot being a single sentence stated by a single blogger. And the color of each dot corresponds to the type of feeling inside, so the bright ones are happy, and the dark ones are sad. And the diameter of each dot corresponds to the length of the sentence inside. So the small ones are short, and the bigger ones are longer. "I feel fine with the body I'm in, there'll be no easy excuse for why I still feel uncomfortable being close to my boyfriend," from a twenty-two-year-old in Japan. "I got this on some trading locally, but really don't feel like screwing with wiring and crap." Also, some of the feelings contain photographs in the blog posts. And when that happens, these montage compositions are automatically created, which consist of the sentence and images being combined. And any of these can be opened up to reveal the sentence inside. "I feel good." "I feel rough now, and I probably gained 100,000 pounds, but it was worth it." "I love how they were able to preserve most in everything that makes you feel close to nature -- butterflies, man-made forests, limestone caves and hey, even a huge python." So the next movement is called mobs. This provides a slightly more statistical look at things. This is showing the world's most common feelings overall right now, dominated by better, then bad, then good, then guilty, and so on. Weather causes the feelings to assume the physical traits of the weather they represent. So the sunny ones swirl around, the cloudy ones float along, the rainy ones fall down, and the snowy ones flutter to the ground. You can also stop a raindrop and open the feeling inside. Finally, location causes the feelings to move to their spots on a world map, giving you a sense of their geographic distribution. So I'll show you now some of my favorite montages from "We Feel Fine." These are the images that are automatically constructed. "I feel like I'm diagonally parked in a parallel universe." (Laughter) "I've kissed numerous other boys and it hasn't felt good, the kisses felt messy and wrong, but kissing Lucas feels beautiful and almost spiritual." "I can feel my cancer grow." "I feel pretty." "I feel skinny, but I'm not." "I'm 23, and a recovering meth and heroin addict, and feel absolutely blessed to still be alive." "I can't wait to see them racing for the first time at Daytona next month, because I feel the need for speed." (Laughter) "I feel sassy." "I feel so sexy in this new wig." As you can see, "We Feel Fine" collects very, very small-scale personal stories. Sometimes, stories as short as two or three words. So, really even challenging the notion of what can be considered a story. And recently, I've become interested in diving much more deeply into a single story. And that's led me to doing some work with the physical world, not with the Internet, and only using the Internet at the very last moment, as a presentation medium. So these are some newer projects that actually aren't even launched publicly yet. The first such one is called "The Whale Hunt." Last May, I spent nine days living up in Barrow, Alaska, the northernmost settlement in the United States, with a family of Inupiat Eskimos, documenting their annual spring whale hunt. This is the whaling camp here, we're about six miles from shore, camping on five and a half feet of thick, frozen pack ice. And that water that you see there is the open lead, and through that lead, bowhead whales migrate north each springtime. And the Eskimo community basically camps out on the edge of the ice here, waits for a whale to come close enough to attack. And when it does, it throws a harpoon at it, and then hauls the whale up under the ice, and cuts it up. And that would provide the community's food supply for a long time. So I went up there, and I lived with these guys out in their whaling camp here, and photographed the entire experience, beginning with the taxi ride to Newark airport in New York, and ending with the butchering of the second whale, seven and a half days later. I photographed that entire experience at five-minute intervals. So every five minutes, I took a photograph. When I was awake, with the camera around my neck. When I was sleeping, with a tripod and a timer. And then in moments of high adrenaline, like when something exciting was happening, I would up that photographic frequency to as many as 37 photographs in five minutes. So what this created was a photographic heartbeat that sped up and slowed down, more or less matching the changing pace of my own heartbeat. That was the first concept here. The second concept was to use this experience to think about the fundamental components of any story. What are the things that make up a story? So, stories have characters. Stories have concepts. Stories take place in a certain area. They have contexts. They have colors. What do they look like? They have time. When did it take place? Dates -- when did it occur? And in the case of the whale hunt, also this idea of an excitement level. The thing about stories, though, in most of the existing mediums that we're accustomed to -- things like novels, radio, photographs, movies, even lectures like this one -- we're very accustomed to this idea of the narrator or the camera position, some kind of omniscient, external body through whose eyes you see the story. We're very used to this. But if you think about real life, it's not like that at all. I mean, in real life, things are much more nuanced and complex, and there's all of these overlapping stories intersecting and touching each other. And so I thought it would be interesting to build a framework to surface those types of stories. So, in the case of "The Whale Hunt," how could we extract something like the story of Simeon and Crawford, involving the concepts of wildlife, tools and blood, taking place on the Arctic Ocean, dominated by the color red, happening around 10 a.m. on May 3, with an excitement level of high? So, how to extract this order of narrative from this larger story? I built a web interface for viewing "The Whale Hunt" that attempts to do just this. So these are all 3,214 pictures taken up there. This is my studio in Brooklyn. This is the Arctic Ocean, and the butchering of the second whale, seven days later. You can start to see some of the story here, told by color. So this red strip signifies the color of the wallpaper in the basement apartment where I was staying. And things go white as we move out onto the Arctic Ocean. Introduction of red down here, when whales are being cut up. You can see a timeline, showing you the exciting moments throughout the story. These are organized chronologically. Wheel provides a slightly more playful version of the same, so these are also all the photographs organized chronologically. And any of these can be clicked, and then the narrative is entered at that position. So here I am sleeping on the airplane heading up to Alaska. That's "Moby Dick." This is the food we ate. This is in the Patkotak's family living room in their house in Barrow. The boxed wine they served us. Cigarette break outside -- I don't smoke. This is a really exciting sequence of me sleeping. This is out at whale camp, on the Arctic Ocean. This graph that I'm clicking down here is meant to be reminiscent of a medical heartbeat graph, showing the exciting moments of adrenaline. This is the ice starting to freeze over. The snow fence they built. And so what I'll show you now is the ability to pull out sub-stories. So, here you see the cast. These are all of the people in "The Whale Hunt" and the two whales that were killed down here. And we could do something as arbitrary as, say, extract the story of Rony, involving the concepts of blood and whales and tools, taking place on the Arctic Ocean, at Ahkivgaq camp, with the heartbeat level of fast. And now we've whittled down that whole story to just 29 matching photographs, and then we can enter the narrative at that position. And you can see Rony cutting up the whale here. These whales are about 40 feet long, and weighing over 40 tons. And they provide the food source for the community for much of the year. Skipping ahead a bit more here, this is Rony on the whale carcass. They use no chainsaws or anything; it's entirely just blades, and an incredibly efficient process. This is the guys on the rope, pulling open the carcass. This is the muktuk, or the blubber, all lined up for community distribution. It's baleen. Moving on. So what I'm going to tell you about next is a very new thing. It's not even a project yet. So, just yesterday, I flew in here from Singapore, and before that, I was spending two weeks in Bhutan, the small Himalayan kingdom nestled between Tibet and India. And I was doing a project there about happiness, interviewing a lot of local people. So Bhutan has this really wacky thing where they base most of their high-level governmental decisions around the concept of gross national happiness instead of gross domestic product, and they've been doing this since the '70s. And it leads to just a completely different value system. It's an incredibly non-materialistic culture, where people don't have a lot, but they're incredibly happy. So I went around and I talked to people about some of these ideas. So, I did a number of things. I asked people a number of set questions, and took a number of set photographs, and interviewed them with audio, and also took pictures. I would start by asking people to rate their happiness between one and 10, which is kind of inherently absurd. And then when they answered, I would inflate that number of balloons and give them that number of balloons to hold. So, you have some really happy person holding 10 balloons, and some really sad soul holding one balloon. But you know, even holding one balloon is like, kind of happy. (Laughter) And then I would ask them a number of questions like what was the happiest day in their life, what makes them happy. And then finally, I would ask them to make a wish. And when they made a wish, I would write their wish onto one of the balloons and take a picture of them holding it. So I'm going to show you now just a few brief snippets of some of the interviews that I did, some of the people I spoke with. This is an 11-year-old student. He was playing cops and robbers with his friends, running around town, and they all had plastic toy guns. His wish was to become a police officer. He was getting started early. Those were his hands. I took pictures of everybody's hands, because I think you can often tell a lot about somebody from how their hands look. I took a portrait of everybody, and asked everybody to make a funny face. A 17-year-old student. Her wish was to have been born a boy. She thinks that women have a pretty tough go of things in Bhutan, and it's a lot easier if you're a boy. A 28-year-old cell phone shop owner. If you knew what Paro looked like, you'd understand how amazing it is that there's a cell phone shop there. He wanted to help poor people. A 53-year-old farmer. She was chaffing wheat, and that pile of wheat behind her had taken her about a week to make. She wanted to keep farming until she dies. You can really start to see the stories told by the hands here. She was wearing this silver ring that had the word "love" engraved on it, and she'd found it in the road somewhere. A 16-year-old quarry worker. This guy was breaking rocks with a hammer in the hot sunlight, but he just wanted to spend his life as a farmer. A 21-year-old monk. He was very happy. He wanted to live a long life at the monastery. He had this amazing series of hairs growing out of a mole on the left side of his face, which I'm told is very good luck. He was kind of too shy to make a funny face. A 16-year-old student. She wanted to become an independent woman. I asked her about that, and she said she meant that she doesn't want to be married, because, in her opinion, when you get married in Bhutan as a woman, your chances to live an independent life kind of end, and so she had no interest in that. A 24-year-old truck driver. There are these terrifyingly huge Indian trucks that come careening around one-lane roads with two-lane traffic, with 3,000-foot drop-offs right next to the road, and he was driving one of these trucks. But all he wanted was to just live a comfortable life, like other people. A 24-year-old road sweeper. I caught her on her lunch break. She'd built a little fire to keep warm, right next to the road. Her wish was to marry someone with a car. She wanted a change in her life. She lives in a little worker's camp right next to the road, and she wanted a different lot on things. An 81-year-old itinerant farmer. I saw this guy on the side of the road, and he actually doesn't have a home. He travels from farm to farm each day trying to find work, and then he tries to sleep at whatever farm he gets work at. So his wish was to come with me, so that he had somewhere to live. He had this amazing knife that he pulled out of his gho and started brandishing when I asked him to make a funny face. It was all good-natured. A 10-year-old. He wanted to join a school and learn to read, but his parents didn't have enough money to send him to school. He was eating this orange, sugary candy that he kept dipping his fingers into, and since there was so much saliva on his hands, this orange paste started to form on his palms. (Laughter) A 37-year-old road worker. One of the more touchy political subjects in Bhutan is the use of Indian cheap labor that they import from India to build the roads, and then they send these people home once the roads are built. So these guys were in a worker's gang mixing up asphalt one morning on the side of the highway. His wish was to make some money and open a store. A 75-year-old farmer. She was selling oranges on the side of the road. I asked her about her wish, and she said, "You know, maybe I'll live, maybe I'll die, but I don't have a wish." She was chewing betel nut, which caused her teeth over the years to turn very red. Finally, this is a 26-year-old nun I spoke to. Her wish was to make a pilgrimage to Tibet. I asked her how long she planned to live in the nunnery and she said, "Well, you know, of course, it's impermanent, but my plan is to live here until I'm 30, and then enter a hermitage." And I said, "You mean, like a cave?" And she said, "Yeah, like a cave." And I said, "Wow, and how long will you live in the cave?" And she said, "Well, you know, I think I'd kind of like to live my whole life in the cave." I just thought that was amazing. I mean, she spoke in a way -- with amazing English, and amazing humor, and amazing laughter -- that made her seem like somebody I could have bumped into on the streets of New York, or in Vermont, where I'm from. But here she had been living in a nunnery for the last seven years. I asked her a little bit more about the cave and what she planned would happen once she went there, you know. What if she saw the truth after just one year, what would she do for the next 35 years in her life? And this is what she said. Woman: I think I'm going to stay for 35. Maybe -- maybe I'll die. Jonathan Harris: Maybe you'll die? Woman: Yes. JH: 10 years? Woman: Yes, yes. JH: 10 years, that's a long time. Woman: Yes, not maybe one, 10 years, maybe I can die within one year, or something like that. JH: Are you hoping to? Woman: Ah, because you know, it's impermanent. JH: Yeah, but -- yeah, OK. Do you hope -- would you prefer to live in the cave for 40 years, or to live for one year? Woman: But I prefer for maybe 40 to 50. JH: 40 to 50? Yeah. Woman: Yes. From then, I'm going to the heaven. JH: Well, I wish you the best of luck with it. Woman: Thank you. JH: I hope it's everything that you hope it will be. So thank you again, so much. Woman: You're most welcome. JH: So if you caught that, she said she hoped to die when she was around 40. That was enough life for her. So, the last thing we did, very quickly, is I took all those wish balloons -- there were 117 interviews, 117 wishes -- and I brought them up to a place called Dochula, which is a mountain pass in Bhutan, at 10,300 feet, one of the more sacred places in Bhutan. And up there, there are thousands of prayer flags that people have spread out over the years. And we re-inflated all of the balloons, put them up on a string, and hung them up there among the prayer flags. And they're actually still flying up there today. So if any of you have any Bhutan travel plans in the near future, you can go check these out. Here are some images from that. We said a Buddhist prayer so that all these wishes could come true. You can start to see some familiar balloons here. "To make some money and to open a store" was the Indian road worker. Thanks very much. (Applause)