Frederic Kaplan

How to build an information time machine

1,183,130 views • 10:20
Subtitles in 25 languages
Up next
Details About the talk
Transcript 25 languages

This is an image of the planet Earth. It looks very much like the Apollo pictures that are very well known. There is something different; you can click on it, and if you click on it, you can zoom in on almost any place on the Earth. For instance, this is a bird's-eye view of the EPFL campus. In many cases, you can also see how a building looks from a nearby street. This is pretty amazing. But there's something missing in this wonderful tour: It's time. i'm not really sure when this picture was taken. I'm not even sure it was taken at the same moment as the bird's-eye view. In my lab, we develop tools to travel not only in space but also through time. The kind of question we're asking is Is it possible to build something like Google Maps of the past? Can I add a slider on top of Google Maps and just change the year, seeing how it was 100 years before, 1,000 years before? Is that possible? Can I reconstruct social networks of the past? Can I make a Facebook of the Middle Ages? So, can I build time machines? Maybe we can just say, "No, it's not possible." Or, maybe, we can think of it from an information point of view. This is what I call the information mushroom. Vertically, you have the time. and horizontally, the amount of digital information available. Obviously, in the last 10 years, we have much information. And obviously the more we go in the past, the less information we have. If we want to build something like Google Maps of the past, or Facebook of the past, we need to enlarge this space, we need to make that like a rectangle. How do we do that? One way is digitization. There's a lot of material available — newspaper, printed books, thousands of printed books. I can digitize all these. I can extract information from these. Of course, the more you go in the past, the less information you will have. So, it might not be enough. So, I can do what historians do. I can extrapolate. This is what we call, in computer science, simulation. If I take a log book, I can consider, it's not just a log book of a Venetian captain going to a particular journey. I can consider it is actually a log book which is representative of many journeys of that period. I'm extrapolating. If I have a painting of a facade, I can consider it's not just that particular building, but probably it also shares the same grammar of buildings where we lost any information.


So if we want to construct a time machine, we need two things. We need very large archives, and we need excellent specialists. The Venice Time Machine, the project I'm going to talk to you about, is a joint project between the EPFL and the University of Venice Ca'Foscari.


There's something very peculiar about Venice, that its administration has been very, very bureaucratic. They've been keeping track of everything, almost like Google today. At the Archivio di Stato, you have 80 kilometers of archives documenting every aspect of the life of Venice over more than 1,000 years. You have every boat that goes out, every boat that comes in. You have every change that was made in the city. This is all there. We are setting up a 10-year digitization program which has the objective of transforming this immense archive into a giant information system. The type of objective we want to reach is 450 books a day that can be digitized. Of course, when you digitize, that's not enough, because these documents, most of them are in Latin, in Tuscan, in Venetian dialect, so you need to transcribe them, to translate them in some cases, to index them, and this is obviously not easy. In particular, traditional optical character recognition method that can be used for printed manuscripts, they do not work well on the handwritten document. So the solution is actually to take inspiration from another domain: speech recognition. This is a domain of something that seems impossible, which can actually be done, simply by putting additional constraints. If you have a very good model of a language which is used, if you have a very good model of a document, how well they are structured. And these are administrative documents. They are well structured in many cases. If you divide this huge archive into smaller subsets where a smaller subset actually shares similar features, then there's a chance of success.


If we reach that stage, then there's something else: we can extract from this document events. Actually probably 10 billion events can be extracted from this archive. And this giant information system can be searched in many ways. You can ask questions like, "Who lived in this palazzo in 1323?" "How much cost a sea bream at the Realto market in 1434?" "What was the salary of a glass maker in Murano maybe over a decade?" You can ask even bigger questions because it will be semantically coded. And then what you can do is put that in space, because much of this information is spatial. And from that, you can do things like reconstructing this extraordinary journey of that city that managed to have a sustainable development over a thousand years, managing to have all the time a form of equilibrium with its environment. You can reconstruct that journey, visualize it in many different ways. But of course, you cannot understand Venice if you just look at the city. You have to put it in a larger European context. So the idea is also to document all the things that worked at the European level. We can reconstruct also the journey of the Venetian maritime empire, how it progressively controlled the Adriatic Sea, how it became the most powerful medieval empire of its time, controlling most of the sea routes from the east to the south.


But you can even do other things, because in these maritime routes, there are regular patterns. You can go one step beyond and actually create a simulation system, create a Mediterranean simulator which is capable actually of reconstructing even the information we are missing, which would enable us to have questions you could ask like if you were using a route planner.


"If I am in Corfu in June 1323 and want to go to Constantinople, where can I take a boat?"


Probably we can answer this question with one or two or three days' precision.


"How much will it cost?"


"What are the chance of encountering pirates?"


Of course, you understand, the central scientific challenge of a project like this one is qualifying, quantifying and representing uncertainty and inconsistency at each step of this process. There are errors everywhere, errors in the document, it's the wrong name of the captain, some of the boats never actually took to sea. There are errors in translation, interpretative biases, and on top of that, if you add algorithmic processes, you're going to have errors in recognition, errors in extraction, so you have very, very uncertain data.


So how can we detect and correct these inconsistencies? How can we represent that form of uncertainty? It's difficult. One thing you can do is document each step of the process, not only coding the historical information but what we call the meta-historical information, how is historical knowledge constructed, documenting each step. That will not guarantee that we actually converge toward a single story of Venice, but probably we can actually reconstruct a fully documented potential story of Venice. Maybe there's not a single map. Maybe there are several maps. The system should allow for that, because we have to deal with a new form of uncertainty, which is really new for this type of giant databases.


And how should we communicate this new research to a large audience? Again, Venice is extraordinary for that. With the millions of visitors that come every year, it's actually one of the best places to try to invent the museum of the future. Imagine, horizontally you see the reconstructed map of a given year, and vertically, you see the document that served the reconstruction, paintings, for instance. Imagine an immersive system that permits to go and dive and reconstruct the Venice of a given year, some experience you could share within a group. On the contrary, imagine actually that you start from a document, a Venetian manuscript, and you show, actually, what you can construct out of it, how it is decoded, how the context of that document can be recreated. This is an image from an exhibit which is currently conducted in Geneva with that type of system.


So to conclude, we can say that research in the humanities is about to undergo an evolution which is maybe similar to what happened to life sciences 30 years ago. It's really a question of scale. We see projects which are much beyond any single research team can do, and this is really new for the humanities, which very often take the habit of working in small groups or only with a couple of researchers. When you visit the Archivio di Stato, you feel this is beyond what any single team can do, and that should be a joint and common effort. So what we must do for this paradigm shift is actually foster a new generation of "digital humanists" that are going to be ready for this shift.


I thank you very much.



Imagine if you could surf Facebook ... from the Middle Ages. Well, it may not be as far off as it sounds. In a fun and interesting talk, Frederic Kaplan shows off the Venice Time Machine, a project to digitize 80 kilometers of books to create a historical and geographical simulation of Venice across 1,000 years.

About the speaker
Frederic Kaplan · Digital humanities researcher

Frederic Kaplan seeks to digitize vast archives of historical information to make maps that move — through time.

Frederic Kaplan seeks to digitize vast archives of historical information to make maps that move — through time.