Jim Blinn, Science Educator and Founding Pioneer of Computer Graphics

In our modern media era, computer-generated graphics are so ubiquitous, so photorealistic, so seamlessly intertwined across our cultural landscape, that it can be easy to forget that this is a relatively new phenomenon. Its prevalence begs some obvious questions: What prompted the need for programmers to turn computers into an artistic tool? What technologies needed to be improved upon - or invented - to make that happen? What benefits has this technology conferred, and to whom?

In the discussion below, readers are invited to delve into the recollections and insights of Jim Blinn, who, as much as anyone in the world, brought computer graphics to life. As an undergraduate in the late 1960s at the University of Michigan, Blinn was pushing the comparatively primitive capabilities of computers to their limits, and as a graduate student at the University of Utah, he was focused on texture mapping, which is the lifeblood of computer graphics. Blinn was naturally attracted to JPL to continue this work, and as luck would have it, he arrived in time for the Voyager Mission. He created graphical visualizations of what the Voyager Spacecraft were seeing - or were anticipated to see - and in doing so, Blinn played a central role in the global excitement of the outer planetary encounters, which rivaled the moon landing as the seminal moment in the early history of space exploration.

When asked if he is ultimately a physicist, a computer scientist, or an artist (the answer is yes), Blinn emphasizes that he is an educator. This is the source of his inspiration, and it is why he happily accepted the opportunity to partner with the late Caltech professor of physics David Goodstein, to create The Mechanical Universe, a landmark television series that, over 52 episodes, covered the entirety of physics from the classical mechanics from the Age of Newton to the bizarre subatomic world of quantum physics. Hailed as a smash success in science education, Blinn's animations made physics concepts both fun and accessible to audiences within and well beyond Caltech. Other career highlights include partnerships with Carl Sagan on the Cosmos series, helping to bring computer animation to Hollywood with George Lucas, and utilizing the resources and reputation of Microsoft to bring computer graphics into the age of high speed internet, vast computer memory, and exponentially growing processing speeds.

Readers are encouraged to visit Blinn's official web page where they can delve into a world of his computations, his art, and his writing. It is delightfully understated in its presentation of computer graphics.

DAVID ZIERLER: This is David Zierler, Director of the Caltech Heritage Project. It's Monday, May 27th, 2024. It's my great pleasure to be here with Dr. James Blinn. Jim, it's wonderful to be with you. Thank you so much for joining me.

JAMES BLINN: Thanks for inviting me.

ZIERLER: Jim, to start, would you please tell me your current or most recent title and institutional affiliation?

BLINN: I was at Microsoft for about 15 years. The official title was Graphics Fellow.

ZIERLER: Now you're retired?

BLINN: Yeah.

ZIERLER: What kinds of things are you doing in retirement? Are you consulting? Are you producing movies currently?

BLINN: Mostly what I'm doing is some more mathematical writing. There's several articles that I have written over the years on the mathematics of computer graphics. Some of them have been reproduced in books, and some of them have not. The ones that have not, I've been buffing up, and I plan on putting them up on my website to publicize them. They're mostly educational rather than research in new mathematics. It's ways of doing things that relates to computer graphics though.

ZIERLER: Jim, some overall questions about your work and career. First, I wonder if you could explain just some computer history? When did computers become advanced enough to use them as tools for animation? How did that happen?

BLINN: I think it depends on what you consider quality animation, I guess. Computers have almost always been used to make pictures that have been used in animation. Very early ones before my time did some physics simulations of the ball bouncing across the screen, just to show the dynamics of it. But I happened to have been lucky enough so that when I got into the business, computers were just getting to the point where they could make really interesting looking pictures. I was able to do that from the beginning of my career, various levels of quality of image, but it was good enough to show what I wanted. My primary interest has been in educational animation. Showing simulations of various physical processes being animated on the screen was a good way of getting a physical intuition about these things.

ZIERLER: What do computers do better as an educational tool to teach physical ideas than more traditional styles of animation?

BLINN: A lot of it is the interaction end of things, which is something I hadn't gotten into too much but nowadays is very popular. There's a lot of educational physics websites you can subscribe to that I have just observed, but I haven't played around with very much, but the idea that you can modify the parameters to something, and see how it works, giving you some physical intuition about what the effects of a physical model is.

ZIERLER: I wonder if you can tell me about the Phong reflection model, and how you modified it.

BLINN: Phong was a graduate student at the University of Utah, and he was the next level of improvement over image quality research there. The idea was to simulate not just diffuse reflection but specular reflection, basically highlights on objects. He crafted a function that simulated how much the highlight would be at each spot on a curved surface based on just models of how more light gets reflected in the direction makes and equal angle to the surface normal that the light direction does. But it falls off a bit from that, so it was somewhat made up to have the properties that he wanted it to show rather than a physical model of exactly what was going on on the surface. When I got to Utah, he had just graduated and left, so I actually never met him. But some of the first things that I did was to implement a rendering algorithm for doing curved surfaces. That was a modification of one that Ed Catmull had done. Curved surfaces are really good at displaying highlights.

I actually went through Phong's thesis, and was a bit confused by the math that he did. I figured out a slightly different model that made sense to me in terms of how to figure out how much specular light the object reflected, which I implemented. So I built on Phongs basic idea but did a slightly different physical model. What is interesting is that, later on, when I did some library research at Utah, poking around at physicists and illumination engineering journals, and how they had modeled light to figure out reflections, the one model that I found that made sense to me turned out to be the same mathematics that I had used. My modification of Phong's model turned out to have a physical justification based on a model of how the microstructure of the surface is made up a lot of little facets pointing in different directions, and statistically how many pointed in the proper direction to show you the highlight. I lucked out. I came up with a model that had a physical justification that's been the basis of specularity that other people then took even further, and made more and more accurate models for different types of surfaces. I was a link in the chain between Phong and people like Ken Torrance and Rob Cook at Cornell, who did more accurate surface simulation models than I did.

The Law of Rendering

ZIERLER: Jim, I wonder if you can explain to me the basis for what we call Blinn's Law, and how that relates to your ideas—

BLINN: [laugh]

ZIERLER: —about rendering over time.

BLINN: When you're trying to do animation, the computers at the time were not fast enough to do it in real time, but you could get a reasonable image up there in 5 to 10 minutes, maybe 15 minutes. For animation purposes, you'd have to do this over and over again, and record the images one at a time. As computers got faster, you could get the same image for a much less amount of time. But what wound up happening was that you decided to make a more complicated image. As the computers got faster, it was a matter of how much patience you have waiting for the image with how long it took, basically. If it took 5 or 10 minutes, and it was a crummy image, that was good enough for you. But if you had 5 or 10 minutes to make a better image, that's what you did. The law is, basically, even as computers get faster, the rendering time stays the same because you keep putting more and more complexity and more and more detail in the image. It's more a matter of measure of your patience and how much you want to wait, and 5 to 10 minutes is a good number. It ranges from 5 or 10 minutes to 2 or 3 hours maybe, but in that range. [laugh]

ZIERLER: From the first time that you formulated Blinn's Law, has that more or less played out to today? Do we see rendering just get better and better and better, and artists willing to spend the same amount of time to create better images?

BLINN: More or less, it's even got worse, in a sense, because the images that I made at JPL, the Voyager flyby movies took between 5 and 20 minutes a frame, something like that. But if you look at the images in Pixar movies, the artists in there just go nuts with just stuff all over the place, with hair and so forth. Some of those images take a couple of hours to do a frame. But now they have render farms of thousands of computers, I don't know if it's in the thousands, but certainly hundreds of computers all working at once, each on a different frame. The computer power has improved in order to be able to accommodate that. But I did talk with somebody at JPL a little while ago, and they said that when they were doing animation these days, it took about the same amount of time to do their frames as it took me to do mine.

AI and Photorealism

ZIERLER: Where is all this heading? Is the ultimate goal to make computer animation almost photorealistic or even better than reality, whatever that might mean?

BLINN: I think we've kind of arrived at that. Look at Pixar movies, and they look, you know, most of them are cartoony in the sense that you're not going to mistake them from a real thing. But certainly a lot of stuff that you see in computer animation nowadays looks pretty photorealistic. Images for TV commercials and so forth, a lot of those—as I understand it, anyway—most car commercials, the car is actually a computer simulation rather than a real car when it's driving around. In special effects movies in Hollywood, the photorealistic special effects, they look as real as you can imagine I think.

ZIERLER: Jim, where do you put artificial intelligence along this continuum? Is it simply a continuation of what human animators have done, or is it something different?

BLINN: I'm not a big expert in artificial intelligence, but I think that it's maybe somewhat orthogonal to computer graphics in the sense that what I have seen is mostly the language end of things, answering questions, and researching data, getting either true or ridiculously false information [laugh] that you spit back of people. But there is some artificial intelligence done in computer graphics, and they can use it to improve the images of pictures, understanding what pictures are supposed to look like. They can adjust maybe a somewhat flaky image, make it look more realistic, based on some models of how the real world works.

ZIERLER: Jim, I'm curious to ask, either based on your education and training, the projects that you've worked on, your sensibilities, how you see the world, at the end of the day, are you a scientist, an engineer, a mathematician, or an artist?

BLINN: I prefer to call myself an educator. My parents were both teachers, and I grew up watching educational films on TV, and I figured that's what I wanted to do. I saw the Bell Telephone Science Series that were produced when I was a kid. The happiest moment in my life was the fact that I got to work on Mechanical Universe, which was like the next generation of that. It was a physics course that I was able to do animation for to help teach the concepts. It was a college-level course. The ones I'd seen when I was a kid were high school level. I was able to build on that and make the next generation of educational animations, and hopefully inspire more people.

ZIERLER: Jim, you mentioned your parents. Let's go back and establish some personal history. Where did you grow up?

BLINN: I grew up originally in two small towns in the middle of Michigan: one called Belding, and one called Greenville. I went to high school there. Population around 7,000, something like that. Television was the main information source for doing things at that time. Then I went to the University of Michigan for undergraduate and some graduate school. That's where I got introduced to computers.

ZIERLER: Jim, as a—

BLINN: I originally went there to be a physics major, and I wound up with a double-major in Physics and what they called Communication Science. They didn't even have Computer Science invented by then, but it was basically doing computers.

ZIERLER: As a student in Ann Arbor in the late 1960s, were you political at all? Were you involved in any movements?

BLINN: Not really, no. Mostly I was so taken with computers, I was playing in the computer lab all the time. I observed the politics from afar but didn't get involved in it really myself.

ZIERLER: Was the draft something you needed to deal with?

BLINN: Yeah. I had a draft number of 57, I think, so I was on the edge of getting drafted. As a freshman, I enrolled in ROTC because I figured if I was going to be drafted, it'd be better to go in as an officer than an enlisted person. But I learned that I would be really terrible an officer [laugh], and so I didn't pursue that. But basically the reason I didn't get drafted was I was too skinny. I grew up all my life very tall and very thin. When I went in for my draft physical, I did not meet the height-weight ratio necessary to be drafted, so that paid off. [laugh]

ZIERLER: Jim, did you pay attention to the Apollo program? Were you captivated by space exploration?

BLINN: Oh yeah, I certainly paid attention to it. Though I was more taken by the unmanned side of things. But I remember watching the first moon landing. I didn't have a TV set myself, but one of my college roommates had a girlfriend who had a TV, so we went over to her house to watch it live as Neil Armstrong steped out on the Moon. Then we both ran outside, and just looked up at the Moon, and said, "There's people standing on there. That's pretty amazing."

Art and Computers in the 1960s

ZIERLER: [laugh] Jim, tell me the story about your time in 1968 when you first figured out how to make art, pictures with a computer.

BLINN: I've told this story before. The reason I got into computers is because I took French in high school. This let me place out of the foreign language requirements, so I had empty spots in my schedule to take computer courses as an undergraduate. The first course I took was very simple programming, and mostly doing mathematical sorts of things. That summer, I got a job with the physics department, scanning and measuring bubble chamber photographs. On the floor above me, there was somebody who had a computer display, and I went and watched a little bit. He played Spacewar! on it. But the second summer, between my sophomore and junior year, I got a job with the engineering school, working on a computer project that was doing research in what they called conversational uses of computers, which is now known as interactive, instead of using punch cards. It had a computer graphics display that was pretty cool. That wasn't the part that I was originally working on, but I was looking in the window of the display room all the time, saying, "Hey, I wish I could get to work with it." By the end of the summer, I was able to talk my way into being the assistant of the graduate student, Jim Jackson, who was working on that, and so that was my first introduction. He and I used to do lots of other experiments. We did some early 3D rotations, line drawing rotations that our very slow computer could do. Just few lines but it could rotate them in real time. But then at the end of that summer, all the graduate students finished their stuff, and graduated, and left. I was a junior, and now I had this computer as basically my own personal toy. [laugh] Everybody had left, and I had the computer to play with for the next several years. I did a lot of computer simulations with Dr. Jens Zorn in the physics department. We did some computer simulations of a proton spin in a magnetic field, that sort of thing. That served as my introduction to doing interactive physics simulations.

ZIERLER: Jim, what was the computer you did all of this work on?

BLINN: It was made by Digital Equipment Corporation. It was called a PDP-9, and it was the size of a bunk bed, and cost a couple hundred thousand dollars at that time. It had a CRT display that was about 12 inches across, which just made black and white line drawings. It was incredibly slow and had incredibly small amounts of memory by today's standards. But it was enough that I could have a lot of fun with.

ZIERLER: Did it use punch cards?

BLINN: No, it used paper tape, punched paper tape. It was also connected to the main university's computer. I could type programs into the main computer, and then have it punch out the tape that I would read into the PDP-9 to run the programs.

ZIERLER: Jim, were you aware that there was a broader if small community of computer animation even at this early juncture?

BLINN: Yeah. I saw things from various places. I went to any computer conference I could find that talked about computers. They had computer graphic sessions. I pretty much kept up with what was possible, what was being done at the time. Various things happened. John Whitney was a computer artist, and there was a show of some of his films that I saw. Jim Jackson, my graduate student colleague, said, "Hey, we can do that too." We went back, and tried to do that for our films on the computer, on the PDP-9, which looked terrible. [laugh] We learned that you had to be a good artist as well as a good programmer in order to do something interesting. [laugh]

ZIERLER: Jim, tell me about the master's program in computer information and control engineering at Michigan.

BLINN: Like I say, in the early days of the computer, the subject was just becoming respectable, in a sense. I was in the department in the engineering school, working on this computer graphics display. I became a teaching assistant for the engineering computer programming classes. At the end of my undergraduate career, I decided to stick around and sign up for graduate school. They had this new computer department called Computer Information and Control Engineering, which was a combination of computers and other stuff. I had to take courses in some things that were not so computer-related. But I was able to stick around and get a master's degree from Michigan in that.

ZIERLER: Jim, were you following all of the exciting developments in computers and chips at this point? Were you following the early days of Hewlett-Packard and Fairchild Semiconductor? Were these relevant for you?

BLINN: Yeah, sort of. It was in the days when information was not so readily available by googling things. But the various courses that were being taught were done by people who were following it. One course I took was programming an Intel 8080—no, it was 8008, I think—chips, very early single-chip computers. I followed that by the other courses that were taught by the university, and also just as much reading as I could find. I had an office in the engineering building with various other graduate students. One of the graduate students came and showed me a thesis that he had just received from the University of Utah, doing computer graphics. That was the first time I'd heard of what Utah had been doing. I sent off and signed up to get a copy of all the publications that they had done on computer graphics, and studied up on that. That's where I went with that.

From Ann Arbor to Utah

ZIERLER: Now, when you wrapped up in Michigan in 1972, did you move immediately to Utah, or was there a break in between?

BLINN: A couple months' break. I had been working at the U of M computing center. I basically was at Michigan for four years as an undergraduate, and then two years as a graduate student, then stuck around for two years beyond that, just as an employee of the computing center, helping keep the graphic stuff going there for the rest of the campus, and finally decided to go to Utah. I had a three-month break, where I just toured the West Coast, and visited a few friends, and visited some national parks. I also went to visit JPL, but they didn't have any visitor center open. So I just looked in the windows. I had heard about JPL before. There was a guy at Michigan named Jim Loudon, who did astronomy lectures for the museum there, the planetarium. He gave lectures about the advances going on at Caltech and JPL and the Mars Lander. I thought at the time, boy, JPL would be a cool place to work, because I was quite interested in astronomy all my life too. But I don't have a degree in astronomy, so I'm not sure what use they would make of me, so I put that in the back of my mind. During my vacation between Michigan and Utah, I went to California, and went to Disneyland for the first time, and went to look in the windows of JPL, and say, "Yeah, the place does exist." Then I basically went off to Utah, and did my thing there.

ZIERLER: Jim, some institutional history, how did the University of Utah come to be a leader in computer animation?

BLINN: It was a combination of the two professors there, David Evans and Ivan Sutherland, who became partners in starting the department. David Evans was at Berkeley for a while, but he went back to Utah since he was in the Mormon church, and Utah was the seat of Mormonism. He went back to Utah to start the computer science program going there. Ivan Sutherland was somebody he'd worked with, and so he went to join there as a faculty member there. They started the computer science department at Utah, and got research money from ARPA to develop computer-rendering techniques.

ZIERLER: What was ARPA's interest in working with these faculty?

BLINN: I'm not sure what ARPA was thinking of getting out of it as far as defense goes, although they did do a lot of things for training military pilots, early flight trainers, flight simulator training, and so some of the early things of having a visual display, simulating what the pilot would see in a simulated cockpit. That's one of the main directions that they were going early on.

ZIERLER: What was your plan? What did you want to do for your thesis when you got to Utah?

BLINN: I didn't have a specific goal in mind, so I just started with the results of the last few people there. I wanted to do graphics, of course, which is what everybody was interested in there—well, not everybody, but most people were interested in there. I got there at the ideal time because the equipment that they had was mostly a PDP-10, a time-sharing system. The display equipment they had was just a tube, where they had to expose the image on film, and then develop the film before they got a chance to see the picture. When I got there, they were just finishing getting a PDP-11 with a new 3D line drawing display from the Evans and Sutherland computer department that was just installed. They were just beginning to build the first framebuffer, which is a computer memory that's tied into a TV. You'll stick the numbers in the memory of the framebuffer, and then it gets displayed instantly on the TV. You get the chance to see it right away. I basically wrote the first program on that to put a pixel on the screen. [laugh] All the research that they had done in computer graphics before then was slowed down by the fact that it was such a laborious process, just getting a picture out. Once we got this framebuffer in there, you could have a turnaround time of trying an idea to see whether it worked within minutes instead of days. I benefited from being there at exactly the right time to have all the coolest computer graphics equipment available to do the stuff that I did.

ZIERLER: I wonder if you can explain some of the advances, even in the short amount of time from when you were an undergraduate to a graduate student at the University of Utah. Between advances in computer graphics, memory, and software, how did all of that come together to do what you were able to do?

BLINN: Computer graphics started out being just line drawings. Either physically, they had physical plotters. You'd drive a pen around on the paper using stepper motors, Or you could display simple black and white line drawings on the screen. When I got to Utah, being able to do color images, that was some of the first things that could be done there. I wasn't the only person doing that; there were other people around the country doing it with their equipment too. One group was at the New York Institute of Technology, which started up about then. They bought the same kind of equipment that we had at Utah, and hired a bunch of Utah people there. I spent a summer there as an intern. They were interested in doing animation for making movies. A lot of my friends were there, and we traded ideas. But I went back to Utah to finish my degree because I knew I needed to get a degree. [laugh] I went to JPL from there.

The Origins of Texture Mapping

ZIERLER: Jim, was your interest in education, was that baked into your thesis project at Utah? Were you trying to do something that did have educational value for the dissertation?

BLINN: Not specifically educational value. It was mostly the next step in realism, in computer graphics, because there were people making faceted models, basically polygonal models. Then they tried to do things that were approximations to curved surfaces, and ways of making them look more smooth, and then ways of making the lighting better. I just piggybacked on that. The next step was just how can I make this picture look better? Texture mapping was just getting started, and so I made some improvements on that.

ZIERLER: What is texture mapping?

BLINN: The idea being that you have a surface that you're displaying. Instead of making it one solid color, you use another image to paint on the surface, and distort the image to fit the shape of the of the surface. It's something that's done so often now, people don't even realize it. Then once you get the idea that you can change the colors of the surface from one spot to the next, you can put patterns on there and whatnot. Then you can start saying, well, what other properties of the surface can we change: the shiniest from one spot to the next, or can we arrange it so that it makes the surface look bumpy by making the surface have wrinkles in it from one spot to the next, which reflect light in slightly different directions? I just did incremental advances to that. How can I make this look cooler, and how can I make these elements look more realistic and less obviously artificial?

ZIERLER: Jim, what would you say the principal conclusions or contributions of your thesis was?

BLINN: It was mostly in rendering and texture mapping. There were three kind of main things that were in my thesis. One of them was an algorithm for rendering the surfaces themselves, which was involved in taking the mathematical description of the shape, and then scanning it out, one pixel at a time, scanning across the screen, which was a mathematical process, and basically inverting two bivariate functions numerically. It was an interesting exercise but one that is completely obsolete nowadays. The other two things were bump mapping, which is a way of making the service look bumpy. Then the third one was the more realistic, light reflection models, which I mentioned I started with Phong's model, and made some improvements with that. Then I went through the library at Utah, thinking there must be physicists who've been studying how light really reflects off of the surfaces. Let me see if I can find out what they can tell us. I spent days just wandering around the engineering library, looking at the spines of the books about, you know, the journal of this, and the society of that, and just found something from the Journal of the Illumination Engineering Society and said, "Oh, this looks great." I poured through that, and I found some actual physical measurements that somebody had made about how much light gets reflected in each direction from the surface. I thought I would maybe digitize those, and use those to feed it. But I kept looking around, and I finally came across a paper from University of Minnesota by two guys named Torrance and Sparrow that had a physical model of the microfacets that I mentioned before, and a statistical model of how much light gets reflected in each direction off the surface. I said, "Wow, this is the jackpot." I could actually understand the math. They were doing the analytics side of it. They wanted to analyze how the surfaces did it, so I had to turn the math around for the synthetic side, saying, "Suppose we had the surface, then how would it reflect light?" The third facet of my thesis was adapting the Torrance-Sparrow light reflection model to be something that's a blend, so to speak, of the light reflection model, and how specularity works on surfaces.

What was interesting about that, which is when I graduated and published the thesis, and then in the computer graphics conferences. Cornell University of had a computer graphics program too. Don Greenberg was the main professor there. One of his graduate students Rob Cook read that, and he got interested in pursuing that even further, getting more and more accurate surface reflections. Don Greenberg kind of said, "Boy, Ken Torrance, that name sounds familiar to me." It turns out that the guy who wrote that paper was now another professor at Cornell. Rob went to visit him, and said, "Hey, we're using your paper and surface reflection models, and can you teach me more about this?" Torrance ultimately got interested in that himself, and moved over to the computer science department, and had a career in computer image synthesis as a result of this multi-step process of my stumbling across his analytical paper, and telling the Cornell people about it, and him getting roped into the subject himself.

ZIERLER: Jim, what were the possible applications of this? Were you thinking specifically planetary science even at Utah?

BLINN: No, not really. Planets are not very specular. [laugh] When I got the bumpy surface model, I just sat down and made as many different odd permutations as I did. I made a strawberry with seeds, and put wrinkles on a teapot. I made a sphere, and using a paint program, I did a quickie a texture map of craters. A cratered sphere was one of the other images I put in my thesis. It was really crude looking, but it was effective.

JPL and the Voyager Program

ZIERLER: Now, when you wrapped up, did you have that initial visit to JPL in your mind? Was that the idea for you? You wanted to go to JPL?

BLINN: Yeah, I did. What happened was Ivan Sutherland, who was the professor at Utah who started the department, had left to go to RAND just about the time I got there. I missed actually working with him directly. But I communicated with him by phone a couple times. He needed an image made for a presentation that he was going to give to the Navy. I synthesized an image of some geometric shapes that he wanted to show. As I was graduating from Utah, Ivan had then moved over to Caltech, and started the computer science department with Carver Mead there. I called him up, and said, "I've just graduated from Utah. Is there a place for me in your department?" He said, "Oh, sure, great. Come on." I said, "To be honest with you, I'm really interested in JPL, so I'll probably go out and hang out at JPL, which was a part of Caltech, and see if there's a project somehow I can attach myself to, and see if I can get involved in what they're doing somehow?" Ivan said, "Well, interesting you should say that because I happen to know that there's this guy at JPL who just bought a complete duplicate of all the hardware that you had at Utah to do your thesis on, and he's looking for somebody to do something interesting with it." [laugh] What could be better? I went there as a postdoc, and I was a teaching assistant for Ivan's course in graphics on campus, and then spent half my time and then my full time at JPL doing "something interesting" with their computer graphics equipment.

ZIERLER: Were you watching Carver and Ivan put together the computer science program at Caltech? Did you contribute to that at all?

BLINN: I don't know if I contributed to it, but I observed it. I sat in on the course that Carver gave on LSI design, and got the general gist of that. But I was never really a hardware type so much, my experience mostly was software. It was interesting, the excitement of it—they were going to make their own chips. They had their own mini computers before that was an off-the-shelf item. But mostly once I got to JPL, I put my attention to simulating the Voyager program.

ZIERLER: Now, how did you get assigned to the Voyager program, and who was the first person that had the idea that computer animation would be an important component of the mission?

BLINN: The guy at JPL that I was working for was Bob Holzman. He basically had—as my understanding of it—he had some money left over at the end of the fiscal year that he needed to spend, and so that's why he bought all this computer graphics equipment, and he wasn't sure what to do with it. When I showed up, he and I worked on getting the thing going, and getting all this Utah software that I had going. But what had happened was the guy who was the chief mission designer for Voyager, Charles Kohlhase, in addition to designing the timing of the missions to get there and see the moons as well as the planets, he had made a film of what it would look like flying by Jupiter and Saturn. It was a line drawing image using a film recorder. He and a guy named Paul Penzo did the software. Somehow or other, I saw that and said, "I can do that line drawing in real time now," using the real-time line drawing hardware that was on our system, even though I couldn't do the color in real time. "This is what I'd been put in the universe to do, to make the color shaded version of this film". Charlie and I got together, and I got data from him and Paul, some of Paul Penzo's code, and cannibalized that and put it together, and put it together with the texture mapping, textures on the surface that's there, because that's what the planets are, even though they are more diffuse reflected; they're not really shiny. All these different things all came together to be able to make the film. I got there about the same time as Voyager was launched. The first Jupiter flyby was like 18 months later. I had exactly enough time to get all the software going, and get the data going, get the animation together, and make a film in time to be shown before Voyager got there. The timing of that worked out amazingly well.

ZIERLER: Did you ever get to talk to Ed Stone and get his views on the value of computer animation?

BLINN: Yeah, I talked to him several times; not specifically trying to convince him of it because once the films were there, the value of them as an educational or marketing tool was fairly obvious, although the Public Information Office wasn't quite so interested in it at first. It was a bootleg project, in a sense, and nobody knew I was doing it, which was good in the sense that JPL and NASA really want their projects to succeed, and so they have to try very hard to make sure everything is done right and done perfectly, and no mistakes. There's a lot of bureaucracy related around that in order to make that work. My project, nobody knew about, and so I was doing it pretty much on my own. If it didn't work, nobody would've known and nobody would've cared. But it did work in time, and so that was great. I didn't have to be quite as formally vetted in order to do this, as most of the projects at JPL were. The situation was also the fact that the Public Information Office was not aware that I was doing it.

They, of course, are responsible for disseminating the information, and making sure that everybody gets it, you know, the news media gets what's going on. They want to be the central distribution authority of what goes out to the public. They didn't want the scientists to feed information to one individual reporter because then another other reporter could say, "Why didn't we get this?" It made sense that they were the funnel through which everything went. But what happened was when Charlie and I finished our film, he showed it to the director of the Public Information Office. The director did not know what to do with it, so he just tossed it in the corner, and said, "That's nice." But it turned out Charlie had a meeting back in Washington with the director of NASA at the time, to brief him on what was going to happen during the first Jupiter encounter. He showed him the film, and said, "Here's a film that I made with this other guy, showing what we expect to be seeing." The director of NASA said, "This is fabulous. We will make copys of this film, and give it to every news media in the country." [laugh] That's how it got publicized originally.

ZIERLER: Jim, some technical questions. Was there anything on the Voyager spacecraft that was specifically designed to help you do computer animations?

BLINN: The imaging system, which sent back pictures, helped because I got the pictures to use as texture maps for the surface. Their measurements with fields and particles and magnetometers was not specifically applicable—the Voyager spacecraft had been designed and launched by the time I got there. They certainly didn't do anything for me [laugh] specifically. But I did get access to the images that came back, and I used them to improve the texture maps that I had.

Dots and Movies

ZIERLER: Maybe it's an obvious question, but why are the images themselves not sufficient to convey to the public what the spacecraft is seeing? Why do you need to animate those images to deepen the impact?

BLINN: The images that came back were not movies, although they were able to put together movies of it. But the movie that I made was produced and released before it got there. When my movie was done, Jupiter was just a very small dot in the imaging. There was nothing to see quite yet, until the actual encounter happened. Just the pictures themselves, of course, are the ultimate goal of that, and those were spectacular when they came back. But partly it was that I was able to show the dynamics of it more. It turned out that the Voyager this dramatic fly through underneath the south pole of the moon Io. That came out really flashy as Io was zooming by overhead as Jupiter gets bigger and bigger. Then you turn around and look at the next moon that comes by, and Jupiter gets bigger and bigger. It was kind of a more dramatic illustration of what was going to go on.

ZIERLER: Now, why were the spacecraft not equipped to take movies? Is that a technological or more of a budgetary limitation?

BLINN: It was technological in the sense that they sent back the images as fast as they could. I think it took 90 seconds to get an image back. Every 90 seconds you'd get another image. They made stop-frame movies, in a sense. When they got them back, they put a bunch of them together to show you zooming into Jupiter, and then show them the red spot circulation. It was not like a real-time movie that was coming back, because the images were about 90 seconds per frame coming back.

ZIERLER: I wonder if you could walk me through the process step-by-step of how you take a still image from the spacecraft, and turn it into a movie that accurately conveys what the spacecraft are actually seeing.

BLINN: The first thing I had was the model of the flight dynamics. Charlie, as I say, was the guy who designed the trajectory, and knew exactly when it was going to get there. I was able to get that data, which was basically the geometric position and shape of the hyperbolic trajectory of the spacecraft by Jupiter, and the elliptical orbits of the moons around. I was able to simulate that just by solving Kepler's equation, and plugging in the time, and finding out where everything is at each time, and then making a view of it, and then getting the surface features on the moons. One of the earlier pictures of Jupiter that came back was a couple hundred pixels across. I was able to take that, and I was able to put that time into my program, saying, "Show me what you saw at that time." Then I did reverse texture mapping, where I wrote a program that would go through and scan out the image, and take the colors out of the image, and put them into the texture map that I was making. I was able to construct a flat texture map from that image, and take a couple of those, and then mosaic them together. That would give the surface of Jupiter fairly accurately for the time we did it. We didn't know what the moons looked like at all, so we hired an astronomical artist to come in and draw a hypothetical map of what Io and Callisto and Ganymede would look like.

ZIERLER: Was animation ever useful for discovery itself? Could you use the animation to better understand the images coming from the spacecraft?

BLINN: I was not on the science team. But there was one spot, I guess. I was very careful to simulate the geometry of it as accurately as I could. There was one time when we were playing it back when they noticed, Charlie and some of the other scientists noticed, that they were doing a star occultation or something like that at exactly the same time as the red spot of Jupiter was going by. They said, oh, we'll get interesting information at that. But I was not really involved in the science planning at all, just mostly mimicking what the scientists were going to do for the visual effect of it.

ZIERLER: Jim, I wonder when you first realized what an impact this had worldwide, the excitement around what the—

BLINN: [laugh]

ZIERLER: —Voyager spacecraft were seeing, and how this connected this achievement in space exploration with helping people understand what was happening.

BLINN: It's hard to describe it. It's like this is what I grew up all my life to be doing. I just lucked into it. There's so many—so I look at my career—so many lucky breaks and happy circumstances and near misses that made it possible for me to be exactly there at exactly the right time with exactly the right equipment, exactly the right people to work with. It's astonishing to me that this happened.

ZIERLER: Did you realize that this wasn't just an American event, that this was a worldwide celebration for Voyager?

BLINN: Yeah. Like I said, I watched some of that from a distance. But the fact that they made copies of this movie, and sent it out, so it was shown across the entire country during the evening news as Voyager is going by. This is what we expected to see. Every local news program showed it. Other people who were doing computer graphics at that time were making movies of different things, and they could show them to small audiences here and there. But this was the first time computer graphics had been shown at such a wide scale. It was just a little lucky break for me that I was the one who made the movie that got that much attention.

The Hollywood Connection

ZIERLER: Jim, I understand there's an origin story of Hollywood animation within JPL. Who were the first people that had the idea that the computer animation that was doing such wonderful things in terms of igniting our imaginations for planetary science and exploration could do the same for film and movies?

BLINN: The main guy, I guess, would be Ed Catmull, who was a graduate student at Utah, and graduated and left just before I got there. But I met him, and his dream in life was making feature films with a computer. He went and directed a project at New York Tech, New York Institute of Technology—the place I was at during one summer—and various people that he hired to do that. The movie industry was not that impressed with computer graphics at first because the quality was nowhere near what a real thing was, even for animation. But it kept inproving. But Ed Catmull, and Alvy Ray Smith, one of the first people that he hired at NYIT, they actually came and worked with us at JPL for a while on some projects before they went off to work for George Lucas to start his computer division. I was certainly not the only person.

But what happened was, since JPL is near Hollywood, and it's like the high-tech place, we had this continuous stream of people coming through for demonstrations of high-tech stuff. I gave demonstrations to anybody who wanted to see it. It helped also break the ice, showing what could be done. I gave a demonstration to a lot of people—Ward Kimball was one of my heroes growing up, as a Disney animator. He made educational films, you know, Man in Space, and Mars and Beyond, which inspired me when I was a kid. I did give a demonstration to him as well as Disny animators Frank Thomas and Ollie Johnston. Gene Roddenberry came by, and I gave a demonstration to him. Various others, mostly people who were building equipment would come by, and get introduced to the idea that it existed, and that it would be something that they might want to play with.

ZIERLER: Jim, did you ever think about joining the film industry yourself, about leaving JPL and going to Hollywood?

BLINN: I did briefly. What happened was, once the two Jupiter flybys were going, that's when Lucas was hiring people to start his computer division. The computer equipment we had by then was getting obsolete compared to what you could get. I figured maybe I should go and join them. Ed Catmull had started the computer division at Lucasfilm at the time. I said, "Can I come and work for you?" He said, "Great. Come on up." I told the JPL people that I was leaving to go to work for the movie industry. The head of JPL at the time, Bruce Murray, said, "Oh no, we don't want to lose you. What would it take to keep you here?" I said, "If you could upgrade the computer from a PDP-11 to a VAX, and get me a full-color framebuffer, that's what I would need." He said, "Great. I'll use my director's discretionary fund to get you these things, if you'll stay." I worked basically at JPL and LucasArts fifty-fifty, half-time for a few months, but then came back full-time at JPL in time to do the Voyager flyby of Saturn movies.

ZIERLER: What were the projects at Lucas Film?

BLINN: Lucas at the time was not interested in animation or the visuals so much as he was interested in helping to automate the film-editing process—which was this very manual thing of strips of film, and trying to figure out how to paste them together into the final movie—and also helping the sound process. The people who went up there had just started working on that and said, "We know that he wants to do animation, but he doesn't realize it yet." They had an animation project, building hardware to make images faster and better as well. That eventually took over once the quality reached a point where it was good enough for the special effects industry to think that they want to use it in their real films.

ZIERLER: Tell me about coming back to JPL in time for the flyby. What was that like?

BLINN: As I said, I was shuttling back and forth every week, and so I didn't have to shuttle back and forth anymore. When the Voyager 1 flyby of Saturn came up, actually I finished that one before I went up to Lucasfilm. The big trick there was of modeling the rings of Saturn. I did something fairly simple somewhat earlier. Pioneer 11 went by Saturn before Voyager, and so I did a simple animation of Pioneer 11 going by Saturn using an incredibly simple model of how the rings reflect light, which wasn't very accurate but looked cool. When Voyager 1 was going there, I did research how clouds of particles reflect light, and statistically how much goes through versus how much gets blocked on the lit side and the unlit side, and adapted that into a computer model so that I could get the ring intensities correct. I think I got the density of ring particles from a star occultation that someone did. The rendering was driven by as much physical data as I could come up with.

Then after the first Voyager flew by I got much better star occultation data, so I used that to do the second flyby movie. That was another interesting thing about the flyby movies too, in addition to just showing what was going to happen, since there were two Voyagers, Voyager 1 went by Jupiter, and sent back all these cool pictures of the moons and Jupiter. Now we had pictures of what the moons really looked like. I was able to scarf up some of those real images, turn those into texture maps, and so I used those as the texture maps for the Voyager 2 flyby movies. The Voyager 2 flyby movies not only showed what Voyager 2 was going to do, but also it showed off some of the results that we got back from Voyager 1. Likewise, with Voyager 1 and 2 going by Saturn, again, it was one of these amazing things that happened that the fact that there were two of them, I was able to use the first one to enhance my database and my models to show off what Voyager 1 found when I did the Voyager 2 flyby movies.

ZIERLER: You're saying that having Voyager 1 and 2 was an absolute boon for producing better computer animation?

BLINN: Well, more accurate models of what I was animating. The machinery of the animation was improved somewhat, but it was mostly getting the surface feature data. I did simple simulation of the volcanoes on Io 1 as part of the Voyager 2 movie. They were not very good but at least gave a flavor of showing the volcanoes on Io in the Voyager 2 flyby movie.

The Joy of The Mechanical Universe

ZIERLER: Jim, when the Voyager spacecraft were going strong, and it was decided that they could keep on going beyond the outer giant planets, straight into interstellar space, what did that mean for your work on computer animation? How do you animate the great black void?

BLINN: By the time that happened, I was still there when Voyager 1 and 2 went by Jupiter and Saturn, and then we got involved in The Mechanical Universe project, which was the physics course that I worked on. Meanwhile, we got the new computer and the full-color framebuffer, so I did an animation of Voyager 2 going by Uranus, again using earth based star occultation data to get the rings right. Then I was mostly wrapped up in The Mechanical Universe project when it encountered Neptune. Some other people at JPL used my software, and enhanced it some, to do the flyby movie of Neptune.

But actually what happened was during Voyager 1 and 2, I made a really quickie movie that I called Previews of Coming Attractions, which was just a 10-second flyby of all four planets using the trajectory data that we were going to fly. So the flyby of Jupiter, flyby Saturn, and then 1986 plan fly by of Uranus. Then in 1989, here's Neptune going by. I did the preview movie which they showed at the end of the final press conference for each of the Voyager 1 and Voyager 2, Jupiter and Saturn. An enticement to come back in several years to see the next planet. But as far as going out beyond Neptune, there's measurements that they made of magnetic fields, and where these solar wind stops. But the imaging part of it, there's nothing to see out there, and everything back near here is too small. The visual part of Voyager was finished after it went by Neptune.

ZIERLER: Jim, you mentioned The Mechanical Universe. Let's move on to that topic. Of course, what initially brought us together was, sadly, the passing of Professor David Goodstein. When did you first meet David, and what was your sense of how The Mechanical Universe project got started?

BLINN: I'm trying to remember exactly what happened. I think Bruce Murray introduced me to David Goodstein originally. Again, this is another one of these dream projects. What happened was Walter Annenberg was ambassador to England during the Reagan administration. When he was there, he saw something called the Open University, which was early morning TV programs where university courses were broadcast throughout the country. When he came back, he said, "I'd like to do something like that in the United States." He funded a bunch of projects from different universities to make a telecourse on their subject of specialty, and we know Caltech's is physics.

We got a contract to do that. I believe Bruce Murray introduced me to David as a potential source of useful visuals to put into this telecourse. Also, the funding to do that also got us able to upgrade our equipment a whole lot. The telecourse, again, was one of these dream projects. I grew up watching telecourses on TV in my small town in the middle of Michigan, and now I get a chance to make the next generation of telecourses. It was a combination of the physics—I had a degree in physics and enough of a mathematical background that I could do simulations of the mechanics and the electrostatics and the whole thing, of the entire course. It was a really intense time because we had to do, I think it was, over 500 different animated scenes throughout the project. And I was also teaching the course in computer graphics at Caltech and an introductory course in computer graphics at the Pasadena Art Center College of Design. At JPL I had an assistant or two, Tom Brown and Sylvie Rueff, working with me on TMU. We upgraded our hardware so that we could put the frames directly out on videotape instead of having to do it on film. Film was what we used for the Voyager flyby movies, we set a 16-millimeter movie camera set on a tripod, looking at the TV monitor. You had to click off a frame, read in the next frame, click. It took six to eight hours to film each Voyager flyby movie. Then we had to wait to get it developed before we could see if it actually worked. But with the new setup with recording directly on videotape, we could see immediately if they worked, and it helped the production a lot because it was like eight hours' worth of animation scattered throughout the The Mechanical Universe project.

ZIERLER: Did you join the project right at the inception, or was it already underway?

BLINN: I started at the beginning. We made a pilot program, and I made animations for that. That came out as Episode 2 in the series. It was a pilot to show the funders that we could do it, basically. So I was there from the beginning, doing the animation. During production I would get a script and whenever there was something marked off about "how do we solve this", I said, "I can do an animation of that," and so went off, and wrote some storyboards. Then I'd show them to David and the production people, and get that approved. Once the pilot program was done it was shown to the Annenberg Foundation to show that we could pull this off. Then they gave us the grant to actually fund the whole project.

ZIERLER: Jim, tell me a little bit more about the workflow of The Mechanical Universe project. How did the ideas start, and how developed were they by the time they came to you saying, "OK, now help us animate this"?

BLINN: It certainly evolved. David had the outline of the course that he taught at Caltech, which he started with that determined what each episode was going to be. He worked with a scriptwriter who wrote a script, a half-hour worth of script, containing scenes of David lecturing, scenes of actual people doing demonstrations with physical apparatus, historical reenactments of various people, and scenes of animation. I would get a copy of the script, and then I would go off and make rough storyboards about what I think should be in there. One of the things that I invented for this was what we called the algebraic ballet, because whenever we wanted to do mathematics derivations on the screen, nobody had done anything like this before. I conceived the idea of having the equation sort of dance around, substitutions happening, things jumping over the equal sign, lightning bolts coming in and canceling things out. I got a lot of input into the visuals of that when they were there.

As I had grown up there were several things that I had seen visually when I was learning physics that impressed me. I was able to make some suggestions like, "I think we should animate this like this." Like the theories of special relativity, when I first learned about it, it didn't make a lot of sense to me until I saw a book called Special Relativity in Diagrams, which showed drawings of Minkowski diagrams. I realized, oh, that's how we should do it. When those came up, I said, "I want to do an animation of this, showing Minkowski diagrams, and showing the transformations of coodinate systems as things moved around." We had a conventional animator draw us some cartoon caricatures of Einstein and Lorentz, which I digitized, and used riding in train cars, looking at each other; one going fast, and one going slow. On the whole, it was about a five-year project for the whole series. I was working on it absolutely every day of the week. I'd meet with David Goodstein every day at two o'clock, seven days a week, because I knew that if I was going to pull this off, I was going to have to do a little bit every day. No waiting till the last minute or anything like that. [laugh] I arranged with David so he would come in. I promised myself I would show him something new every day, just to keep myself on track. That's what he did, and that's what I did.

ZIERLER: What was it like working with David? What did you learn about physics in being able to work with him so closely?

BLINN: He was great to work with. He had a great sense of humor. He had to manage the project between the production people and the Caltech staff. Tom Apostol came on to help keep the mathematics correct. Some other physicists came on to help write the textbook that went along with it. He was very appreciative of what I was doing, and had all good suggestions. But I met with him for half an hour every day, and then he was spending the rest of his time [laugh] talking to the production people.

ZIERLER: Was The Mechanical Universe strictly a Newtonian-focused project? Were there aspects of quantum mechanics covered in the project?

BLINN: Oh yeah, sure, we went the whole undergraduate thing. The first project was just for the first term, so we went through basic mechanics. The culminating epoode was planetary orbits, which we had plenty of data around to show. We showed how the Voyager gravity assist worked. Then we got a contract to do the second half of the series. The first 13 episodes were mechanics, and the second half were electricity, magnetism, thermodynamics and quantum mechanics. We did a lot of that. They did a lot of live action experiments. Then I did a lot of animated demonstrations of how electrons move in wires, and how magnetic fields interact with particles, then the relativity series, and some stuff with quantum mechanics showing electron orbitals.

ZIERLER: Jim, what were some of the most difficult or challenging concepts or laws in physics that you found as you were attempting to animate them?

BLINN: Since it was an undergraduate course, I was pretty familiar with the physics that we were showing. I had to do a lot of pulling from my experience in different directions, both in computer science and in numerical analysis. I had to write programs that would simulate the electric fields, numerically calculating what the magnetic fields were doing as objects moving around. I don't know if there's any one thing more challenging than the others. I'd also had some ideas on how to do quantum mechanics too. It was mostly my pulling all my knowledge off the shelf that I had accumulated over my life, and saying, "I know how to do this now, but I just have to sit down and make it work."

ZIERLER: What kind of feedback were you getting as The Mechanical Universe program was gaining such a large audience?

BLINN: We got a lot of good reviews of it. Mostly, while we were working on it, I was head down, cranking out the movies as best as I can. I was able to get some feedback from the computer graphics community. Every year, there's a conference called Siggraph for computer graphics people, and they have a film and video show where people showed off their work over that year. I was able to do a clip reel of the animated bits from The Mechanical Universe each year for about five years, and I put my own little hopefully amusing narrattion behind it. That was a chance to show off what I had done to my colleagues, the computer graphics community. The film/video show was several thousand people in the auditorium, and they're showing films from Pixar and from various research groups and then mine and others. But I always got a really good reaction from my colleagues with the little bits that I showed there. That was very gratifying. I should mention, TMU and my siggraph demos are on YouTube.

Carl Sagan and Cosmos

ZIERLER: Now, what was the sequencing? When did you start working on the Cosmos program? Was it during The Mechanical Universe?

BLINN: Oh no, Cosmos was way before TMU. That was basically during the Voyager 1 Jupiter flby, pretty much. A guy named Gentry Lee was at JPL and was a good friend of Charlie Kohlhase, so he had seen what we were doing. Gentry Lee was Carl Sagan's business partner in doing Cosmos, and so through him we got a contract to do some animation for Cosmos. I was first introduced to the existence of JPL by watching Carl Sagan giving talks about it back when I was in Michigan, and thought, "Wow, that'd be cool place to work." Then when I was a graduate student in Utah I remember reading read in the TV guide that Carl Sagan is trying to put together a television course about how the universe works. I thought, "Man, it sure would be fun to work on that."

ZIERLER: [laugh]

BLINN: I had no idea how that could possibly happen. When I finished with Utah, I got to JPL, and X knew Y, Y knew Z, and Carl Sagan came in. This was early on in the development of the software, so it turned out that we were not able to do anything really elaborate for Cosmos. Mostly it was black and white line drawings, which showed Kepler's laws and things, and they also used clips out of the Voyager flyby movies. The two main difficult things we did for Cosmos was one on human evolution and one on DNA replication. For the evolution scene we had to start from a one-celled organism and evolve it through different animals to human beings. We had an artist work with an evolutionary biologist to draw key stages along that progression. I put together some software, an interpolation program that would take two drawings, and change the shape of one line from the first drawing to the shape of the second drawing, which is what a lot of other people were doing to do animation. But I did my own version of that. Then my assistant at the time, Pat Cole, would digitize those and match up which lines evolved into which other lines. So human evolution was one of the more difficult things we did. Then finally near the end of the project, they said, "We want to show how DNA replicates, and we were thinking of doing it with models, but we don't know. Can you do this? Here's a sketch of what we want it to look like." They had this sketch of this DNA molecule. It's like, my god, this is way beyond what we can do with computer graphics these days on shaded images. But I went home that night, and I was thinking, in order to do this, what you need to do is to simulate the electron density field around each atom, and then run an isosurface through it. You could have an algorithm that would do that by sorting the atoms in a quadruplly nested loop and etc. Hmm I bet I could pull this off. I came back, and I said, "I think I can do this." Again, this was like a nightmare project, working day and night to get this special purpose program that would make atoms blob together when they got close together. I got a database of the molecule from a colleague from the National Institutes of Health, Tom Porter. We had the location of the atoms, 3D coordinates of the atoms.

Then I tinkered and got a special animation program and rendering program going, and it took 20 minutes to make each frame. It was a really, if I may say so [laugh], a spectacular program to do the rendering on this thing, but it took forever. We were doing the rendering, and we had actually two PDP-11s at the time. This was before we got the VAX. We were doing frames on each one of those and then we put them together. We only had enough time to do every fourth frame, so we did that and showed every fourth frame. We showed it and we said, "Here's the best we can do." They said, "Wow. This is really good, but could you do every frame now, because it's a little too jerky?" There's some rule about, you know, there's never enough time or money to do it right, but there's always enough time or money to do it over. [laugh] We said, "OK, let's do that." But, meanwhile, we borrowed time on two or three other PDP-11s around JPL and on campus. They had this thing going where one was making 20 frames, and we'd get that on mag tape, and we'd drive that across town, and merge that in with the other frames, and finally pulled that off, and filmed it, and it came out. It came out pretty good. That was the DNA replication scene.

ZIERLER: What was it like working with Carl Sagan? What was he like as a person?

BLINN: He was a really nice guy. Again, I didn't work with him directly. I was nervous meeting him for the first time when he came in for just a demonstration that we could do it. But he was a nice guy, and he was very smart and perceptive, and also very appreciative of what I was able to do for him. But mostly I worked with other production people who were responsible for putting things together. But I had dinner with Carl a couple of times, and went to several parties that he was at. But that was the extent of it. I didn't work with him real closely.

ZIERLER: Jim, I wonder if you can explain animating especially the first episode about galaxies and the size of the universe. How do you deal with scale? How do you deal with the size of the universe and accurately conveying that in animation?

BLINN: I didn't really do too much of that. [laugh] Mostly it was planetary flybys. There were a couple of scenes in Cosmos of galactic dynamics that we did. Charlie Kohlhase actually wrote a program himself. He worked on the Cosmos project as well as myself. He did one of the animations showing dots for particles moving around showing planetary accretion. And I did a simulation of galactic dynamics where I just constructed some functions that did roughly what the physical models were supposed to look like. It didn't come off looking very good, and so I think they actually replaced it when they re-released it later on for the second release of Cosmos. Most of the galactic dynamic things they did in Cosmos were done with paintings that an astronomical artist had done.

ZIERLER: You mentioned DNA. How much work did you do on biological systems for Cosmos?

BLINN: There was the evolution sequence and the DNA sequence. Those were the only two biological things that I did. Most of the rest was planetary orbits.

ZIERLER: What about Mars? Did you do a lot of work animating Mars?

BLINN: No. I never got involved in that. Mars was done by the time I got there. I don't know if there were any projects for Mars that happened while I was at JPL. Mars things were either before I got there or after I left. But one of the first pictures that I got to tinker with was some of the Mars photographs, just to put them on the screen to test out our display. I did some experiments with noise reduction filters to get rid of some of the dropouts, but nothing particularly momentous there.

ZIERLER: Jim, of course, one of the main themes in Cosmos was the search for extraterrestrial life, the search for aliens. Were you involved in that at all? Were you involved in thinking about how to animate life forms beyond our own?

BLINN: Not really, no. I have a feeling that with the speed of light and so forth, it's going to be something that we're never going to know whether there's things out there because it takes too long to visit them, and the conditions for life happening are pretty narrow. I'm not sure if I expect that we're going to be communicating with aliens at any time soon.

Focus on Writing

ZIERLER: Jim, let's move on to your writing career. Of course, the column Jim Blinn's Corner for IEEE, how did that get started?

BLINN: It was another good confluence of ideas. I had been teaching computer graphics courses at Michigan and Utah and Caltech for some number of years. There were a bunch of ideas I had on how the mathematics could work in solving certain graphics problems. These are not research papers so much as more a textbook sense of things. But I did not have the energy or the stick-to-itiveness to write a textbook. But I knew the IEEE had this magazine, and I thought maybe it would be useful if I could write a regular column in there with this month's mathematical trick. It turned out the editors were also interested in finding somebody to write a mathematical column like that. Again, we bumped into each other, and they said, "Yes, great." Now I was stuck having to write an article every month.

I'm a lazy writer, and it was hard for me to get them in by deadlines [laugh], but I managed to do it, although I missed a few issues. Actually, the magazine changed to bi-monthly, two months [laugh], not every month now. But I did this for 20 years, 83 articles. I kept coming up with new things, mostly mathematical ideas, some of which were mainstream and some of which were my own personal take on how mathematics works. Most of the articles were about the math used in computer graphics and others were just observations about the computer graphics industry as a whole. The editors were very supportive and forgiving of my lateness of getting things in. It was also an interesting travel through the technology of publishing things, because the first time we did it, I had to write it up and just send them something typed out on paper with hand-drawn illustrations. Then we figured out how to do equations on a phototype setting machine that I could rent time on. Then later on, a few years later, I found desktop publishing software, so I could use that, and get the equations and illustrations done that way. In the final ones, I think I was able to use Microsoft Word and the equation editor and Visio illustrator, and do everything on that. I got to ride through the progression the industry of desktop publishing of mathematics, learning about it as I did it.

ZIERLER: Jim, did you have any idea that the column would last for as long as it did?

BLINN: Kind of, yeah. I ran out of energy after a while. I had some more ideas of things to write, but I had other demands on my time, and so it closed down after 20 years. But the columns from the magazine were reprinted in three compilation books by Morgan Kaufmann Publishers, who has done a lot of computer graphics books. But there are about fifteen that were done since the last compilation that I'm actually right now in the process of polishing up and adding to and improving a bit. I'll just slap them up on my website sometime with a Creative Commons sort of copyright. I've spent most of my time mathematically these days working on something called tensor diagrams, which come from ideas which I have put together from various mathematicians that I'm just totally fascinated with. It's is not very mainstream these days,but I think it's a fun way of looking at the mathematics of how to look at the geometry of the sorts of shapes that equations can generate. That's my fascination these days.

ZIERLER: Jim, do you have a favorite column from the Corner?

BLINN: Probably my favorite ones were the ones that were done on tensor diagrams, about how to represent lines in three-dimensional space, and how to find roots of cubic polynomials. I think those are my favorites.

ZIERLER: Did you use the column as a platform to call for changes in technology or in software?

BLINN: Well, not really. I wrote a couple of columns about how to get a paper accepted at Siggraph, because I was on the papers committee a couple times, and so the suggestions were to people on how to write a paper that would be good enough to be accepted by Siggraph. Another column on how to attend a conference, what sorts of things to expect to see. That wasn't so much interested in changing people's attitudes as I was in telling them what to expect and how to fit in and work with the community.

ZIERLER: Jim, at the same time you were at Caltech working on the Project Mathematics!, what was that?

BLINN: That was basically a prequel to The Mechanical Universe. When TMU was done, Tom Apostol wanted to do a similar series on high school level mathematics, because one of the things that we found in showing The Mechanical Universe was that a lot of high school students didn't have the mathematical background necessary to understand the college level courses. He got a grant to do a similar thing, although not quite as elaborate as The Mechanical Universe, to do a series on high school level mathematics. He wrote the scripts for it. Basically I moved into the offices on Caltech that used to be Mechanical Universe offices. Again, it resulted in an upgrade to hardware. I moved everything over to PC clones and some cheaper videotaping machines, and so we were able to generate the animation, again, 5–10 minutes of frame but still with the cheaper equipment. We did a bunch of those. After a while I got burned out, to be honest with you, having video deadlines all the time. I backed off on that at about the same time that the funding ran out. We actually had some more we were going to do but we just never got a chance to. Then I moved on to Microsoft.

ZIERLER: Jim, at JPL, did you not work on another flagship mission after Voyager?

BLINN: I did some simulations for the Galileo project while it was still seeking funding, basically showing the proposed spacecraft flying by the various moons of Jupiter. They might have used the animations to help get funding. I hope I helped in that. Then there were some animations of the four (non US) spacecraft that were sent to encounter Halley's comet. And I worked with some ressearchers that had mathematical models of fields and particles around Jupiter and around Uranus to show what they looked like. And I did some simple animations of the moons of Jupiter to show off the surface textures that we now had. I had the help of my assistants Tom Brown and Sylvie Rueff in most of these. And after I moved my attention to Project Mathematics! some other other people at JPL did some stuff with the software that I left behind.

ZIERLER: Was your sense that if not you working directly on these missions that the idea of there being a computer animation component to next-generation planetary exploration missions, was that sort of sealed in at that point? Was that sort of part of the JPL infrastructure?

BLINN: I hope so. I did the Voyager animations. After the first one became popular, the other ones were starting to get expected, and so I did those. Some other people took over the whole animation department there now that's doing animation in various things, which is great. I'm glad if I helped get that going. Although, nowadays, computer animation's everywhere, and so whether I was there or not, I'm sure they would've had a department by now, anyway.

Building the Graphics Program at Microsoft

ZIERLER: Tell me about moving over to Microsoft. What were your motivations?

BLINN: Mostly it was to join a bunch of my friends and colleagues who were going there to help Microsoft get their graphics program in gear properly. They were designing a graphics system. But when I got to Microsoft, I actually got distracted by writing the IEEE columns and getting involved in the theoretical math I did, so I didn't really contribute a whole lot to Microsoft's graphics community so much as doing some mathematical documentation of how geometric mathematics works. Since I was in the research department, that was OK for a few years.

ZIERLER: I wanted to ask. There was obviously a research culture at Microsoft that allowed for this flexibility?

BLINN: Yeah. They supported me for 14 years or something like that doing this stuff, and writing IEEE articles primarily, and giving whatever advice I could to the graphics people. I was not the only graphics person there. Jim Kajiya and Alvy Ray Smith and later Turner Whitted were there, and a whole bunch of other people who I'd worked with before in other places. One of the attractions was to go and work with my buddies again.

ZIERLER: Tell me about developing this new mathematical notation scheme that simplifies the algebra.

BLINN: Since it's pictorial, I'm going to have to describe it in words. But the basic idea is it's like taking vectors and matrices and tensors as rows of numbers that you combine together by what's called a dot product, also known as a tensor contraction. Instead of writing them as grids of numbers, you write them as little circles with arrows in between. It's very similar to Feynman diagrams in the sense that you have a bunch of objects that are interacting with each other. With a Feynman diagram, each of the arrows represents an integral of some sort of density function, as I understand it. In the tensor diagram thing, it's more just a matrix multiplied by a vector type of thing. What's interesting about that is there are some rules that you can do about reconnecting the arrows that allow you to change one diagram into one with a different shape that still represents the same quantity. You can factor polynomials with it, and you can represent stuff that has been represented in the past in more complex ways. I first got the idea of it from a physics book, which was called Diagram Techniques in Group Theory, which talked about Feynman diagrams and other sorts of diagrams. There are lots of dialects, basically, of diagrams, and other mathematicians have used them in somewhat different ways. I think the earliest reference I got was James Sylvester wrote about them in 1900 as ways of doing polynomials. But his had some deficiencies, some ambiguities that the way I'm doing it don't. Anyway, I'm just having fun with that, showing how some of the simpler aspects of, you know, how do you find roots of polynomials? Right now, I'm finishing off the series on finding roots of a cubic polynomial, which has been done forever. But I put together all the ideas I could find, found the best ideas, and put a few ideas of mine in there, and put some nifty diagrams of what the shapes of the space of cubic polynomials looks like.

ZIERLER: Do you see the work on cubic polynomials more as applied math or pure math?

BLINN: It's both. It's pure math in the sense that it gives you a sense of what things can happen with cubic polynomials. It's applied math because I'm trying to avoid round-off error. There's a lot of discussion of how to get accurate results with limited computer arithmetic. The thing is it's a balance between engineering and pure math in that pure mathematicians will say, "Yeah, we know how to do this stuff. We don't need this stuff." The engineers are saying, "Boy, this is too theoretical for me." I'm in the middle of that. I'm just entertaining myself and anybody else who wants to read it. There's a group of mathematicians I'm sending this out to, who are helping check my arithmetic. I'm getting some good feedback from that.

ZIERLER: Did any of this work contribute to Microsoft's business? Was it ever profitable for Microsoft for you to be engaged in this work?

BLINN: No. [laugh]

ZIERLER: Just purely basic research that Microsoft was happy to support?

BLINN: Yeah. There's some aspects to this which could have been developed a little bit in cryptography, but I never really pursued that.

ZIERLER: Being a graphics fellow at Microsoft, did that improve Microsoft graphics?

BLINN: I hope so.

ZIERLER: [laugh]

BLINN: I made some suggestions here and there, and they had a graphics group that was doing things. I was not the only one there, and so there were a lot of other people in the graphics group who focused more on graphics, and doing graphics research, and how to improve things, and fine-tune algorithms. The idea is, as the hardware improves, real-time images become more and more easy to do. There are a lot of people in their graphics group that would figure out how to milk the most interesting things out of the hardware that you could get.

ZIERLER: Was the graphics group at Microsoft in competition with other outfits at other major technological corporations like Amazon or Google or Apple?

BLINN: I don't think so. A lot of people were doing things. I don't think Google even existed while I was at Microsoft. Apple had its own research group. There was a lot of research done in video compression, both at Microsoft and at Apple, and each one had their own thing. But the video compression that most people use now is neither of those. It's the MPEG standards. Just generally having people work on things increases the cloud of knowledge around so that the good ideas can pop out eventually.

Current Interests in Tensors

ZIERLER: Jim, when you retired in 2009, what did you want to focus on? What did you want to free up your time to look at?

BLINN: I'm taking care of my family nowadays mostly. I'm also, like I say, doing a little math nowadays. I've become a YouTube addict. [laugh] There's lots of good stuff on YouTube to entertain me.

ZIERLER: You're still working on the tensor research?

BLINN: Yeah.

ZIERLER: What are some of the big open questions? What keeps you engaged in this topic?

BLINN: It's a matter of taking all these disparate algorithms, and finding a common thread between them, and figuring out how to improve things. The cubic polynomial thing, there are a lot of different ways of solving cubic polynomials, so I try to pick the best ones and add some of my own ideas. Next I'm going to start on quartic polynomials. There are lots of different ways of doing that and I've got some ideas on how to find a good overlying principle that all these others are special cases of.

ZIERLER: Jim, I want to ask some overall retrospective questions for the last part of our talk. Let's start first with some of the major awards you've won. In 1983, you got the NASA Exceptional Service Medal for the Voyager flyby animations. What was it like receiving that medal, and what were you being recognized for?

BLINN: It was for the flyby movies, which the main good benefit of the flyby movies was that it helped popularize the project. They helped people know what was going on. I was lucky enough to be able to produce that and to help the project in that way. The award was for doing that and for helping publicize Voyager to people and to help them understand where their money was going. [laugh]

ZIERLER: Working with Bruce Murray and Carl Sagan along with Lou Friedman, the development of the Planetary Society, do you see your work as ensuring that there would be a future in planetary exploration for JPL that the Planetary Society was addressing the concern that after Voyager, there might not be anything in the pipe?

BLINN: If what I did helped, I'm glad. I think planetary exploration is great fun, and something we should keep on doing, and so anything that my contributions have done to promote that, I'm really happy for.

ZIERLER: Jim winning the MacArthur Fellowship in 1991, what did that allow you to do, given the monetary value of the fellowship and just the prestige of being recognized in this way?

BLINN: [laugh] I was totally floored when I got that, to be honest with you. But it was during a time when we were still trying to get funding for Project Mathematics!, and so it helped me keep employed while we were getting the funding for Project Mathematics!, That was one of the main benefits of it.

ZIERLER: I asked you before, you know, you emphasized that you're an educator. Of course, you were elected to the National Academy of Engineering in 2000. Did it feel really special that this tremendous organization of engineers recognized you as among their own?

BLINN: It's certainly a great honor being recognized. The NAE, I think, is largely an advisory organization, one that's meant to advise Congress on good ideas, what ideas are good ideas, what ideas are bad ideas. I'm not sure I have a whole lot useful to contribute in that regard, so I just accept the honor as it is. I don't think I've been much use to them in terms of doing any advising to Congress.

ZIERLER: Then more recently in 2020—I don't know how to say it—is it the Ub Iwerks Award?

BLINN: That was great too. Ub Iwerks worked for Disney for many years, and also had his own animation studio. But he was interested in the technology of animation, and so the Ub Iwerks Award is given out to people who have advanced the technology of animation. Again, it's something I was totally surprised to get. But it was really good to have what I did considered important enough to get this award.

ZIERLER: Jim, as an educator primarily—I know it's so hard to quantify these things—but how do you think about the impact that you've had on young minds, even older minds all over the world, helping them visualize science through this magic of animation? How do you think about that?

BLINN: My favorite accolades are when someone tells me that they got into Physics or Math because of my animations. But I think of myself as a link in the chain of things I used to watch when I was a kid that inspired me, and I was able to make a bunch of things that inspired other people, and they'll make things that will inspire somebody else again. I was lucky enough, as I mentioned before, the timing and the lucky breaks and the various near misses that I avoided that got me a career in doing something more spectacular than I could have possibly dreamed of when I was a kid. I'm just the luckiest person on the Earth.

The Steady Build of Technological Advances

ZIERLER: Jim, from the perspective of being an animator using computers, over the course of your career, from the late '60s to now, do you see technological advances in computation as a steady, smooth increase, or have there been revolutionary jolts in advances that have affected how animation is done and how good it can look?

BLINN: I would say it's more gradual. People who liked doing computer animation have been following it and improving it, and improving it, and improving it. The people who were most interested in the picture quality finally reached a threshold where computer animation got good enough for them to use as one of their tools. It's been, I would say, more gradual, but gradual is odd in the sense that things that were difficult, ridiculously difficult, you wake up one morning and suddenly it's not ridiculously difficult anymore due to advances in computer memory sizes and computer speeds. The first framebuffer that I used at Utah used 1K memory chips and needed 2,000 of them in order to make enough memory to make the simplest image on the screen. Nowadays that's so small amount of memory that we don't make computers that small anymore. Each year it's this Moore's law, or rather Moore's observation, of the improvement of things. Every two years, it's a doubling and whatnot. It's been gradual, and suddenly things pop into existence that were impossible before. I guess it's like the boiled frog phenomenon, which is not actually a real thing. Get the frog in the cold water, and slowly heat it up, but actually frogs will jump out when they get too hot. But the level rises slowly, slowly, slowly, and suddenly the quality levels are up, apparently suddenly, it's above your quality tolerance now. Now you can use this.

ZIERLER: Jim, of all the things that you've animated, all the ways that you've educated people in science, what are some of the most difficult scientific issues or concepts to put to animation, and how have you gotten around them?

BLINN: I would say quantum mechanics is one of the most difficult. I have some ideas about that that I haven't seen anybody else do yet, but I'm not sure. One of the disappointments in my life, which I don't remember exactly when this happened, but along the time between Cosmos and The Mechanical Universe, there was a possibility of my working with Richard Feynman to do a computer animated series. He and I met several times to talk about what we could do. He even wrote down Schrödinger's equation on a napkin for me to show how we can simulate QM. I hope I have that someplace. [laugh] It's not something I would throw away, but I don't know where it is anyway. Anyway, he also called me up one day with some ideas on how we could do the sound effects for different types of particles. But that never got funding, and it was disappointing we never got a chance to need to pursue that any further. Where were we going with this? [laugh]

ZIERLER: Difficult concepts to animate.

BLINN: A lot of it has to do with scale. Things happen too fast or are too big or too small or something like that, and so getting scale to be understandable. There are many people doing this now. You go on YouTube, and there's all these fabulous animations of physics and mathematics that I just love, and people experimenting with how to show these things. I'm an enthusiastic fan of all this stuff, myself.

ZIERLER: Is there any animation that you've done that is most personally meaningful to you, either because of how scientifically satisfying it was or the impact that it had educationally?

BLINN: All of them are, basically. I think the DNA animation for Cosmos, I was really happy that I was able to pull it off and it looked good. I think it showed a description of how DNA works pretty well, given what we knew at the time. What was interesting about that was that there's enzymes involved in pulling the DNA apart, which we didn't have a good structure for. I just used some kind of fuzzy blob to represent those, which is good because if we had known the exact structure, the images would have been too complicatied for me to do at that time. [laugh]

ZIERLER: [laugh]

BLINN: I was able to get away with doing something simple, and make it doable, given the time. But the animations that others have made where you can see molecular structures is just so fascinating to me.

ZIERLER: Finally, Jim, last question, looking to the future. You mentioned the metaphor of being a chain in the link. For young people today who are interested—

BLINN: I'm a link in the chain.

ZIERLER: A link in the chain, right.

BLINN: [laugh]

ZIERLER: For young people today who are interested in pursuing a career in computer animation, what advice would you give them?

BLINN: Be born in 1949—

ZIERLER: [laugh]

BLINN: —and to go to graduate school when the first computer images were being made, and be able to be in a place where all the easy problems haven't been solved yet.

ZIERLER: This is your own biography, of course.

BLINN: Yeah. [laugh] I feel bad for people now. For one thing, there's lots of opportunity for showing things. You can do stuff, and you instantly have an audience of thousands or millions of people, which is great. But it also means that there's also thousands or millions of people also doing a similar thing as well. Somebody who wants to make an impact, I applaud you, and I hope you can do it. It now becomes more a matter of the personality and the skill of doing things rather than the technology.

ZIERLER: Jim, this has been a wonderful conversation. I want to thank you so much for spending the time with me.

BLINN: Thanks for inviting me.

[END]