What happened before the big bang?

This is a frustrating question because first, it presumes that just because we know there was a big bang, we understand it back to t=0 or 'before'. This is like presuming evolutionary biology can explain the origin of life itself, which it cannot, since we need to know chemistry and the conditions of the early earth to address it. Similarly, we simply don't have a physical theory that works back to t=0 so we cannot answer the question. But more than that, all of our current ideas suggest, alas, that it isn't a good question!

Saying that always frustrates the listener, but science often tells us that naive questions aren't good questions because they make presumptions that are unwarranted. Because space and time are coupled to matter and energy in general relativity, it is eminently plausible that if space spontaneously popped into existence, so did time. Namely, time and space are both classical concepts that may have a limited domain of validity, and so the question what was 'before the big bang" may simply be a bad question because there was no 'before'! Time didn't exist.

That answer of course makes no one happy, but it may be true. As I often say, the business of the universe is not to make you happy.

"Who is the next Alan Turing?"

As a technology historian, I get asked "Who is the next Alan Turing?" or "Who is the next John von Neumann?" It is impossible to say! No one expected Alan Turing to be the next Alan Turing, or John von Neumann to be the next John von Neumann. Turing was 24, and von Neumann 26, upon arrival in the United States. I'm at a loss for an answer—except to point out that we gave both of them visas, during a 1930s job market that was just as bad as today. 

How does one justify having worked and continuing to work in the financial sector?

As a financial engineer for the past twenty-five years, the question I have increasingly come to hate being asked is: Wouldn't the world be better off if people like you used their skills on real scientific and engineering problems? Given the taxpayer-funded bailouts of the past few years, the crony capitalism, the refusal to prosecute banks for money laundering, I ask myself this question too. How does one justify having worked and continuing to work in the financial sector?

Some people answer this question by claiming that financial engineering makes markets more efficient. That may or may not be true, but to me efficiency isn't a goal. Here are some of my answers.

First, I think everything on earth is interesting and worthy of understanding, provided that you try to understand it honestly, provided that you try to see the world as it really is. 

Second, like most people I have some measure of ambition and vanity. In 1950, Norman Mailer commented in a letter to William Styron:

"I didn't write [The] Naked [and the Dead]  because I wanted to say war was horrible, or that history is complex, resistant, and almost inscrutable, or because I wanted to say that the coming battle between the naked fanatics and the dead mass was approaching, but because really what I wanted to say was, "Look at me, Norman Mailer, I'm alive, I'm a genius, I want people to know that; I'm a cripple, I want to hide that," and so forth."

Mailer's "Look at me" is a good part of what drives any writer or scientist, and what drives capitalists too, to do what they're good at.

Third, I feel it's my duty to try to convey what I understand from experience, namely that financial models are attempts to describe the relatively calm ripples on a vast and ill-understood sea of violent and ephemeral human passions, using variables such as volatility and liquidity that are crude but quantitative proxies for complex human behaviors. Such models are roughly reliable only as long as the world doesn't change too much. When it does, when crowds panic, anything can happen.  Models are therefore useful, but intrinsically fallible. To confuse a model with the world of humans is a form of idolatry, and dangerous. This understanding, I hope, will enable practitioners and academics to use models more wisely.

"How well do you understand what your cloud software does?"

When a company trumpets cloud security designs, they will eventually be breached. If a cloud service is supposed to be robust and "always on", it will repeatedly go down. When private data is supposed to be segregated, it will sometimes leak.

Cloud software violates our expectations often enough that we are rarely surprised. Perhaps we maintain unrealistic expectations because it is organically impossible for our brains to become completely cynical.

If I am asked to explain exactly how a cloud design will perform, I cannot answer, but I am also not perturbed. I can fall back on an inexhaustible spectrum of excuses. There are theorems that limit our ability to understand software perfectly, like the famous Halting Theorem, and one can always blame the human element, the bull in our china shop.

However, engineering is ostensibly going on, and it might seem reasonable to a questioner to press for at least a predictable rate of failure. Other schools of engineering generally provide such guidance. Why is it so difficult in computational designs?

Before engineering can yield predictable results, enough variables must be tied down that an expectation can be fully specified. In the case of information, the most specified situation is chip design, and it therefore yields the most predictable results. In fact, in that case, specificity is maximized, and the result is Moore's Law, a steady course of improved results.

This is the real difference between software and hardware. We can carefully specify the environment a chip is connected to (a circuit board), while software connects to reality, that vague ideal that neither science nor engineering ever fully conquer. 

Fortunately, we can recall cases of unreliable information system designs undergoing reform until they became reliable. For the first couple of decades, personal computers were notoriously unstable. They used to crash all the time. I remember Larry Tesler, the user interface pioneer, crashing an early Mac just by moving the mouse in a certain way, not even clicking.

The tendency to crash is a result of what I call the "brittleness" of software. It breaks before it bends. A tiny error can have great consequences.

Naturally enough, when we gain more visibility into how an information system is connecting to its environment, it becomes possible to make it more reliable. The first cloud service most consumers connected to was an error reporting service to help a computer company debug its operating system. Both Apple and Microsoft initiated such services as soon as many of their customers went online. The cloud provided visibility into software failures. There was finally a way to close the empirical loop in the real world. As a result, modern personal devices rarely crash. I have talked to young people who have never seen a personal device crash.

What the cloud gives, the cloud takes away. While devices have stopped crashing, for the most part, the way they connect together has become unreliable. Brittleness reigns anew. It is now completely commonplace for devices to connect when they shouldn't (violating privacy and security desires) or to fail to connect when they should (as when you can't get onto a cloud service, or when it's hard to back up your data.)

So naturally, we seek to close the empirical loop once again. So far, we haven't done so. We imagine the cloud as an all-seeing eye, but actually it is haunted by blind spots. It does not provide us with adequate visibility into its own failures, in the way it looked down upon lowly PCs to tell us about their failures.

Clouds spawn more diversity in the environment surrounding our devices than clouds can measure. Therefore, relying on data alone will not be enough. We need to do more than we did to make PCs reliable.

This is a frontier of computer science. Don't ask.

No one wants to be asked such a question

I would like to propose to you the explanation why Sendhil's question will not "fly", and my answer is serious, it is not a joke or a logical exercise.  

Here is the reason:

Q:  WHAT'S THE QUESTION ABOUT YOUR FIELD THAT YOU FEAR BEING ASKED?

A: The question about my field that I fear being asked is: "WHAT'S THE QUESTION ABOUT YOUR FIELD THAT YOU FEAR BEING ASKED?"

No one wants to be asked such a question.

Truly !!!

"Isn't That Obvious?"

When I used to teach introductory social psychology, at some point in the semester I let it slip to my students that I was the oldest child in a family of five siblings. Their response was always the same: "You're totally the oldest child!" When I asked why, they came up with an impressive list of supporting attributes (I assume they were primarily positive because I was grading them): "You're smart" "hard-working" "funny" "successful" and on and on.

But then I played a little trick on my students. I told them the truth: I am in fact the youngest of five. Their response? A one-second pause, followed by: "You're totally the youngest child!" Why? "You're smart, hardworking, funny, successful" and on and on again.

Unlike some other branches of science, social science has an unusual curse: regardless of what social scientists discover, laypeople tend to feel that they "knew it all along" such that even the most counterintuitive finding quickly feels, well, obvious. (In contrast, people don't usually think that the general theory of relativity is obvious after their first—or fiftieth—encounter).

Another example from the classroom: I tell students to think of reasons why the phrase "birds of a feather flock together" is true, and then to think of reasons why the phrase "opposites attract" is true. Again, an impressive ability to support the veracity of the statements is evident—despite the fact that the two phrases are in direct contradiction, both feel completely obvious in isolation.

Social scientists refer to this phenomenon as hindsight bias: once we know something, we are unable to not know it—making it feel as though we've always known it. In fact, now that you've learned what hindsight bias is, you probably instantly had the feeling that you've always known about hindsight bias.

Some of the most brilliant and insightful findings in social science are brilliant and insightful precisely because once they are explained to us, they feel deeply and powerfully true—even though one moment before, we believed that the exact opposite was deeply and powerfully true.

An example. It makes perfect sense that the more someone is paid to complete a series of tasks, the more tasks they will complete. All else equal, who wouldn't work harder when being paid twice as much for completing the same task? Well, it turns out that in some cases, paying people more can actually make them work less. A lot less. In particular, when people love some task because they find it intrinsically rewarding, paying them to perform it "crowds out" their motivation. Students who love to study because they enjoy learning study less when they are suddenly paid for their grades; musicians frequently lose their love of music once they "sell out."

Crowding out seems obvious now, right? But it doesn't to the many people who continue to believe that higher and higher wages will invariably lead to better performance. Just like my students, once we hear one truism (birds of a feather) the whole world seems to align; but when we hear another truism (opposites attract), the world instantly realigns.

The thing is, it's rarely all A or all B, despite our inclination to see the world that way. Under some circumstances, birds of a feather do flock together; under others, opposites really do attract. Under some circumstances, paying more does improve performance; under others, it greatly harms it. Rather than see both A and B as true, social scientists explore the moderating and mitigating factors that allow us to predict when each is true. In part, social scientists test whether—and under what conditions—the theories laypeople have about the world are accurate.

Here's how to combat the "isn't this obvious?" dilemma. When I am ruining a party by describing some new research project, I often first describe the results exactly backwards: "People who we told to do X were far more likely to engage in behavior Y than people who we told to do Z." The response is invariable: "we already knew that." When I then inform people that in fact our results show that it was people told to do Z who are more likely to engage in Y, I can prevent hindsight bias—though inevitably, at the cost of further running the party.

"How can we have this much data and still not understand collective human behavior?"

If you want to study the inner workings of a giant organization distributed around the globe, here are two approaches you could follow—each powerful, but very different from each other. First, you could take the functioning of a large multinational corporation as your case study, embed yourself within it, watch people in different roles, and assemble a picture from these interactions. Alternately, you could do something very different: take the production of articles on Wikipedia as your focus, and download the site's complete edit-by-edit history back to the beginning: every revision to every page, and every conversation between two editors, time-stamped and labeled with the people involved. Whatever happened in the sprawling organization that we think of as Wikipedia—whatever process of distributed self-organization on the Internet it took to create this repository of knowledge—a reflection of it should be present and available in this dataset. And you can study it down to the finest resolution without ever getting up from your couch. 

These Wikipedia datasets—and many other sources like them—are completely public; the same story plays out with restricted access if you're a data scientist at Facebook, Amazon, Google, or any of a number of other companies: every conversation within people's interlocking social circles, every purchase, every expression of intent or pursuit of information. And with this hurricane of digital records, carried along in its wake, comes a simple question: How can we have this much data and still not understand collective human behavior? 

There are several issues implicit in a question like this. To begin with, it's not about having the data, but about the ideas and computational follow-through needed to make use of it—a distinction that seems particularly acute with massive digital records of human behavior. When you personally embed yourself in a group of people to study them, much of your data-collection there will be guided by higher-level structures: hypotheses and theoretical frameworks that suggest which observations are important. When you collect raw digital traces, on the other hand, you enter a world where you're observing both much more and much less—you see many things that would have escaped your detection in person, but you have much less idea what the individual events mean, and have no a priori framework to guide their interpretation. How do we reconcile such radically different approaches to these questions? 

In other words, this strategy of recording everything is conceptually very simple in one sense, but it relies on a complex premise: that we must be able to take the resulting datasets and define richer, higher-level structures that we can build on top of them. 

What could a higher-level structure look like? Consider one more example—suppose you have a passion for studying the history of the Battle of Gettysburg, and I offer to provide you with a dataset containing the trajectory of every bullet fired during that engagement, and all the movements and words uttered by every soldier on the battlefield. What would you do with this resource? For example, if you processed the final day of the data, here are three distinct possibilities. First, maybe you would find a cluster of actions, movements, and words that corresponded closely to what we think of as Pickett's Charge, the ill-fated Confederate assault near the close of the action. Second, maybe you would discover that Pickett's Charge was too coarse a description of what happened—that there is a more complex but ultimately more useful way to organize what took place on the final day at Gettysburg. Or third, maybe you wouldn't find anything interesting at all; your analysis might spin its wheels but remain mired in a swamp of data that was recorded at the wrong granularity. 

We don't have that dataset for the Battle of Gettysburg, but for public reaction to the 2012 U.S. Presidential Election, or the 2012 U.S. Christmas shopping season, we have a remarkable level of action-by-action detail. And in such settings, there is an effort underway to try defining what the consequential structures might be, and what the current datasets are missing—for even with their scale, they are missing many important things. It's a convergence of researchers with backgrounds in computation, applied mathematics, and the social and behavioral sciences, at the start of what is by every indication a very hard problem. We see glimpses of the structures that can be found—Trending Topics on Twitter, for example, is in effect a collection of summary news events induced by computational means from the sheer volume of raw tweets—but a general attack on this question is still in its very early stages. 

And you shouldn't stop there—you can and should ask about the potential dangers of recording, stockpiling, and analyzing so much data on human behavior. In doing so you'd tap into an ongoing conversation bringing together scientists, policy-makers, large Internet sites, and the public, focused on a changing landscape around notions of individual privacy. It's important to push forward on all these questions, because in a world where both the possibilities and the risks are evolving rapidly, the worst thing you can do is not ask. 

"What about me?"

I am willing to bet that every psychologist, no matter what aspect of the human mind he or she studies, has experienced something like the following more than once. You meet a stranger, mention that you work as a psychologist, and then the strangers asks, "Are you analyzing me?"

I dread this question because it reflects both a superficial misunderstanding of what many psychologists do for a living, but it also highlights a deeper limitation in psychological science. The superficial misunderstanding comes from the long shadow that Sigmund Freud still casts on our field. Although many psychologists working as clinicians and therapists do indeed analyze individuals, trying carefully to understand their problems, many work as scientists trying to understand how the mind works just as a chemist tries to understand how chemical bonds work. I, and every other psychologist you'll see on the Edge site, is of the latter type.

This misconception is easy to fix, and the great popular writing of many authors from this very site is shortening Freud's shadow. But I dread this question more because of the deeper issue it raises. "Are you analyzing me?" implies that I, as a psychologist, could indeed analyze you. The problem is that psychological science has, and always will be, a group-based enterprise. We randomly assign volunteers in our experiments to one condition or another and then analyze the average differences between our conditions relative to the variability within those conditions. We do not analyze individuals in our experiments, nor we do know why some people within a given condition in an experiment behavior differently than others. Our understanding of our own research participants is relatively coarse.

All sciences work this way. In medicine, for instance, doctors prescribe drugs because the average outcome of those in the treatment group of a drug trial were better than the outcome of those in the placebo group. Not everyone in the treatment group improved and some improved more than others, but enough got better that doctors think it's likely that you'll get better if you take this drug as well. But as a psychologist, I often field questions that call on me to offer more individualized answers than our science can warrant. I'm asked to give precise advice and recommend exact solutions when what we can offer is general advice and broad solutions that may or may not apply exactly to your particular problem. I dread having to explain all of this to people. In fact, I dreaded trying to explain it in this little essay.

So if you should happen to sit next to me on a train or a plane, I'll happily start up a conversation with you and explain that I'm a psychologist. Just rest assured that I am not, in fact, analyzing you. 

To Jon Kleinberg:

Your story of Gettysburg makes a persuasive case that we have "too much data". But consider this: if I went to one of my colleagues and said, "The real problem of social science is that we have way too much data," they'd laugh at me (more than they already do!).  And they'd proceed to list off dozens (hundreds) of things we have few measurements of. 

So I am torn: do we have too much data or too little data? This conundrum is at the heart of so many of the issues surrounding "big data". Yes, Twitter has a firehose of data. And yes it is "too much" in the way you describe. But isn't it too little in another sense? It seems like a very thin (but very, very long) slice of social life. 

To Samuel Arbesman:

Your essay made me wonder whether or not we are at fault here.  Are we communicating science correctly? If the heart of science is the recognition that truth is transitory, always ready to be over-turned by better evidence, then should we not be communicating it that way? The popularization of science always seems to trend in the other direction, to emphasize the great discovery. 

To Adam Alter:

Adam, your question made me wonder about the Berlin-Kay findings on the universality of color categorization in language? What do psychologists now believe on that? Putting aside defects such as color blindness, is there any sense in which we all categorize colors similarly? Do you know of any psychologists who have a forthcoming book on color that might be able to help us answer this question? 

"My Granny saw a real ghost—how do you explain that?"

For more than twenty years my field was parapsychology. I started out as a wide-eyed student, convinced of the reality of telepathy, clairvoyance, precognition, spirits, ghosts and all things psychic. A few years of increasingly careful experimentation convinced me that most of the phenomena did not exist. So I turned to investigating some genuinely puzzling experiences.

This was far more satisfying. For example, alien abductions can turn out to be sleep paralysis; a very scary experience that can easily be misinterpreted, and the creepy feeling of déjà vu is an inappropriate feeling of familiarity related to temporal lobe lability.

So if someone says "I had an out of body experience – how do you explain that?' I can help them. I can explain about the construction of the body image in areas near the temporo-parietal junction and how research has discovered that disturbances to this part of the brain induce sensations of flying, floating and leaving the body. If they say "I keep waking up terrified and I can't move and I know there's someone—or something—in the room with me. Am I possessed?" I can explain about sleep paralysis; how the normal paralysis that occurs during dreaming can carry on into a half-waking state that includes buzzing and humming noises, crawly feelings on the skin, and sexual arousal. If they say "When I was a kid I used to be able to wake up in my dreams and control them but I can't do it anymore. Why not?" I can explain about lucid dreaming; the physiological state in which it occurs, the people most likely to have it, and I can even suggest ways of inducing it.

But those wretched ghost stories drive me mad. The problem is not that they cannot be explained but that I am always given far too much completely irrelevant information (My Granny's house is a hundred years old and an old man once died there and it was the full moon and she'd been thinking about …..) and none of the information I need. Indeed the information needed is usually unobtainable so long after the event. So I have to reply "I'm sorry I cannot explain what happened to your Granny," and accept the smug look that says "Ha Ha, I thought so. You scientists won't accept that there are more things in heaven and earth ……"

I want to scream. And this is one of the reasons I eventually left the field for good, unwilling any longer to be "renta-skeptic" for all those TV and radio shows; no longer willing to go on explaining why coincidences are not what they seem to be; no longer able to endure the hate mail from true believers or the cruel accusations of "If only you had an open mind ….."