Nassim Nicholas Taleb of the Black Swan fame looked like he confused his Jaipur Literature Festival audience more than clarifying their doubts with his pet theories of uncertainty, randomness and antifragility. Edited excerpts of his replies to audience questions:

Big Data is a Nightmare

Big data is a nightmare; I agree that data works in specific cases. If you were the NSA or very focused on one question and you go to data, you may be able to get an answer. You might get an answer if you think there is a prevailing theory; you may be able to use big data.

Let me tell you little about noise and how people are naïve when it comes to big data. I testified against funding people for using data, they wanted a few 100 million to forecast some crisis, I went to testify,  there were a few bunch of democrats against me, they wanted to fund as they felt a centralized observatory would be able to  detect the next crisis. The fellow before me showed what he can do with big data. I asked him, do you realize that before the 2008 crisis we had 600 million correlations in finance, what did they do with this data? He said they were missing something. They are always going to be missing something.

We never had more data before the crash yet it was totally unpredictable. The more correlations you have, what we have now is a tragedy of data as people think data is good. We have a lot of variables but very few data points per variable. We should have the opposite, small and large dimensionality problem. To give an example, every time I add a new variable you have correlations to so many different variables, it swells exponentially. 85% of the research is false because correlations rise to the surface when look at data. But on the other hand in a controlled experiment 8% of the results are okay. In a controlled experiment, a small data set is okay.

I wrote Fooled by Randomness about it 15 years ago. I need to read it often otherwise I fall into a trap. If you have high frequency data you will only see noise.

How to help people deal with uncertainty is decrease the number of people in higher education. Because it’s an S curve, like the Indian prime minister (Narendra Modi) said ‘I rather have 1000 schools than one university. The more educated the person the more incapable they are at handling uncertainty.

How can we apply the black swan idea in personal life or in uncertainty?

Make a list of 95% of what shortens your life expectancy, like don’t ride on motor cycle in India, don’t smoke, if you apply these you will reduce the problem by 95%. Take risks elsewhere. My whole idea is not to reduce your risk taking but channel it where it’s more entertaining or more lucrative.

On Climate Change

People fail to understand this about climate change, the more uncertainty we have about the models, the more we should be concerned about. Discrepancy means things are not clear so the forecast on climate gets a lot worse. Let’s not mess with Mother Nature.

It comes back to the idea of fragility, if you mess with Mother Nature, if you go too far, non linear, it gets a lot worse. So the best thing is to avoid these debates on climate change and act as if the worst is going to happen. If people exercise the same logic as they do with climate change as they do with buying car insurance, where they don’t have an estimation of the risk. You cannot afford, the climate is way above everything we worry about the planet. The GMO (genetically modified organism) is a very big problem underestimated by so-called scientists. With climate we should be doing something immediately, to reduce pollution, or spread the risk across other pollutants.