As I explained before, the reason is because an individual's willingness to rebel is essentially private information, and nobody is going to tell you what they really think in an authoritarian society. Furthermore, an individual's willingness to march openly against the regime depends on what he or she thinks others will do, and that cannot be ascertained in advance either. But when conditions are right and some triggering event occurs (which can be almost anything), then you can get a rapid and unexpected revolutionary cascade, as more and more people decide that it is safe to express their previously-concealed resentment and that doing so is likely to succeed.
Instead of being angry with the U.S. intelligence agencies, therefore, Obama should be reserving his ire for his foreign policy advisors, who have been screwing up U.S. Middle East policy for over two years now and who may be in the process of snatching defeat from the jaws of victory yet again. If the news reports I've seen are correct, the United States is now getting behind a political transition that will be orchestrated by the new Vice President Omar Suleiman, a close Mubarak associate. It's not even clear if the United States now thinks Mubarak has to step down. Instead, Secretary of State Clinton seems to be suggesting that we need to help VP Suleiman "defuse" the street demonstrations, which would remove most of the impetus for change.
An unnamed "senior U.S. official" has also suggested that the Obama administration is dead set against a substantial political role for the Muslim Brotherhood. Indeed, the official reportedly suggested that what the United States wants is a purely "secular" government in Egypt (i.e., one with no Islamist influence) as if that's even possible in a country that is overwhelmingly Muslim.
It's early days, of course, and as FP's Josh Rogin reports here, there is a potential legal nightmare trying to revise Egyptian law in ways that would permit a genuinely "free and fair" election. But I worry that the Obama administration is about to repeat the same mistake that the Bush administration made in the Palestinian legislative elections of 2006. After insisting that the elections be held, the United States simply refused to accept the results of the elections when we didn't like the winner (Hamas). Are we now going to keep our thumb discreetly on the scale in Egypt, to make sure that a post-Mubarak government continues to dance to Washington's tune? When will Washington learn that you cannot simultaneously proclaim your commitment to democracy and freedom and then insist on dictating who is allowed to win?
The other problem is that Suleiman doesn't have much (any?) credibility as a steward of democratic change. I suggested a couple of days ago that one way he could bolster his position would be to help push Mubarak out (and to make it clear that he is doing so), and to openly declare that he (Suleiman) will serve only as a caretaker and not run for office himself in the next election. I'm not at all sure that these measures would work, however, and the anti-government forces might well see him as no different than Mubarak himself. That certainly seems to be their reaction thus far. And if subsequent reforms are mostly cosmetic and individuals or groups associated with the old regime end up retaining power in a subsequent election, they are likely to have no more legitimacy than Mubarak has right now. And the U.S. image in the region, which is bad enough already, will take another big hit.
So the United States has two long-term challenges. The first is to make sure it is not once again perceived as working to quash a genuinely representative government in Egypt. The second is get ready to accept the results of that process, even if the people we might prefer don't win.