It would likely bounce off! The envelope fabric is much tougher than it might appear. It is possible to fly a balloon with a hole large enough for a man to go through as long as the hole is not at the top of the envelope.

The chances of a bird getting close enough to fly into a balloon are very slim. While some birds have been seen flying near a balloon, the noise of the burner, size of the balloon and presence of people in the basket keeps a bird from getting too close.