how does an optical illusion work ?

Answer 1

There are two basic kinds of optical illusion. I don't know if the categories have accepted names, but the first, I'll call "objective" and the other "subjective" since those names seem to fit. The catagories are not mutually exclusive, and many have aspects of both.

An objective illusion is an image that is geometrically similar in appearance to something else that is more common or easily understood and, therefore, expected. An example of this is the pencil dipped in water at an angle. The image of the pencil projected the retina is indeed the same as that of a broken pencil, which is more familiar. That is, the light rays are actually bent by the water interface.

A subjective illusion is one which exploits the psychology of visual perception (how the brain works). An example of this is the wavy pencil. It looks like it bends like rubber because of the relative delay in the brain's processing direct versus peripheral vision, but the image on the retina is actually straight.

Here's a reference that has a different way of catagorizing them:

Answer 2

There are many different kinds, can you be more specific?

There is an optical illusion when looking into a body of water at an object such as a fish or a rock. The light bends at the plane of the surface of the water. The illusion deceives the viewer as to the location of the object. It would appear to be in a slightly different location.

Another optical illusion is the wavy appearance of a distant object when viewed through masses of different temperature air. An example would be looking across the surface of a hot parking lot at a distant object.

Optical illusions would rely upon the fact that optical physics create the illusion not human perception.