What happens to a material which undergoes plastic deformation?

It does not return to its original length/shape when the force is removed.

To add to James' reply, when you apply a force to extend the material you do Work on it (Work = 1/2 x Force x extension). Whilst the material is deforming elastically the work is stored in the material as elastic potential energy.

When you exceed the elastic limit, any extra work now goes to deforming the material permanently- it takes energy to re-arrange the microstructure of the material. The energy that was stored as elastic potential energy is still largely recoverable, depending on the material.