The chain of events leading to this skin change starts with a stimulus such as cold or fear. That stimulus causes a nerve discharge from the sympathetic nervous system, a portion of the autonomic (involuntary) nervous system. The nerve discharge causes contraction of little muscles called the arrectores pilorum (the hair erector muscles). Contraction of these muscles elevates the hair follicles above the rest of the skin. And it is these tiny elevations we perceive as goose bumps.
The words used to describe this condition are very curious and quite colorful. Goose bumps are also referred to as "gooseflesh." A fancier term for this familiar phenomenon is "horripilation." Horripilation was compounded from the Latin "horrere", to stand on end + "pilus", hair = hair standing on end. (If you think "horripilation" sounds horrible, you're right. The word "horrible" also came from the Latin "horrere" and referred to something that was so awfully dreadfully frightful that it made your hair stand on end.) Medicine does not use a horrible term such as "horripilation" and rarely resorts to the commonplace words, goose bumps or gooseflesh. Medicine has a special term, "cutis anserina" for goose bumps. But it goes back to the goose again, since "cutis", skin + "anser", goose = goose skin.
Some biologists believe that goose bumps evolved as part of the fight-or-flight reaction along with heart rate increases that send the heart racing while blood rushes to the muscles to give them additional oxygen. A similar phenomenon, bristling, in fur-covered animals may have made them look larger and more frightening and kept them warmer by increasing the amount of air between hairs which traps body heat. But in people there seems to be no practical purpose for goose bumps except, of course, to make our skin crawl.