According to some authors, its name may be derived from the worship
of Castor and Pollux in the Roman period.
Though there are archaeological remains to be found in the village -at El Tajo Algarin there is a collection of dolmens and prehistoric tombs, the highlight of which is the Giant"s Dolmen (Dolmen del Gigante)- which bear witness to earlier settlers, the first signs of a village date back to the 16th century, when a group of natives of Zahara inhabited what is now the village street of Calle La Fuente with the intention of cultivating the adjacent land. It was around this primitive nucleus that the present-day village appeared. In 1834, El Gastor became independent from Zahara.
The village is the home of the "El Gastor Fandango" an unusual form of flamenco song.
Jose Marin Gonzalez, "El Niño del Gastor" (The Child from El Gastor). Flamenco singer.