The temple (pura in Balinese) is built at the edge (ulu) of a 70 meter high cliff or rock (watu) projecting into the sea. In folklore, this rock is said to be part of Dewi Danu’s petrified barque.
Though a small temple was claimed to have existed earlier, the structure was significantly expanded by a Javanese sage, Empu Kuturan in the 11th Century. Another sage from East Java, Dang Hyang Nirartha is credited for constructing the padmasana shrines and it is said that he attained moksha here, an event called ngeluhur (“to go up”) locally. This has resulted in the temple’s epithet Luhur.