Bus numbers 102, 383, and 390 from the main Puerto de la Cruz bus station in Calle Hermanos Fernández Perdigón, 1, all go via the Taoro bus stop (outside the Hotel Parque San Antonio), which is less than 100 metres from the library.
Coming into town, the 102 comes all the way from Santa Cruz, and so is an excellent way of travelling in from that direction. The 353 comes from the bus station in La Orotava.
By taxi, ask for La Biblioteca Inglesa, Calle Irlanda 5.