The building has not always showcased works of art. Indeed, it was built in 1852 as a winter shelter for the orange trees that lined the garden of the Tuileries Palace. Before this, the orange trees were housed in the Grande Galerie of the Louvre.
At the request of Emperor Napoleon III, the new structure was built on the garden terrace along the Seine, known as the "waterfront terrace", in a record time of four months and according to the plans drawn up by the architect Firmin Bourgeois (1786-1853). The structure closely resembles a greenhouse: its southern façade facing the river is made of glass to let in the light and the heat from the sun. The opposite façade, facing the rue de Rivoli, is almost entirely windowless so as to avoid the north winds.
The main entrances are located on either side, to the west and east, and were decorated by Louis Visconti (1791-1853), the architect responsible for the renovations at the Louvre. The doorways framed by columns echo the decoration of the Tuileries Palace. They are topped by triangular pediments sculpted by Charles Gallois-Poignant, representing cornucopias, plants and ears of corn in connection to the site's function.
Following the fall of the Empire in 1870 and the fire at the Tuileries Palace in 1871, the Orangerie became the property of the State. It continued to be used to store the orange trees and as the setting for various events: horticultural, musical and artistic shows, banquets, contests, dog shows, etc. until 1922.