The Imperial Library

The Elder Scrolls 1: Arena, The Elder Scrolls 2: Daggerfall

Only three Constellations appear in Arena and Daggerfall: the Mage, the Thief and the Warrior, in the character generation quiz screens. Every question in the quizzes has three answers, one for each class. Giving a class' answers brings the player closer to that class, which leads to the stars associated with that class becoming brighter. The games have identical character generation quiz screens.

The pictures below are with the stars at their brightest.

The Mage
The Thief
The Warrior

The Elder Scrolls Adventures: Redguard.

The game that introduces the rest of the Constellations: the Apprentice, the Atronach, the Lady, the Lord, the Lover, the Ritual, the Serpent, the Shadow, the Steed and the Tower. It has detailed information about them and even a puzzle. Murals of all the Constellations except the Serpent can be found in the game's Dwemer Observatory and all of the Constellations can be observed from the telescope. There is also a book about Constellations with pictures of the Warrior and his Charges: The Firmament. 

Contellation	The Firmament	Mural	Telescope
The Apprentice
The Atronach
The Lady
The Lord
The Lover
The Mage
The Ritual
The Shadow
The Steed
The Thief
The Tower
The Warrior
The Serpent

In Redguard, the Lord is the Constellation threatened by the Serpent. Therefore, they are in the same part of the sky. I have separated the two. The original image can be seen on the left.

Redguard even had two star maps, one on the ceiling of the Dwemer Observatory and one on a patch of ground in the Yokudan camp.

To help you, here is the correspondence between Dwemer symbols and Constellations. Note that Guardian symbols are blue, Charge symbols are yellow and the Serpent symbol is red.

The Apprentice		The Mage		The Tower
The Atronach		The Ritual		The Warrior
The Lady		The Shadow		The Serpent
The Lord		The Steed
The Lover		The Thief

On the Yokudan Star Map, the squares represent the Guardians and the circles represent the Charges.

In addition, by combining all of the Telescope images, one can create the star map below.