Just getting caught up. Yesterday was very busy!
Yes that power adapter will work just fine for LED's. As the others have said, you will need resistors for the LED's or they will last about 1.3 seconds.
The one thing I would worry about is the output voltage. These types of adapters have specifications to them that allow for some tolerances in the voltage. So it might be listed as a 12V output, but if the tolerance allows for +-15% it could actually be outputting 13.5 V or 10.5V. If it has a high output, you will just burn up every LED you hook up to it.
I would invest in a multi-meter pretty fast if you are getting into working with LED's. That way you are sure of your voltages and your resistors. Also, if you continue to work with electronics, a bench power supply will become a happy option soon.