If you want to use a small battery, some of the button cells would work fine.  I think a CR2032 would work, and it won't take up much space.  It is a 3.2 V battery so you won't need resistors if the LED's are white.

It just won't power the LED's for a long time.   So don't leave the switch on.