Do I have to keep the sigil stones in the game Oblivion?

Hi everyone! I was curious about a question. When you destroy a Oblivion gate, you have to take the sigil stone out of its place from the top of the main tower. Now do I have to keep them? Or can I drop them right when I get out? Will they reappear again if I drop them???