Extremely super newb question, what's L1/2/3 Cashe? I never understood that.
As explained earlier by Lupin and sdedalus, it's essentially very fast (but very small) memory that sits on the processor die (i.e. packaged with the processor), and is used by the processor to store data and instructions that it works with. Data that it can't find in the cache will have to be pulled from main system memory or your storage devices, which slows things down tremendously.
The main difference between processor cache and your system memory is that your caches consist of
SRAM, while system memory consists of
DRAM. Without reading those articles, all you need to know is that SRAM requires more components, is more complicated to make and thus is more expensive (so you only have a little of it). DRAM is simpler and cheaper to make, so you have more of it (just compare the circuit diagrams of SRAM and DRAM to get an idea of that).
In addition, DRAM has the disadvantage of being "leaky" (as it's based on capacitors) so it has to be refreshed every so often. Of course, during this refresh no data can be read or written, which further adds to access delay.
If I've successfully piqued your interest,
Gustavo Duarte has an
excellent writeup of the various things in your computer's processing pipeline that slow it down. In particular, note the latencies involved in accessing cache, memory, hard disks, and the Internet, as well as the respective sizes of the caches. Hope that's enough reading for you

[edit] In case that's not enough, here's a book/writeup discussing it in more detail, and with comparison graphs to boot.
Link:
What every programmer should know about memory (
PDF version)