Many people start out with some sort of box or jar with a lid. While this keeps them all in one place, it doesn't stop the coins from scraping against each other and tarnishing each other. There are many ways of storing coins to protect them from damage.
The cheapest way is small (usually 2" x 2") paper envelopes that hold each individual coin. The envelopes should be made from acid-free paper, since the acids found in regular paper envelopes can discolor a coin over time. Acid-free envelopes are available in coin shops or through an archival supply store. You can also get them at office supply stores but those won't always be acid-free. The advantages of envelopes are they're cheap, you can label each one, and each coin can have its own one instead of dumping them all in a jar or box.
I like being able to see the coins without removing them from their covers. To do this, you can use "flips" (a two-pocket piece of plastic that folds in half so the top half flips up), or small plastic re-sealable baggies. Flips are a little more expensive, but I like being able to see the coins without handling them directly. The problem with these is that some plastics are made with a chemical called PVC that can also discolor or permanently damage coins over time. If you do get a plastic holder for them, make sure it's mylar or some plastic that does not contain PVC. Coin shops and coin supply stores sell these, as well as some archival supply stores. Also, hobby stores will sell plastic covers for baseball cards with no PVC. They make good temporary holders for coins, but since they're open at the top to slide the card in, air and moisture can still get in and corrode the coin over time. (There is some debate about whether or not Ziploc bags will damage them. Most experts agree that they're safe for short term storage but not suitable for long-term storage.)
Another option is cardboard or plastic coin albums. These are very handy for those who collect series of coins- many different years of the same size coin. Cardboard albums are often made for a specific kind of coin (cents, nickels, dimes, quarters, etc). Plastic album pages have a certain number of pockets, each of the same size. Some common sizes are 2" and 2 1/2" pockets. A page may have 16 pockets or 25 pockets if they're small, or only 9 pockets if they're large. You can buy album pages that are PVC-free, but they tend to be expensive.
For my own collection, I use PVC-free flips to actually hold the coin. The top half of the flip holds a small card with a description of the coin inside. I then put the flip into an album page; the album pages do contain PVC, but this way they don't actually touch the coin. The coins are sorted alphabetically by country, so I put a letter divider page with a tab on it at the start of each letter. The pages are stored in a 3" 3-ring binder with D-shaped rings so that the pages lie flatter than they would in a regular 3-ring binder. The 3-ring binders you can get at any office supply store. I found that for my collection, this was the best compromise between cost, availability, and protection. If you have more valuable coins or a bigger budget, you may want to go with some of the higher-end holders like Air-Tite or slabbing.
For the most protection, a coin can be "slabbed", or encased in hard plastic, often done by a professional who also grades the coin and includes the grade in the label. However, a beginner coin collector will most likely not run into a coin that's valuable enough to be worth slabbing.
Silica gel is not actually a gel, but small hard crystals that absorb moisture. They are strongly recommended for people who store coins in safety deposit boxes or who live in humid areas. Humidity can ruin coins so putting some silica gel in the box that the coins are kept in will help preserve them. A small amount is all that's needed. It can be bought at coin supply stores and gun supply stores since it's used to prevent guns from rusting in storage. Again, most beginners won't need this right away but it is something to consider as your collection grows or if you live in a humid climate.
However you decide to store them, security is an issue. Collectors with a very large and/or valuable collection may put them in a bank safety deposit box, but most beginners won't need to take that precaution. Just keep them out of plain sight, maybe in a locked box or drawer. If you have any really valuable pieces, don't mention it too much and keep them in a safer place than the rest. Besides helping you to organize your collection, a detailed inventory may help police identify your coins in case of theft; they can check local pawn to see shops if someone tries to sell them. Realistically, though, coins that are stolen are very rarely recovered.