How did they come up with all the different letters of the alphabet?

Answer 1

I'll try to summarize something about the ORIGIN of this whole "alphabet" system. After the origin comes a lot of borrowing and adapting by various cultures, which is something that can't be spelled out in great detail here, though if you run a web or wikipedia search for each of the letters (or even look a the beginning of the section for each letter in a good dictionary) you can get a lot of that.

ORIGINS

The origins of our alphabet, and even the origin of the order of the letters, is hinted at in the very WORD "alphabet". This word comes from the first two letters of the Greek's system -- alpha and beta.

But these Greek names are rather odd. In fact, in Greek they are meaningless! That's because the Greeks borrowed both the letters and the NAMES for many of the letters from the seafaring traders of Phoenicia (who were active throughout the Mediterranean).

The Phoenicians did not invent this system themselves, but were part of a cluster of related "Northwest Semitic" languages spoken centered in the regions later called Syria & Palestine (a group that, incidentally, includes Hebrew and the Aramaic dialects) who had been using this system for centuries. The letter-names are derived from actual Semitic words which BEGAN with the sound that letter is used to represent (e.g., "aleph", meaning "ox", "beth" meaning "house").

Many believe the inventors of the alphabet were speakers of a Semitic language living in or near Egypt around 2000 B.C.. So it is possible that the IDEA for an alphabet owed something to a part of the system of Egyptians hieroglyphics. (They might also have known something of the cuneiform system of writing developed in Mesopotamia and spread through most of the ancient Near East.) In that case, perhaps the groupings of the letters and words may also be suggested by something in Egyptian learning and culture, or at least in the culture of this one group. But unfortunately, all evidence of how this might have happened is lost to us.

BUT we CAN see from some of the ancient letter forms that many if not all of the letters originated as simplified pictures of those words. (For example, the earliests forms of the letter "mem" --used for the m sound -- look like waves of water, and the words "mem" MEANS "water".)

---------------------

ADAPTATIONS - New Letters / Sounds

Now that does NOT account for ALL the sounds of our modern letters, or even for all the letters themselves. That's because, as soon as the Greeks borrowed this system they modified/adapted it for their own use. The Greek language had a slightly different set of sounds from the Phoenician they were borrowing from, so they might drop a letter or take a letter that represented a sound Greek did NOT have and use if for something else, including for VOWEL sounds which the Semitic alphabets did not mark. They also ended up inventing some NEW letters to fit their needs. (Both these sorts of changes have been made in later alphabets as well.)

CHANGING FORMS

There are other later developments in Greek, Latin, etc. that further changed the specific forms of the letters. For example, our modern letters "J" was originally an alternate form of "I" used in certain positions in words. Similarly, the form "V" ended up giving us "U" and "W". In both cases, the alternate of 'biforms' came mostly to be used to indicate their own sounds (though you can still see places where "w" functions the same as "u", e.g., the W of "how" is equivalent to the U of "house").

(Note that, when letters start as variations of other letters, they are usually placed right next to the original letter in the alphabet. When OTHER letters are added --included when a new form of an old letter is brought back-- it was been added at the END. So, the Semitic alphabet, and the earliest Greek ones ended with the "T" letter ['tau']; everything after that is an added or re-added letter.)

Of course, the idea of connecting the letter forms with pictures was left behind a LONG time ago, long before the Phoenicians handed their form of the system on to the rest of the world. So the actual letter forms have been stylized and simplified many times till the specific origin of each can scarcely be seen.

Also, the original letters were originally all "capitals"/"upper case". Our modern "lower case" letters developed by a series of steps beginning with Latin cursive forms (adaptations of the letters made to make them easier to write more quickly). For the most part we can still see the relationship between our upper and lower case letters, so this process is not too difficult to understand.

Answer 2

Imagination. All these letters we use are the newest version of a series of characters that initially were images of words beginning with the respective sounds.
http://en.wikipedia.org/wiki/alphabet
http://en.wikipedia.org/wiki/Phoenician_alphabet
http://ask.metafilter.com/35550/Why-A