400 gigs of hard drive = 320: why?

Question

Okay, we all know the "disappearing gig" act of any large hard drive and are accostomed to it. I know it has something to do with formatting but have never really understood it.

Why does a 400 gig external harddrive lose 30 gigs when installed on a system? Why does a 30 gig iPod only have 22 gigs of storage? Is it even POSSIBLE that the OS to run a harddrive is 30 gigs?? I find that incredibly hard to believe. So where do all those gigs go? Thanks!

What I mean is, how can the OS to run a harddrive be 30 gigs when a complete OS like Windows is not nearly that large?

Answer 1

Actually, you didn't lose it. You just don't do math like a computer does. Computers think in binary (everything is a zero or one) and they don't use the base 10 decimal system like we do.

A bit is one zero or one.
A byte is 8 zeroes or ones.
A kilobyte is 1024 bytes (not 1000 bytes as is commonly believed).
A megabyte is 1024 kilobytes and so on.

So doing some math here, a true 320 Gb drive would actually store 343597383680 bytes. So in this case you might buy a drive whose manufacturer said it was a 340 Gb drive and it would format and you would see only 320Gb of space.

Another reason bytes disappear is that the BIOS on some systems can't keep up with the new larger hard drives in terms of calculating the space for all those heads, sectors, and cylinders. When that happens, the operating system picks something as close as it can support and when it formats the drive, you lose a fair amount of space. Most drives will keep about 8Mb out of the drive for the system to store drive configuration information.

Here's an even more authoritative answer from my favorite folks at Acronis:

"Nothing is wrong, exactly; you've just stumbled onto one of the facts of life about today's computers: The difference between hard disk size and usable space.

As a general rule, you can expect to lose about one-quarter of the space on a disk drive to things such as formatting and hidden partitions. You'll probably lose a smaller percentage on a smaller disk and more on a larger one. You'll lose toward the high side if the disk is the main disk on a system that came with the operating system installed and toward the low end if it's a second or third disk you've installed after you bought the computer. In other words, if you got 225 Gb out of a 250 Gb disk, you beat the guesstimate.

That lost capacity goes several places. One of them is the difference between formatted and unformatted size. Hard disk manufacturers quote the unformatted size of their products, but you have to format the disk to use it. This process of dividing up a disk into blocks and setting up the tables and such to record and manage each block takes space.

Then there are the hidden partitions. On computers that are shipped with the operating system installed, manufacturers have already installed one or two partitions containing a complete system image, boot information, and even parts of the Windows operating system. They do this in part because it makes computer troubleshooting easier and in part because it keeps you from using your recovery CD to copy Windows onto other computers in violation of your licensing agreement.

Should your primary (system) partition be destroyed due to a virus or malware, your computer vendor might opt to use the image on the hidden partition to restore the system to the way it was when you first got it. You should have a backup plan in place where you regularly create an exact image of your hard disk. That way, should your disk fail, you can restore the latest image.

In cases when no backup image exists, some vendors provide a final, fall-back strategy. When you restore a base image from these hidden partitions, all the data, applications and configuration you added are lost. It is no exaggeration to say that using an image from a hidden partition should be a last resort for repairing a drive.

However, as mentioned, this is also a way of enforcing Microsoft's licenses. This might be convenient for Microsoft, but it is not beneficial for the user. Customers should always ask their systems vendors to provide them with a copy of the operating system on a CD, along with a valid registration key. That way, if you have to restore the operating system, you can do so.

Finally, in some cases, the capacity wasn't there to begin with. If you want to get technical about it, a KB should be 1024 bytes, not 1,000, because you're dealing with binary (base 2) notation. But when most people, including marketers, say "Kb," they mean 1,000 bytes. Similarly, a megabyte (MB) should be 1,024 KB, not 1,000, and so on. That terminological leakage adds up by the time you get into multiple gigabytes. However your computer reckons such things as disk capacity in binary and to a computer, a KB is 1,024 bytes. This is the standard practice in the industry and even though there has been a lawsuit filed challenging the "truth in advertising" over the 1,024 vs. 1,000 byte debate, it's not likely to change.

There is not a lot you, a user, can do about any of this capacity. The formatting loss in inevitable, the terminological difference is embedded in our system and messing with hidden partitions just to get more disk space is not a good idea.However, understanding why your disk storage capacity isn't 100% as the advertisement says, should be a comfort."

Answer 2

I think I can explain at least part of the reason some space is lost after formatting.

Imagine you have a file cabinet with lots of folders in it. You labelled all the folders with numbers from 00001 to 65536, but that is the only label on it.

Next, you put papers in the folders containing your tax bills, car repair bills, insurance policies, instruction manuals, travel maps, and maybe some letters from a high school sweetheart. When you put these in, you just put them in the order you received them, so they are not sorted alphabetically or by any logical order.

How do you find the travel maps if you don't remember they were put in folder 01401?

What the operating system does on a computer (and I'm sure on an iPod too) is reserve some space for an index about where you can find certain items. So when you say "find me the travel maps", it looks down its list for "travel maps" and finds these, then goes to the actual folder and pulls up that stuff for you.

There is also an area reserved for what is called a bad block table. This second index keeps track of what spots are available on the system and which were flagged as "unusable" because of problems reading or writing to that area.

There may also be a table that keeps track of what spots on the disk are used and what spots are still available. When the system has something to store on disk, it figures out how much space is needed, then looks at the table to see what areas can hold that data. Once it determines an area is to be used, it flags it for storage, saves the data, then updates all the appropriate tables so everyone is up to date.

On a small hard disk or iPod drive, the space required for these special tables and indexes are smaller because they do not actually have the room to store a lot. The larger disks have to reserve more space for this because they can hold more info.

Hope this helps.

By the way, on a side note, you should also be aware that not only does the larger hard disk need to reserve more space for the areas I spoke about, they also have a tendency to use up the space faster than smaller disks or smaller partitions. Imagine someone gives everyone in a class a full 8.5 X 11" paper to write on, but all you needed was to sign your name. They can't give your page to someone else, so all the extra space is reserved and wasted because you had such a small amount that needed to be stored. This same concept applies to computer storage. If you have a 20K file to store but each cluster on the disk is 8192K in size, you may have just wasted 8172K of space.

Answer 3

Thats whats so confusing. The OS can't possible take up so much space; XP only takes about 1-2 gig, depending on the version. I've heard that when they say it's 400 gig on the box for example, it's real TOTAL space is about 6 gig less for an 80 gig hard drive. So then some space is used up by the OS, so the 30 gig could be for other programs.

Good luck!

Answer 4

Hehe, it's an old manufacturing trick. Hard drives have what are called checksums, these are numbers attached to each byte to verify that the correct data was written. There's a formula to compare each byte with its checksum and possible repair the data, or mark it as corrupt. So, typically each byte is 8 bits. There are usually 2-6 checksum bits, so about 10-14 bits per byte. Manufacturers calculate total bytes including checksum bytes, which you can't use for data. So there's about 2-6 bits per byte missing from your hard drive size. Windows will only set aside several megs of storage, and the OS is only about 1-3 gigs, so the OS isn't stealing your extra space.

Answer 5

The OS likes to arrange a drive into the format it is most comfortable with. Consider your desk:

Suppose your desk has 30 square feet of space on it. At first glance, it would seem like you would be able to put 30 sq ft worth of stuff on it, right? Well, no. Because after you have arranged everything on the desk the way you like to use it, there isn't as much room left. That is what your OS is doing when it formats the hard drive. It is arranging everything in the way it likes it to be, and a little bit of space gets lost because of it.

Answer 6

well for a start the hard drive is prob not entirely empty. Maybe you have windows installed on it taking up ~5 GB.

Then the main reason is dependant on your filesystem. This is basically the layout of the disk as viewd by your operating system.

In order to be able to immediately go to data on your filesystem it keeps a table, tree or similar data structure which indexes the filesystem, and this overhead is what is eating up your space.

Answer 7

Yes, it is possible, even probable.

Go into disk management (right-click My Computer --> click "Manage" --> click "Disk Management".

From there you can view the hard drive pathways, including the active partition (probably C), and any logical drives on the extended partition...possible D.

These will tell you how much space has been allocated. You may have some unallocated space that isn't being used. That can be fixed using "Dynamic Disks" (if you have XP Pro) or Partition Magic.

Answer 8

This :
http://www.dewassoc.com/kbase/hard_drives/hard_drive_size_barriers.htm

Covers more than I suspect you want to know, but it explains some of the basics about drive format-ing and drive limitations.
It also has some great links to drive format. storage and info.

Answer 9

Your BIOS may need to be flashed to support 400GB, because it currently supports no more than 320GB. That's the most plausible reason why hard drives register less storage space than they actually have.