The IBM PC was powered by an Intel 8088 processor.
That processor could address 1Mb of memory. Yes, a whole luxurious megabyte! Trust me, that was a big figure in 1981.
However, you didn't get the whole megabyte. Oh no. Things like video cards and the built-in BASIC need some RAM to use, so IBM reserved the upper 384Kb of RAM for that purpose. That gave the machine 640Kb of RAM to play with.