The term refers to the bus width of the CPU. CPUs with a bus width of 64 Bits can access a considerably larger address space (i.e. more memory, in theory 264 GiB = 16 billion GiB = 16 EiB (ExbiByte)). Moreover, 64 Bit CPUs are significantly faster, because 64 rather than 32 Bits can be transported in a single cycle. There are no longer any 32 Bit CPUs in the HPC field.