A portable program largely has to do where the data is saved for that program. Usually user data is saved in the username\AppData\ folder as per Windows policy. However, it doesn't make it very portable as you would lose all of your settings/user data associated with that program if you were to blindly just copy the program files from their folder in program files.
Another problem is that a program may choose to store settings and information in the registry. If you were to copy the program files to another computer there would be no registry entries to read from, so the program would likely just fail.
One last problem is that a program may choose to use shared libraries that will not be present on the system that it is copied to, thus making the program fail. Often times, especially with some open source software, an install of a program warrants an install of another program such as the GTK runtime environment or DirectX. If GTK/DirectX wasn't installed (or even if the correct version was installed!) on the copied to OS, then the program will fail.
So, for a portable release to exist, it must combat all of these deficiencies. Usually the settings are saved locally to the same folder as the program, there are no registry entires (say, they use the same folder for those settings as well with a SQLite database which is what Chrome uses). Also, programs that require an install of other software aren't likely to be made portable unless there is someway to include the installed software and the libraries in a portable manner.
For more information visit Wikipedia.