In Windows XP, the "Safely Remove" feature would completely turn off power to the associated USB port. (If the device is self-powered, its behavior depends on the device itself, in response to the comments above.)
In Windows Vista and above, this feature has been intentionally removed:
Microsoft Support: The Safely Remove Hardware feature does not stop power from being provided to a USB device in Windows Vista
Cause: The Safely Remove Hardware feature does not stop power from being provided to a USB device in Windows Vista. By design, the port to which the device is attached is not suspended in Windows Vista when the device is soft removed.
This is how you can enable it per device VID&PID or globally:
Microsoft Support: USB Port Remains Active for Disabled or Safely Removed USB Device
Resolution: To work around this problem, a registry value can be set to cause Windows Vista, Windows Server 2008, Windows 7, and Windows Server 2008 R2 to revert to the Windows XP and Windows Server 2003 behavior of disabling the USB hub port to which a Soft Removed USB device is attached. This workaround can be applied on a per-device basis (for all instances of a specific device (Vendor ID, Product ID, and Revision), or globally (for all enumerated USB devices).
[...]
Global Setting
To apply the workaround globally for all enumerated USB devices, add a
REG_DWORD
value namedDisableOnSoftRemove
that has a value of1
to the following registry subkey:HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\usbhub\HubG
Use the following command to add the global setting value:
reg add HKLM\SYSTEM\CurrentControlSet\Services\usbhub\HubG /v DisableOnSoftRemove /t REG_DWORD /d 1 /f