Make sure to delete
/Library/Audio/Apple Loops
/Library/Application Support/GarageBand
Note the slash before, it's the Library on your Macintosh HD's root, not the one in your home folder. Beware that this will also include sound effects for iLife and could have unforeseen consequences.
The new GarageBand will require you to download all additional content – the App Store version just provides the basic application if I recall correctly.