I haven't seen that behaviour before. 9600 8N1 seems reasonable, at least those are the default settings for a Brocade 6510.
What happens if you power-off the switch? Does the terminal emulator continue to see output? If so I would suspect that some other service on the Ubuntu system is configured to use the same port e.g. a login or getty process, and this might be interfering with your communication to the Brocade. I don't have any Ubuntu systems here, but UNIX/Linux login/terminal sessions are typically configured via /etc/inittab.
Alternatively, Brocade have a setup tool called EZSwitchSetup (really) which uses Ethernet based communication to connect to their switches. However I think that this package requires a Windows client system to install onto. I'm not aware of any Linux based version.