It is best if you create a new database, make a backup of the original (live) database and restore it in the new database. Then you upload the forum software to a new "subdomain" or folder on your server, connecting the new version 6.5 to the new database.
You run the upgrade scripts and then start applying the modifications you need. Once you are done and happy, all you need is to download the files from the new location and upload them to the old one (live site). Following that you will need to run the upgrade script so that the database is upgraded as well.
You cannot run 6.4 and 6.5 from the same database to the best of my knowledge.
A more convenient way is if you do all this on your computer after having installed Apache (or IIS), php and mysql instead of using the space and bandwidth of your server