I will show you my my.cnf that has specs similar to yours, and it works flawlessly. Took a bit of tweaking and testing for a few days, but I think I have found the optimal config
[mysqld]
max_connections=500
skip-innodb
query_cache_limit = 1
query_cache_size = 32
query_cache_type = 1
interactive_timeout=100
wait_timeout=100
connect_timeout=10
thread_cache_size=128
key_buffer=16M
join_buffer=1M
max_allowed_packet=16M
table_cache=1024
record_buffer=1M
sort_buffer_size=2M
read_buffer_size=2M
max_connect_errors=10
Now.. one more thing to note which is fairly important, but I assume you are using mysql4 correct? Because the query_cache is new to 4, so if you were using 3, it would be pointless to add those.
there are other options you can setup as well to tweak things like mysqldump and databsae repairing and such in this file... but the options I choose there will give you hopefully the most performance improvements up front.