You shouldn't need to ever prune the files in the folder unless they're taking up a large amount of space as they should auto overwrite when the associated URL is accessed.
The cache is in place so that your backend database server doesn't get pinged to death with data requests, this cache by default is 24 hours, but if you for whatever reason wanted to have it update more often or less often you have the option to do so with offsetting the cache parameters previously mentioned.
If you have cache files from malformed user or bot requests you can purge them manually if you wish, but any request to that same url will generate a new cache file (or update said cache file) after the request completes.
Most of these files are mere bytes long, larger files for larger forum indexes. All we're really fetching are formatted URLs for existing content so I doubt it's going to make much of an impact to purge files.
But it should be 100% safe to purge any files located in the cache directory, but we recommend keeping the index.php file to redirect people/bots to the web root.