Heh, if you think the indexes in 6.3 are redundant, wait until you see the index files we use in 6.4...
We store serialized Perl data structures rather than extruded data for two reasons:
1) Parsing time. OpenFileAsArray & split()ing results in massive slowdowns. Parsing a Perl data structure is a comparatively lightweight function
2) Simplicity. It's far easier to RequireVars (thus do()) something.