If you are looking into the realm of learning to develop cgi applications on that box, then by all means go with Activestate's Perl, Apache for win, php 4, and mysql.
Apache runs just fine on windows (though better as a service on win2k or NT), and is a far easier (IMO) envrionment to develop on. The windows version of Apache is questionable (stability-wise) for a production environment, but for dev I've never had ANY issues. Since apache is the most popular web server, it's best to learn that from the start. IIS just annoys me (others feel very differently about that -Cal :D- but Apache is the defacto standard)

PHP is much easier to setup on windows, as is mysql. Click install. That's about 99.9% of it.
My windows box runs Perl 5.6.0, Apache 1.3.14, mod_perl 1.24, PHP 4.04pl1, and MySQL 3.23.33 with no problems, and all of it was a cinch to get in and running.
Perl modules are very easy to install, using Activestate's ppm script. When that doesn't work (no module available in their ppd format), I downloaded nmake which works just like make on unix, and I just install the modules like on a normal unix box.
I far prefer Unix enviros, but it's much easier for me to develop on my win box (using my tools) and move my stuff over to unix when done.
--mark
--mark