I'm working on a script which pulls tide data from NOAA, parses it and the inserts into my PAL page. You can see it working at http://www.fish-tails.net
Anyway, I need a little help on regexp, and maybe some suggestions.
Ideally, I would like to pull the data from NOAA, strip html completely out, and then insert the data into mysql. Once the data is stored, I can then write a script so that the user can select the reference stations they want to be displayed on the home page, or in their My Home.
Currently, I have it loading directly to the home page without stripping html. Here is the code I'm using.
echo "I'm running some tests now on the tides program I'm writing. Please use these with discretion. Thanks. DATA from NOAA"; echo "<br><b>Simpson Creek, A1A highway bridge 2002 Water Level Predictions</b>"; echo "<br>(Reference station - Fernandina Beach)"; echo "<br>Corrections - Time Diff. (High 0 hr. 05 min. Low 0 hr. 19 min.)"; echo "<br>Hgt. Diff. (High *0.84 Low *0.65)"; echo "<br>All times listed are in Local Time, and all heights are in Feet referenced to Mean Lower Low Water (MLLW)."; echo "<br><hr><table><tr>"; echo "<th width="60">Date</th><th width="60">Time</th><th></th><th width="40">Ht.</th><th> </th><th>Time</th><th></th><th width="40 ">Ht.</th><th> </th><th>Time</th><th></th><th width="40 ">Ht.</th><th> </th><th>Time</th><th></th><th width="40 ">Ht.</th></tr><tr><td>";
echo "<font face='arial'>";
echo $head[0];
echo "</td></tr></table>";
You will note I have two $url with one commented out. The one that is commented is the one I think woul be the easiest to work with, but then I'm not sure, which is why I'm here.
The second one is the one that I'm using on my home page. Now, first off, I only want 4 days displayed, but if you look at my home page a new line with a fifth day starts. I've tried preg_replace and ereg_replace but to no avail. Anyone have a good understanding of regexp that can help me out?
Regarding the database. Looking for suggestions here. Should I set up a cron (I have an idea what it does but have never used it) to pull the data I want nightly, insert it into a db as one record, then have my pal box pull from that record? I would think that way there would be faster page load times. I'm open for suggestions here.
Thanks. For the moment it draws on their server each request. I have to learn more regarding cache. Originally I wanted to get the data and insert into my db, but I haven't gotten that far yet. The best approach would be, I think, to cache it. Any help in this area is appreciated.
I use to use some commercial scripts that would strip and cache the sites I wanted. These worked great until they stopped maintaining them. It was in perl and the code was garbled, they used regex and formatting to get them squared away. It is much nicer to see the open code so adjustments can be made. My php skill are beginner level still... was focused on learning the perl language but moving to php I think is a better option. I hope to help in the future as I can get up to speed... Will kep my eyes open for some php cache scripts we can use on this sort of thing. Might be something over at the php project site.
I know what you mean! After all, we do have fishing sites that need reports. I fished out of a kayak for the first time the other day. A challenging experience.
Seems to be fairly slow loads since I added the extra tides. Searched for a cache script we could use on it. Benedetto Patrice code posted on webdev
Code
// First create a folder named "cache" where the files will be written <br />// define the fonction <br />function BZ_cache ($url,$cacheName) <br />{ <br />global $cache ; <br />global $QUERY_STRING ; <br />// You may use the naming convention of you choice here <br />$filename = "cache/".$cacheName.",".$QUERY_STRING.".html" ; <br /><br />// You may use test the date of the cache here <br /> if ( file_exists ( $filename) ) <br /> { <br /> readfile ($filename) ; <br /> return 1 ; /// Ok i'v send the html page <br /> } <br />else <br /> { <br /> if ( ! isset ($cache ) ) <br /> { <br /> $fcontents = join ('', file ($url."?".$QUERY_STRING."&cache=t")); <br /> $fp = fopen ($filename , "w"); <br /> fwrite ($fp, $fcontents ); <br /> fclose ( $fp) ; <br /> return 0 ; /// i'v to execute the file <br /> } <br /> } <br />} <br />// then put this line at the TOP of you file ( before any include ) <br />// if ( BZ_cache ("COMPLETE URL WITHOUT PARAMS" , "AN_IDENTIFIER" ) ) exit () ; <br /><br /><br />// Example in the file http://azerclic.labynet.org/doc.php3 <br />// the first line is <br />// if ( BZ_cache ("http://azerclic.labynet.org/doc.php3" , "doc" ) ) exit () ; <br />// you will see the cached files in the folder http://azerclic.labynet.org/cache/ <br />// That's all <br />
Just did a cache with a cron job on a tide script puller. Had to do a rewrite with help from one of my members to add other tides for the pal.
I have the script pulling the data witha regex and saving it to a text file.... then I use a include on the text file. Set the cron up to run the script so it keeps the text file updated. Loads like a dream... Thanks for the inspiration to get me over the hump on this. Using this same style of a feature to fetch the latest news and regulations, river levels.... next is weather. Very close to having this setup for a mysql database, with cron for river levels. It will also have user login cookie so users can save there tides/river levels of choice.
BTW I used Fastcron.com for remote pull on the scripts since it was cheaper and very easy to setup. Cron job is the way to go!!
Not sure whether to include this with language files or not.. but probably will once its done.
Mine tide program crashed.. doesn't regex out the tide data from noaa properly so it returns a blank data field. Noaa must have changed something and my regex no longer works.
Mine is broke... not sure why it doesn't work but comes back with no tides. Anybody care to take a look see if it works on there server or if its just me.
Umm, that's only 9.95. Actually, you can see the tides in a calendar format here. The new year is coming up, so I gotta get it squared away in a couple of days. Not enough time in the day. sigh!
Um, yeah, I forgot. It's done. I've got it running out of a database, i.e. first you extract the data from NOAA, insert it into a database. Then your script pulls it from the database so it's tons faster. No need for cron because the tide predictions are done from certain 'harmonic constants' , so basically no matter when you go to NOAA and get tidal predictions, they are the same each time you go because they are using the same 'hc's' each time. So, you can get the tides and insert them into a db one time during the year, and just check them occasionally to ensure that the data matches what you have with what NOAA has.
You can see the working version of the 'Tide Sticker' at http://www.fish-tails.net in the left hand column. It's not in a pal bax because I'm currently not using IIP for my home page due to some weird problems. However, it's just a matter of inserting the code into a cusompalbox. You can see it working in a pal box at http://www.fish-tails.net/index3.php (the one that was giving me problems and suddenly is working again). That's IIP 5.3.1, but it shouldn't matter which IIP you're using since it's just basic php code added to a a pal box.
I also have a calendar running at http://www.fish-tails.net/threads/tidecal.php , which has a drop down box to grab the station for where ever you want and displays it in calendar format. It pulls from the same tables as the Tide Sticker. Though it is working flawlessly, there is some code cleanup I want to do for the calendar.
I know you want the Tide Sticker. Send me a PM with an email and I'll send you the scripts as soon as I do some cleanup. There is one file for getting the data and inserting into a table. The other script is for the Tide Sticker itself. Once you've installed them and you fel like it's worth something to you, you can let me know and I'll give you my paypal address.
If you want the calendar as well, that can be dones as well, but as I stated, I need to do some definite code cleanup for that. Let me know if you want that one as well, but I need more time to clean it up.
Please keep in mind that I have many Florida fishing sites that would love to have my site go belly up, but they want my tides calendar and sticker really bad. Plus, they're trying to figure out how I've got weather (integrated with threads) on my site as well (members only feature). So, with that being said, you must agree NOT to divulge any of the code for either the tides calendar or the sticker to anyone. I am hoping that one of the big players (no names but they have some big fishing publications out, as well as big fishing sites) might see my scripts valuable, which would equate to a big "Fish ON!" inangling terms.
Anyway, send me an email in the PM. I'll get it to you soon as I can.
Donate to UBBDev today to help aid in Operational, Server and Script Maintenance, and Development costs.
Please also see our parent organization VNC Web Services if you're in the need of a new UBB.threads Install or Upgrade, Site/Server Migrations, or Security and Coding Services.