Previous Thread
Next Thread
Print Thread
Rate Thread
Joined: Aug 2000
Posts: 3,590
Moderator
Moderator
Offline
Joined: Aug 2000
Posts: 3,590
UBB.classic backup and automation tutorial

INTRODUCTION

Every UBB.classic owner and/or administrator has tried at least once to backup all the content of the bulletin board on their own computer or an other server.

Of course, you can simply fire up your FTP client and download your cgi-bin, noncgi and members folders directly one file by one file. This solution, even thought it works well can take a couple of hours even for someone running on a xDSL line. The goal of this tutorial is to show you an alternative way of doing a backup which is much quicker and can easly be automated.

This tutorial works ONLY for users who run their server under a UNIX/LINUX environment.

You will need a Telnet/SSH account to your server in order to use the following commands but if you don't you should be able to run the full script using a CRON job (I will come to this later).

If you need a telnet/SSH client you can use PUTTY a free client available for all platforms.


PART 1 - Discovering TAR

So what is this magical trick I am going to use in order to backup my UBB.classic in a breeze? Well I am simply going to compress all the files together before downloading/transfer them.

As you know UBB.classic runs on flat files, each file being simple ASCII text contains many characters that take up space (spaces, line break etc.) by compacting everything you can fit a 50-100Mb UBB.classic into a files that is only a few megs.

Assuming you are already logged on your server go to your webdir like this

Code
[/code]replace public_html with the name of your dir, it can be www, httpdocs or something else.

then we are going to tar up everything (in this tutorial I use ubbcgi as the cgi-bin folder and ubb as the noncgi folder).
I also assume that the members folder is inside ubbcgi.

We will use the ubb directory as as example, each directory will compacted in its own tar file in order to make retrieval of information easier.

Code
</pre></div></div>so what does it mean? with <span style="font-weight: bold">c</span> I create a new file, <span style="font-weight: bold">f</span> means that what follows is the name of the file I want to create. Why do I use a dot slash before the 2 directories name? Well dot means the current directory so <span style="font-weight: bold">./ubbcgi</span> means that ubbcgi is right under the current directory.<br><br>lets see what we get:<br><br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
on my test server my backup tar file is 36 Mb which is still quite big we can do better by compacting the archive a second time thru gZIP.

lets start again:

Code
</pre></div></div>note that this time I added a <span style="font-weight: bold">z</span> to show that I want to compress the file through gzip and I gave my backup file a <span style="font-weight: bold">tar.gz</span> extension.<br><br>again, lets see what we get.<br>This time my file is only 7.2 Mb, good? We can still do better    <img src="<<GRAEMLIN_URL>>/smile.gif" alt="smile" title="smile" height="15" width="15" />   .<br><br><span style="font-weight: bold">PART 2 - Advanced TAR</span><br><br>The ubb directory contains the cache directory with all the frequently seen pages. these take up space too so the best solution would be to clear the cache before doing a backup but we can't do this since we want an automated solution and also because we don't want to rebuild the cache after each backup.<br><br>lets use then the <span style="font-style: italic">--exclude</span> command.<br><br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
without the cache I go down to 4.2 Mb (the 1234ABCD is the code of your cache it is unique for each UBB.classic)

but then without the cache folder I won't have all my statistics, no problem lets just add it.

Code
</pre></div></div>THERE! We now have our full forums backed up in a much smaller file and we didn't even have to keep the non necessary files.<br><br><span style="font-weight: bold">PART 3 - Sending the file by FTP</span><br><br>In this part we will see how to send the file by FTP using only one command line and no prompts, off course you could use the FTP command to do it but this way enables more in terms of making our automatic backup script later on.<br><br>For this I use a script called ncftp which is often precompiled on the servers, if you don't have it you can find it here:    <a href="http://www.ncftp.com/ncftp/"   target="_blank">http://www.ncftp.com/ncftp/</a>   <br><br>here is the command I will use:<br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
Please note that the remote path is based on where you arrive with the ftp account so for example if you start in /home/user and want your file to be in /home/user/backup then just enter backup as your remote folder

Even though this works we don't want to have the login and password in the command line because anyone who can see the process will be able to see them. So lets create a config file, remotebackup.conf for example with the following informations:

Code
</pre></div></div>we can then transfer our backup file using the following command:<br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
Remember to change permissions on the file so no one else can read it:
Code
</pre></div></div><span style="font-weight: bold">PART 4 - Creating the actual script</span><br><br>OK, so now we have all the parts we need for our script lets put them together to create our script. <br><br>This time we will add one tar line for each of our directories.<br><br>Here is a first example:<br><br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
you will need to transfer this file in ASCII mode and chmod it 755 in order to run it.
To run the file type:
Code
</pre></div></div><span style="font-weight: bold">PART 5 - Making it all automatic - CRON basis</span><br><br>To do this we will use the CRON function. The following is based on  <a href="http://www.itworld.com/Comp/2378/swol-0825-unix101/pfindex.html"   target="_blank">ITworld\'s tutorial</a>  on how to use cron.<br><br>A cron table file is made up of one line per entry. An entry consists of two categories of data: when to run a command and which command to run. <br><br>A line contains six fields, unless it begins with a hash mark (#), which is treated as a comment. The six fields, which must be separated by white space (tabs or spaces), are: <br><br><ol style="list-style-type:decimal;"><li>Minute of the hour in which to run (0-59)</li><li>Hour of the day in which to run (0-23)</li><li>Day of the month (0-31)</li><li>Month of the year in which to run (1-12)</li><li>Day of the week in which to run (0-6) (0=Sunday)</li><li>The command to execute</li></ol><br>The first five fields are the "when to run" while the last is the actual command that needs to be run. <br><br>An entry in the first five columns can consist of: <br><br><ul style="list-style-type:disc;"><li>A number in the specified range</li><li>A range of numbers in the specified range; for example, 2-10</li><li>A comma-separated list consisting of individual numbers or ranges of numbers, as in 1,2,3-7,8</li><li>An asterisk that stands for all valid values</li></ul><br>Note that lists and ranges of numbers must not contain spaces or tabs, which are reserved for separating fields. <br><br>A sample cron table file might be displayed with the crontab -l command. The following example includes line numbers to clarify the explanation. <br><br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
Lines 2 through 4 contain comments and are ignored. Line 6 runs the command some_command at 30 minutes past the hour. Note that the fields for hour, day, month, and weekday were all left with the asterisk; therefore some_command runs at 30 minutes past the hour, every hour of every day.

Line 7 runs the command another_command at 15 and 45 minutes past the hour for hours 1 through 3, namely, 1:15, 1:45, 2:15, 2:45, 3:15, and 3:45 a.m.

Line 8 specifies that sunday_job is to be run at 1:25 a.m., only on Sundays.

Line 9 runs monthly_report at 3:45 a.m. of the first day of each month.

Line 10 is a typical cron table entry error. The user wants to run a task daily at 3 p.m., but has only entered the hour. The asterisk in the minute column causes the job to run once every minute for each minute from 3:00 p.m. through 3:59 p.m.

Line 11 corrects that error and adds weekdays 1 through 5, limiting the job to 3:00 p.m., Monday through Friday.

PART 6 - Making it all automatic - using CRON

Now that we covered the basis of making a CRON file, lets use it to automate the calling of our backup script.

lets say we want to make our backup every day at 11 PM, our cron file will look something like that:

Code
</pre></div></div>Once you have decided when to run your backup just add your file to your crontab like this:<br><br><div class="ubbcode-block"><div class="ubbcode-header">Code</div><div class="ubbcode-body ubbcode-pre" ><pre>
Note: doing it this way will result in replacing any existing crontab with the one in the file if you are unsure you can first list the content of the crontab using
Code
</pre></div></div>you can then edit it directly using [code]
but adding extra informations to your file and then adding the file to cron is the easiest way to do it.

Note 2: some servers have some execution time limit for cron, so if your backup script worked when you called it from command line but did not from cron then contact your system administrator to allow for more execution time.

Sponsored Links

Link Copied to Clipboard
Donate Today!
Donate via PayPal

Donate to UBBDev today to help aid in Operational, Server and Script Maintenance, and Development costs.

Please also see our parent organization VNC Web Services if you're in the need of a new UBB.threads Install or Upgrade, Site/Server Migrations, or Security and Coding Services.
Recommended Hosts
We have personally worked with and recommend the following Web Hosts:
Stable Host
bluehost
InterServer
Visit us on Facebook
Member Spotlight
Gizmo
Gizmo
Portland, OR, USA
Posts: 5,833
Joined: January 2000
Forum Statistics
Forums63
Topics37,573
Posts293,925
Members13,849
Most Online5,166
Sep 15th, 2019
Today's Statistics
Currently Online
Topics Created
Posts Made
Users Online
Birthdays
Top Posters
AllenAyres 21,079
JoshPet 10,369
LK 7,394
Lord Dexter 6,708
Gizmo 5,833
Greg Hard 4,625
Top Posters(30 Days)
Top Likes Received
isaac 82
Gizmo 20
Brett 7
WebGuy 2
Morgan 2
Top Likes Received (30 Days)
None yet
The UBB.Developers Network (UBB.Dev/Threads.Dev) is ©2000-2024 VNC Web Services

 
Powered by UBB.threads™ PHP Forum Software 8.0.0
(Preview build 20221218)